Enjoy The Silence

Posted on 4th May 2013

Unfortunately I've been having problems with my outbound mail from my personal mail server. I recently updated the server, and although the mail software was also updated, the problem seems to be regarding resolving IP addresses to the Virgin Media nameservers. I can resolve DNS lookups via my laptop, but the server gets stuck and times out :(

If I don't get it fixed soon, I'll have to look at using Gmail on a more permanent basis, so that I can at least reply to messages again and know they're going to get through. Although I don't mind using Gmail on occasions, storing all my mail with them does make me a little uncomfortable.

So if you haven't heard from me, and have been expecting a response, it's likely that I replied, but it never got through. By all means send me a reminder so I know to send a reply through Gmail to make sure you get the reply.

I've also been quiet on my blog for quite sometime. I have been rather busy with work, family, work, Perl, open source and work, and finding time to write here has been low on my list of things to do. However, I have a few posts coming soon to help redress the balance.

File Under: life
NO COMMENTS


Lost In The Echo

Posted on 26th August 2012

I've just released new versions of my use.perl distributions, WWW-UsePerl-Journal and WWW-UsePerl-Journal-Thread. As use.perl became decommisioned at the end of 2010, the distrubutions had been getting a lot of failure reports, as they used screen-scraping to get the content. As such, I had planned to put them out to pasture in BackPAN. That was until recently I discovered that Léon Brocard had not only released WWW-UsePerl-Server, but also provided a complete SQL archive of the use.perl database (see the POD for a link). Then combining the two, he put up a read-only version of the website.

While at YAPC::Europe this last week, I started tinkering, and fixing the URLs, regexes, logic and tests in my two distributions. Both distributions have had functionality removed, as the read-only site doesn't provide all the same features as the old dynamic site. The most obvious is that posting new journal entries is now disabled, but other lesser features not available are searching for comments based on thread id or users based on the user id. The majority of the main features are still there, and those that aren't I've used alternative methods to retrieve them where possible.

Although the distributions and modules are now working again, they're not perhaps as useful as they once were. As such, I will be looking to merge both distributions for a future release, and also providing support to a local database of the full archive from Léon.

Seeing as no-one else seems to have stepped forward and written similar modules for blogs.perl, I'm now thinking it might also be useful to take my use.perl modules and adapt them for blogs.perl. It might be a while before I finish them, but it'll be nice to have the ability to have many of the same features. I also note that blogs.perl.org also now has paging. Yeah \o/ :) This has been a feature that I have been wanting to see on the site since it started, so thanks to the guys for finding some tuits. There was a call at YAPC::Europe for people to help add even more functionality, so I look forward to seeing what delights we have in store next.

File Under: opensource / perl / website
NO COMMENTS


Every Day Is Exactly The Same

Posted on 15th July 2012

Recently Mark Keating of the Enlightened Perl Organisation created a new Google Calendar for Perl community events, particularly for Perl Monger group meetings. As I haven't been updating the other calendars I have access to for some time, it gave me the push needed to clean-up my script, and post all the forthcoming events to the calendars.

I've now updated the Birmingham.pm events page, to display the new calendar, as well as the West Midlands Tech Events calendar.

If you have access to any similar calendars, you can now update them with Perl (if you weren't already), with the aid of my helpful script. Feel free to use and abuse as you wish. Note that you will need to have a login to Google Calendars, and have access to the calendars you are submitting to.

File Under: perl
NO COMMENTS


Nomzamo

Posted on 19th May 2012

Earlier this month, on the 5th May 2012, The Paul Menel Band, stripped down to Paul and Steve, together with special guest Martin Orford, provide support to Pendragon at The Assembly in Leamington Spa. They may have been the supporting band, with an early time slot, but word had got out that doors were opening early, as nearly half the venue was full by the time the guys took to the stage. A great turn out, as even the Assembly staff noted.

Paul and Martin came on stage first for a rousing rendition of the Piano/Vocal Version of No Love Lost. A great start to the set, and got everybody eagerly awaiting the other delights from the Nomzamo 25 Years Anniversary Tour. This tour was a celebration of the release of Nomzamo, the album Paul and Martin first joined forces on, when they were in IQ back in 1987. 25 years later, the songs are as classic as they were back in 1987. Paul then welcomed Steve to join the two on stage. The Nomzamo set then continued with one of the best live performances of Promises (a personal favourite) and Nomzamo, the title track of the album.

To break the set up, the band added a few surprises throughout the set. The first surprise was a track from Martin's latest solo album, The Old Road, with Paul and Steve doing a fantastic job of Ray Of Hope. Common Ground began with Steve sitting on the floor with the acoustic guitar, having forgotten to arrange to have a stool on stage, and ended with some great electric guitar fret work, much to the appreciation of the audience. The next surprise, of sorts, was the Paul Menel Band song She's Up On The Chair Again, taken from the forthcoming album. Then on to the final song from the Nomzamo part of the set, Still Life, with yet more amazing guitar solos from Steve.

As so to the final song of the night. Although I knew ahead of the set what it would be, I hadn't seen the rehearsals, so wasn't quite expecting the result. As Steve stepped forward to the mic I was just expecting some backing vocals, and was just as surprised and impressed as everyone else when he took on the Roger Waters vocals for Comfortably Numb. Paul and Martin handled Dave Gilmour's vocals to great effect too. It was an unusual song for the set, but an absolute delight too.

Seeing Martin on stage again after so many years, was great, to see him playing and singing was fantastic, to see that he and Paul sounded as good as they did 25 years ago, if not better, was a joy. The following weekend they repeated the performance at De Boerderij, Zoetermeer, Holland, to just as much applause. Hopefully, this won't be the last time we see Paul and Martin on stage together, as there is already talk of an Are You Sitting Comfortably 25 Years Anniversary Tour in 2014 :)

If you missed the gig, fear not we recorded the whole event, and you can now watch the full set on YouTube. You can either watch via the playlist, or by clicking the links below to view the individual song performances:

Photos coming soon.

File Under: gigs / menel / music
NO COMMENTS


Young Parisians

Posted on 10th April 2012

Did I mention I went to Paris to take part in the 2012 QA Hackathon? Did I remember to mention all the cool stuff I got done? Well if you've been hiding for the past few weeks, have a look at the last couple of posts :)

As per usual, while there I took my camera along. However, unlike many previous visits to Paris, I didn't do any sight-seeing. And that includes failing to wander around the venue we were in and discovering the real submarine among other things, that others found while taking a breath of fresh air.

Instead I spent my time hacking away, and only occasionly coming up for air for food, drink and some camera action.

With over 40 people in attendance, it was going to be difficult to capture everyone, but I think I managed it. If I did miss you, my apologies. It was great to meet so many friends old and new, and a real pleasure to finally put faces to names that I've known for a while, but not had the opportunity to meet in person.

So many great things happened in Paris, and I'm really looking forward to see what we can achieve in London for the 2013 QA Hackathon. See you there.

File Under: community / hackathon / opensource / paris / photography / qa / testing
NO COMMENTS


Parisienne Walkways

Posted on 3rd April 2012

And so to the final part of my notes from the 2012 QA Hackathon.

CPAN Testers Report Status

After asking several times, Andreas thought he finally understood what the dates mean on the Status page for the CPAN Testers Reports. He started watching and making page requests to see whether his requests were actioned. On Day 3 he pointed out that the date went backwards! Once he'd shown me, I understand now why the first date is confusing. And for anyone else who has been confused by it, you can blame Amazon. SimpleDB sucks. It's why the Metabase is moving to another NoSQL DB.

The date references the update date of the report as it entered the Metabase. The last processed report is the last report that was extracted from the Metabase and entered into the cpanstats DB. Unfortunately, SimpleDB has a broken concept of searching. It will return results before the date requested, and regularly return the sorted results in an unsorted order. As such the dates you see on the Status page may go backwards in time! I'm not going to try and fix this, as it will all work as intended with the new system.

Missing Reports

There have been several questions relating to missing reports over the past few years. Sometimes it just needs me to refresh the indices, but in other cases it may be due to the fact that SimpleDB omits reports from a request. Did I mention SimpleDB sucks? In a request to the Metabase, I will ask for all the reports from a given date. The results are limited to 2500, due to Amazon's own restriction. In the returned list it will often omit entries, due to its ignorance of sorting in the search request. I have gone through the Metabase code on several occasions and can verify it does the right thing. SimpleDB just chooses to ignore the complete search request and returns what it *thinks* you want to know.

Ribasushi questioned me about one of his modules that had been released recently, which still had no Cygwin reports listed, even though he sent a few himself. Further investigation revealed that they are indeed missing from the cpanstats DB. Although they did enter the Metabase, they never came out again.

To resolve this I have been revisiting the Generator code to rework the reparse and regenerate code to enable search requests for missing periods, in the hope that this will retrieve most of the missing results. If it doesn't, then I will be asking David to produce a definitive list for me, and I will make specific requests for any missing reports. The Generator code has been updated in GitHub to include all the performance improvements that have been in live for some time too.

Erronously Parsed Reports

Every so often the parsing mechanism fails and stores the wrong data within the cpanstats DB. These days it seems to only affect the platform, OS version and OS name. I'm not quite sure what is happening, as reparsing the report locally again produces the correct results. This uses the same routine to parse the report, so why they occasional fail remains a mystery. However, to combat this, I  now have a script that can run and search periodicly for this erroneous data and attempt to reparse the results. It can then alert me when it can't fix it and I can investigate manually. The have been occasions where the report can't be parsed due to the output being corrupted on the test machine, which unfortunately we can't always resolve. Sometimes there are enough clues within other parts of the report that point to a particular OS, but sometimes we just have to leave it blank.

It seems in putting some of this code live before leaving the hackthon, I accidentally reintroduced a bug. Slaven was quick to spot it and tell me about it, but unfortunately it was too late for me to fix it, as I needed to leave and catch my flight home. It should be fixed by the time you read this though, so all should be back to your regular viewing pleasure :) With the new script I've written, it should hopefully find and fix these errors in the future, as well as alerting me to fix the bug again!

Thanks Again

So that was the 2012 QA Hackathon. The show ended with a group photo, although a few were missing due to their early departures home, but I think we got most of us in. Including Miyagawa, who was taking the picture. The traditional thanks yous and good byes ensued and then Andreas and I headed off to begin our adventure getting the airport! The next hackathon, the 2013 QA Hackathon, will be in London. We'll have the domain pointed to the right place just as soon as Andy gets the website up and running. I look forward to a lot more involvement for next year, as we have been steadily growing in numbers each year. There has already been some significant output, but the event is much more than that. It's a chance to take to people face to face, discuss ideas and plan for the future. Expect more news for CPAN Testers soon.

Once again I would like to thank ShadowCat Systems for getting me here, and for being a great supporter of the QA Hackthons, as well as many other Perl events over the years. Thanks too to Laurent Boivin (elbeho), Philippe Bruhat (BooK) and the French Perl Mongers for making the 2012 QA Hackathon happen. The Hackathon wouldn't have happened without the generosity of corporations and the communities that donate funds. So thank you to ... The City of Science and Industry, Diabolo.com, Dijkmat, DuckDuckGo, Dyn, Freeside Internet Services, Hedera Technology, Jaguar Network, Mongueurs de Perl, Shadowcat Systems Limited, SPLIO, TECLIB", Weborama, and $foo Magazine. We also have several individuals to thank too, who all made personal contributions to the event, so many thanks to Martin Evans, Mark Keating, Prakash Kailasa, Neil Bowers, 加藤 敦 (Ktat), Karen Pauley, Chad Davis, Franck Cuny, 近藤嘉雪, Tomohiro Hosaka, Syohei Yoshida, 牧 大輔 (lestrrat), and Laurent Boivin

Meanwhile, Dan & Ethne would also like to thank Booking.com for their silly putty ;)

File Under: hackathon / opensource / paris / perl / qa / testing
NO COMMENTS


Party In Paris

Posted on 31st March 2012

I'm currently at the 2012 QA Hackathon working on CPAN Testers servers, sites, databases and code. It has already been very productive, and already I have two new module releases.

CPAN::Testers::WWW::Reports::Query::AJAX

This module was originally written in response to a question by Leo Lapworth about how the summary information is produced. As a consequence he wrote CPAN::Testers::WWW::Reports::Query::JSON, which takes the data from the stored JSON file. In most cases this data is sufficient, but the module requires parsing the JSON file which may be slow for distributions with a large number of reports. On the CPAN Testers Reports site, in the side panel on the distribution page, you will see the temperature graphs measuring the percentage of PASS, FAIL, NA and UNKNOWN reports a particular release has. This is glean from an AJAX call to the server.

But what if you don't want an HTML/Javascript styled response? What if you wanted the results in plain test or XML? Enter CPAN::Testers::WWW::Reports::Query::AJAX. Now you can use this to query the live data to for a particular distribution, and optionally a specific version, all the result values and get them pack as a simple hash to do with as you please.

I anticipate this might be most useful to project website who wish to display their latest results from CPAN Testers in some way. They can now get the data, and present it however they wish.

CPAN::Testers::WWW::Reports::Query::Reports

Now we get to perhaps the bigger module, even though its smaller than the one above. This module is perhaps most useful to all those who are trying to maintain a version of the cpanstats metadata from the SQLite database. As mentioned previously the SQLite database has been giving us grief over the past year, and we haven't gotten to the bottom of it. Andreas suspects there is some unusual textual data in some reports that is causing SQLite problems when it tries to store it. I'm not quite convinced by this, but as I'm only inserting records, I'm at a lost as to what else be the cause.

The SQLite file now clocks in at over 1GB compressed and over 8GB uncompressed, and is starting to take a notable amount of disk space (though considerably smaller than the 250GB+ Metabase database ;) ). It is also a significant bandwidth consumer each day, which can slow processing and page displays, as disk access is our limiting factor now.

Enter CPAN::Testers::WWW::Reports::Query::Reports. This module uses the same principles as the AJAX module above, but now accesses an new API on the CPAN Testers Reports site to enable consumers to get either a specific record or a whole range of report metadata records. Currently the maximum number of records that can be return in a single request is 2500, but this may be increased once the system has been proven to work well. Typically we have around 30,000 reports submitted each day, so to allow consumers to make best use of this API, I will look to increasing the limit to maybe 50,000 or 100,000. I want to impose a limit as I don't want accidental requests being sent to consume the full database in one go, as again this would put a strain on disk access.

The aim of the module is to allow those that currently consume the SQLite database, to more regularly request smaller updates and store the results in any database they so choose. Even into a NoSQL style database. It will ultimately reduce the bandwidth, data stored and processing to gzip and bzip2, which then means we can reallocate effort to more useful tasks.

If you currently consume the SQLite database, please take a look at this module and see how you can use it. I plan to include some example scripts that could be drop-in replacements for your current processes, but if you get there first, please feel free to submit them to me too, and I will include them with full credit. If you spot any issues or improvements, please also let me know.

CPAN Testers Platform Metabase Facts

This morning we had a CPAN Testers presentation and discussion hosted by David Golden. As there is plenty of interest from a variety of parties about CPAN Testers, it was a good opportunity to highlight an area that needs work, but which David and myself, as well as other key developers in the CPAN Tester community, just don't have time to do. Breno de Oliveira (garu or IRC) has very kindly stepped forward to look at one particular task, which we have been wanting to write since the QA Hackathon in Birmingham, back in 2009!

Breno has written a CPAN Testers client for cpanminus. At the moment its a stand-alone application, but it may well be included within cpanminus in the future. As part of writing the application, Breno asked David and myself about how the clients for CPAN::Reporter and CPANPLUS::YACSmoke create the report. Due to the legacy system we came from (email and NNTP) we still use an email style presentation of the reports. However, it has always been our intention to produce structured data. A CPAN Testers Report currently has only two facts that are required, a Legacy Report and a Test Summary. However there are other facts that we have already scoped, except they are just not implemented.

Back last year the Birmingham Perl Mongers produced the CPAN::Testers::Fact::PlatformInfo fact, that consumes the data from Devel::Platform::Info (which we'd written the previous year). The problem with the way test reports are currently created, is that we don't always know the definite platform information for the platform the test suite was run on. Reports, particularly in the Perl Config section, can lie. Not big lies necessarily, but enough that it can disguise why a particular OS may have problems with a particular distribution.

Breno is now looking to produce a module that firstly abstracts all the metadata creation parts from CPAN::Reporter, CPANPLUS::YACsmoke, Test::Reporter as well as his own new application, and puts them into a single library that can then create all the appropriate facts before submitting the report to the metabase. Hopefully he can get this done during the Hackathon, but even if he doesn't, we're hopful that he will get enough done to make it easy to complete soon after. Once we then patch the respective clients to use the new library, we will then start to be able to do interesting things with how we present reports.

The CPAN Testers Reports site only displays the legacy style report, which for most is sufficient, but it really would be nice to have some specially styled presentations for particular sections, or even allow user preferences to show/hide sections automatically when a user reads a report.

CPAN Testers Admin site

This is a site that I have been working on, on and off, for about 4 years, before we even had a Metabase. As a consequence it has been promised at various points and I've always failed to deliver. Now I have release the modules above, and there have been several comments already about having such functionality, I think I need to put some focus on it again. I have shown Breno the site running on my laptop and he has given me some more ideas to make it even more useful. It'll still be awhile before its released, but this will likely be down to running with some beta testers first before a major launch, just so it doesn't break the eco-system too badly!

Essentially the site was written to help authors and testers to highlight dubious reports and have them deleted from the system. Although the reports won't actually be deleted, they will be marked to ignore, so that they can be removed from JSON files and summary requests, as well as on the CPAN Testers Report site. This will hopefully enable us to get more accurate data, and bogus reports about running out of memory or disk space can be disregarded.

However, following Breno suggestions, I will look to making the site more public, so that authors can more easily see the reporting patterns without having to log in. The log in aspect will still be needed to flag reports, but the alternate browsing of reports by testers will be much more accessible.

Thanks

I would like to thank a few people who have helped to get me here, and have enabled these QA projects, not just CPAN Testers, to advance further.

Firstly I would like to single out ShadowCat Systems, who have very kindly paid for my flight here. Thanks to BooK and Laurent for organising the event, and to all the sponsors and Perl community who have provided the funding for the venue, accommodation and food for the event. It has already been very much appreciated, and hopefully the significant submissions to GitHub and PAUSE are evidence of just how worthwhile this event is.

Thanks also to all those who are here, and are helping out in all shapes and forms to help Perl QA be even better than it already is.

File Under: community / hackathon / opensource / paris / qa / testing
NO COMMENTS


Things Don't Mean What They Used To

Posted on 26th March 2012

Back in the '90s a young band touted themselves round the Midlands and sometimes further afield. Unfortunately I never got to see them, but did acquire a cassette tape at some point and remember being impressed. I was later told that the keyboard player was my mate Pete Spoz, who I'd got to know as one of the Jellyheads, who used to come along to many of the early Ark gigs. The tape is now somewhere in the loft in one of many boxes og hidden gems.

A few months ago, Pete told me that Giovanni (Pete's brother) was reforming The Sordid Details with Mick Couch, Rick Cox and himself. Here was my chance.

Sadly due to other commitments, of the three Re-Onion dates, Bromsgrove is the only one I could make. So on the 18th March, over to The Hop Pole in Bromsgrove I went for my first Sordid Details gig. Full of friends and family, the night was set for a top gig.

The band began as The Sordid Details, with a great selection of their own classics, along with a couple of covers, the first of which, Wish Away, they were joined by their good friend Ash on vocals. After about an hour they closed the first set with The Stranglers classic No More Heroes.

The second set began with a selection of songs from their days as Stereogram, when they played as a three-piece after Pete left. Pete rejoined them on stage for a run through a selection of covers, and a few more of The Sordid Details classics, before Dave from Jam DRC peformed some guest vocals with the band.

The night ended all too quickly, even though the band had been playing for over two hours! A fantastic night and I'm only sad that I won't be able to see them play The Flapper & Firkin in Birmingham on Saturday 31st March. If you happened to be in town on Saturday and fancy a great night out, you'll not be disappointed with The Sordid Details. Hopefully they'll come out retirement again some day, so I see if my bootleg tape recorder still works!

File Under: bromsgrove / gigs / music / photography
NO COMMENTS


The Reasons Why

Posted on 25th March 2012

For those that follow the conference surveys, you'll be pleased to hear that I have now put the results of both the Israeli Perl Workshop and the German Perl Workshop online. These are the first events this year to take advantage of the surveys, although several more are to come.

This marks the second survey for the German Perl Workshop and notes some small differences, while it was the first for the Israeli Perl Workshop. I hope the future organisers can make use of the results and that they allow me to continue the surveys with these workshops next year, and for the years to come.

Although the Israeli Perl Workshop was in English this year, Gabor and I are hoping to be able to provide the survey in Hebrew next year. The German Perl Workshop marked the first survey not in English last year, and it helped to start building up a language pack, which can be used to plugin to the survey software. I plan to formalise this during the year, so that other events, using languages other than English, can still take advantage of the surveys.

Thanks to all the organisers and the survey participants for taking the time to respond to the questions. It is very much appreciated.

File Under: conference / opensource / perl / survey / workshop
1 COMMENT


Breathe

Posted on 15th February 2012

Cold Air

I am just air
A breeze of coldness
Hold me if you dare
I have travelled far, I am breathless

This a poem written by Ethne, aged 7.

Considering she had no help with the words, it's quite a profound first poem. She wrote it on her MagnaDoodle, so Nicole managed to preserve the effort before it got erased :)

Well done Ethne.

File Under: ethne / family / people / poetry
NO COMMENTS


<< Page 2 Page 4 >>

Some Rights Reserved Unless otherwise expressly stated, all original material of whatever nature created by Barbie and included in the Memories Of A Roadie website and any related pages, including the website's archives, is licensed under a Creative Commons by Attribution Non-Commercial License. If you wish to use material for commercial puposes, please contact me for further assistance regarding commercial licensing.