Release Post Mortum / Ireland / Labor

The Release

As most Flex users know, on Thursday night we released version 4.4.29 of Flex, skipping over four version numbers to get there.  In our last post we talked about how our release automation tools make it tricky to retag release candidates as GA releases, so we just burn minor revision numbers when release candidates get rejected.

In this case QA rejected the release candidate not once, but four times, initially for fairly serious reasons and later on for nit picky glitches that most users wouldn’t notice.  We did the rework anyway and late Tuesday night QA cleared 4.4.29 for release.  We’ve instituted a policy that requires us to send pre-notifications for a release at least 24 hours in advance (except in the case of emergency releases), so the pre-notify went out on Wednesday morning and the release was pushed to everyone Thursday night.

The morning after a release always makes us a little nervous and our fear was that all these policy changes and extra time for UI regression testing wouldn’t make any difference — we’d end up with a mess to clean up the following morning all the same.  Luckily that’s not what happened.  Friday morning was very quiet.  We had one customer notice that sorting for the container builder screen had been left out when the component was redesigned for storage containers, but other than that issue the only errors we saw come in were related to serialization changes — typical stuff related to having a stale version of the Flash client.  Hopefully the recent validation logic we’ve added to warn the user when using a stale version of the client gave everyone the necessary heads up.

All in all, a very quiet morning after for us.  We all know the definition of insanity is doing the same thing and expecting different results.  But the dark underbelly of this maxim and our worry going into this last release was that doing something different would still yield the same results.  Score one for a more formal release process.  Hopefully the QA success on this last release is repeatable going forward.

Flex Europa

Most Flex systems, regardless of where the customers are located, are hosted in cloud data centers located in Northern California and along the Columbia River in Boardman, Oregon.

Recently we’ve built up a reasonably large number of customers in Europe and it seemed to us like there might be enough critical mass to justify a dedicated server for European customers.  The hope was such a move would increase response times for European customers, but there was no way to know this for sure.  Moving the server closer seemed like a logical way to speed things up, but since there are so many factors that contribute to response times, there was no way to know but to give it a try.

We added a new cloud instance located in Dublin, Ireland and moved all Flex customers located in Europe and Africa to this instance.  One of our customers, Gareth Jeanne from Stage Sound Services in Cardiff, was kind enough to send us this screen shot from his browser’s performance monitoring plugin:

The interesting bit here is from the graph at the top, where it shows that the response times, at least for this user, are more than twice as fast as they were when the data had to cross the Atlantic.  Very encouraging data.  Our user base in Australia, New Zealand and the Pacific Rim is growing.  A little more growth there and we might consider adding a server in Singapore.

Labor Party

Chris, Devon and I had a great design session this morning where we sketched out the initial functional requirements for the much anticipated labor module.  I’m hesitant to discuss new features while they’re still speculative, but suffice it to say that we’ll be rolling out some labor management tools that will be the first of their kind in the industry.

I’ll be taking our requirements doc and doing technical design today, with scheduling and implementation starting later this week.

Leave a Comment