We sat there in complete disbelief. Here we were at a mega event watching a large brand deliver the end of day keynote. The speaker got on stage and took to his speech. Pop. He started to speak and got through a few more sentences before the POP. Again. Once that second and third pop hit, the event production team got into high gear with their agile testing. But could they fix it in time...
The event was a great event up until that point. From the deep blue and purple hues of the lighting to great social streaming TV's they had in the centre, it was clear the event was polished. In fact, it can probably be assumed that they tested as many things as possible.
Unfortunately, for those of us in that room that afternoon, the reality was that there was a bug in that audio gear and equipment. Most specifically, that bug had a massive impact for both the keynote speaker himself and also the audience. Of course, the immediate reaction from the tech team onsite was to begin applying hotfixes as quickly as possible.
One immediate hotfix they applied was to switch out the headset (a pretty obvious initial first swipe). For a brief moment, the tech team felt a sigh of relief as the keynote speaker was able to continue his remarks. Until…
Pop. It happened again but this time, with more power. It's quite possible that this pop wasn't actually that much louder but the audience was definitely feeling the impact of this one. Perhaps psychologically we had been lulled into a false sense of security. As the keynote continued, more fixes from the developers were deployed with little success. Because they could not reboot the whole system, they were never able to find the bug at the heart of the issue.
If this all sounds familiar to you, that's because it mirrors what we often do for software testing and bug fixing. The traditional testing method involves testing something once before production and then certifying it for release. In the case of the event I attended, the audio team had most likely done a test before the event began. However, it wasn't until much later that bugs in their systems developed.
In much the same way, if we focus on a testing practice that only tests a product or service when it is ready for production, we can end up in similar hot water. Your testing practice should embrace or mirror an agile software development approach. That means that your team members are testing frequently and in short bursts. Let's examine how this might have improved the keynote speech mentioned above.
Had audio/visual team at the event been an agile team, they may have tested the audio setup between different speakers. There were breaks between each of the speakers and this could have afforded a quick round of acceptance testing that should have caught this bug. Because they only tested the one time it led to issues later on that damaged the event.
Think about the customers you serve. They expect and demand that your products and services work under all kinds of conditions. When you begin to think how your customer thinks about your product you start to understand how it affects testing. It should become clear that holistically your agile testing approach needs to incorporate automated testing, regression testing, unit testing, manual testing, test cases and potentially many types of testing methodologies. As a product owner, member of the development team or test team member, this can be very overwhelming. But, it doesn't have to be.
I did feel bad for the team who had put so much effort into the event. For the technicians who were trying to apply hotfixes though, it was too little too late. The product was already out in the hands (or eyes) of the public. Had they approached the event with an agile testing mind set, they could have avoided catastrophe and a negative on an otherwise great experience.
When organisations have an agile development approach to product development, testing should happen at the same point in time. Thus, feedback can occur all throughout the process and help to improve the overall result. Integrating this type of feedback helps to reduce the time developers have to spend once the product is released triaging and fixing bugs. Here's why.
Developers who have to context switch or go back to a previous version of their software to fix a bug lose large amounts of mental focus and performance. According to the American Psychological Association (and this great blog from Trello), "it takes an average of 25 minutes to resume a task after being interrupted". So, if we can get the developers the feedback they need when they need it, we can reduce the overhead of context switching. And this is really where agile testing shines. Because it is less about the specific type of testing and more closely linked to the process it means we are free to come up with own approach.
Many of the high performing teams we work with have figured out that the faster they can test and get feedback into the hands of developers, the faster they can release with higher quality. In order to accomplish this, you need to have teams of testers who are able to jump on requests and deliver bug reports developers can use. There are a few ways to accomplish this:
1. You can hire 15-20 of the best QA folks in your area or geolocated around the world. But if you do it this way you're going to have to deal with hiring costs, employee overhead and finding the headcount in your budget. Not an easy task.
2. Or, you can use a crowd of QA professionals to handle your agile testing needs. A crowd of QA professionals from Global App Testing doesn't require headcount, won't increase employee overhead and greatly scales your ability to find critical bugs as fast as possible.
For lots of companies, agile testing can seem like a daunting task when even basic QA isn't happening or where your company isn't nimble enough to adapt. Just remember to take a step back and realise that the core ethos of agile testing centres around quick, early and professional testing. If you think about it from that perspective then your decision-making process will be guided in the right direction.
To find out more about agile testing or how we refer to the future of QA as QAOps, get in contact with us today.