How we help you automate 20% more of your tests
Are you “elite” yet? The 2021 state of devops report by Google suggests that now over a quarter of teams are. Here’s my takeaway: elite status is not only achievable, but could eventually constitute a norm. A “medium” quality team, formerly 37% of respondents, is now at 28%. “Low” has gone from 15% to 7%. Software teams are moving forward, and so should you.
To test effectively, you have to automate some of your tests. As a manual testing company we know that better than most – our crowdtesting solution is often used as a ramp to help facilitate an automation-first solution. I know this is counter-intuitive, so I wanted to explain how.
Automate more, faster
We’ve seen hundreds of businesses automate their tests, and there’s a few failure categories we can identify as part of that journey you should watch out for. The first is that many businesses simply misjudge the extent to which they will automate. A friend working at our integration partner TestRail once remarked that businesses say they’ll automate 20% more than they do. But when I asked what he meant, the numbers got a lot worse.
This was the TestRail annual survey, and of the 6,000 surveyed businesses, the mean automation rate was 42%, with 63% planned for the following year. The following year, the same question – and the same answers. 43% had been automated, 61% would be automated next year. The trend continued: most recently, 38% of tests were automated and 56% were planned for the following year.
It didn’t necessarily surprise me that businesses were going backwards some years. But it did surprise me that engineers and QA managers were continuing to get it wrong.
That’s because when my friend said 20%, he meant 20% of their entire test suite. Businesses believe they will automate +20% each year and do automate -1%: they automate 2100% less than they expect.
That scale of misjudgement is a problem. One of the reasons automated tests are so attractive is because they are a virtuous cycle, eliminating work to improve productivity with each round. Ran backwards, a virtuous cycle is a vicious cycle – and if you budget to have no setbacks you can end up with piling manual work. “Flaky tests” and “not enough time” were voted the number #01 and #02 reason attendees did not automate more of their suite in a joint webinar we ran with TestRail.
That’s why Global App Testing – as a flexible testing solution providing instant scale in the form of human testers – is a suitable bulwark against automation difficulties. Not only can we bolster the capabilities of in-house QA and QE teams by providing testing services which span the devices, countries, and physical environments which are hard to do in-house. But your automation queue – by nature of its vicious-circle status – should have its test cases executed by an external partner, so that you can focus on automating it.
The history of automation tells us that as some jobs get automated, other jobs become available. Put crudely, we think that the same is true of testing. More AI will lead to more software releases, and even as the proportion of testing which must be manual reduces, there will be more tests to execute there.
Curious about what % of your tests we can help you automate?