A complete guide to exploratory testing in Agile
Exploratory testing in Agile is a testing technique used in software testing to increase robustness and find issues that test cases miss. This is an informational page – for our commercial page on exploratory testing, look here.
Exploratory testing is an agile software testing technique in which a number of software testers look for bugs in an application without a script.
Typically, testers would use domain knowledge and their creativity to explore software seeking defects, vulnerabilities, and/or usability issues. Exploratory testing enhances test coverage and enhances the level of robustness of a QA approach.
Exploratory testing and test cases are opposite approaches.
Test cases are pre-written, detailed instructions that specify expected inputs, actions, and the expected outputs of those actions. Test cases ask testers to follow exact sequences and report bugs. In contrast, exploratory testing asks testers without a script.
For that reason, exploratory testing and test cases are complementary testing approaches. Test cases give you a full overview of core user flows and a methodological sweep of the entire application. Exploratory tests identify bugs which reside in common reservoirs of issues and account better for different kinds of users. We recommend both.
For the most part, exploratory testing can be done in a timeframe which would support an agile sprint. Exploratory testing with external testers can also be performed within staging environments if the necessary permissions are given.
In general, the benefit of exploratory tests is that you can find more bugs and improve your user experience to a greater extent than by looking at test cases below. (An additional benefit is that you don’t have to write test cases to get started.) But we’ve broken down why this is below.
An “unknown unknown”, according to the famous syllogism, is “something that you don’t know that you don’t know” – so a bug which isn’t in an area you would think to write a test case to check for. By the fact of writing and reusing your test cases, you are continuing to investigate the same parts of the product for bugs over and over again… and you would be surprised at the size of bugs we’ve seen creep through despite repeated test case use.
An “edge case” refers to the scenario or set of input values at the extreme values or limits of the expected input range of a particular function. Exploratory testing draws out more rigorous adversarial behaviour to the software in order to find issues. To give the example of a surname box in a form – test cases may cover the intended use, but experimenting with unusual values might cause the form to break.
If users behaved in the way we expected, this would be a much easier industry. It’s very difficult to build test cases which accurately reflect real patterns of user behaviour. To continue with the example of a signup form, say for example page three of the form asks the reader to expand on something they said on page one, this might trigger a lot of users to “double back” click. This is the kind of behaviour which QAs tend not to anticipate in test cases.
Our testers are professionals. They test software for a living, and they do so for the largest software applications in the world. By removing the script, you invite them to use their expertise to help you identify bugs in areas which their expertise suggests bugs would be.
Given that exploratory testing in agile does not require any test cases to be written, you can get started straight away.
The overall results of the above are more robust approaches to testing leading to better software quality. The advantage of better software quality is difficult to describe because it’s commercial and depends somewhat on the
Why do you want a better user experience? Theoretically, we know that all of the decisions relate to the commercial incentives of a given industry – that the user acquisition, retention, and engagement, relate to software quality in a direct way and that decisions are made with those goals in mind. But in a broader sense, most of the product professionals we know want to make great software and deliver amazing experiences for their users. Don’t you?
The cost of a bug in production to fix is several times higher than the cost of a bug caught in a staging environment. This is part of the reason we advocate for exploratory testing in agile – incorporating exploratory testing into your release cycle will drive down your QA costs over time, because the costs of catching bugs after they’ve been released to production are significant.
Exploratory testing on a live product can be a great way to touch base with your experience and troubleshoot the real experience your users are having.
Global App Testing is a QA service which incorporates techniques of exploratory testing suitable for agile sprints in addition to traditional test cases. We boast over 90,000 testers available in 190+ countries and territories, with 60,000 different devices. GAT also enables you to select your exploratory testing service in one of the following ways:
Double check that your software is free from functional defects with a traditional exploratory test focused on functional outcomes. All global app tests can be undertaken in less than 48 hours (many are faster “express tests” which are 6 hours) so you can get your software functionally tested to fit within an agile sprint.
Provide Global App Testing with both the implemented software in a staging environment (or a live production environment) and a copy of the design and our testers can look for discrepancies left to be fixed. Find everything from look-and-feel discrepancies, to functional discrepancies, with this service.
Test your software in a range of environments, including devices, OS versions, OS, or different kinds of web browsers, to ensure that your software still works great in every environment with compatibility testing.
Identify potential barriers to user success in less than 48 hours with a usability test by software professionals using the exploratory testing methodology.
If you’re interested in hearing more about testing with GAT, you can chat with us here about your QA and everything our services could do for you.