Why is mobile testing difficult?

Estimates suggest that there are over six billion smartphone users worldwide. With that prevalence, it should come as no surprise that over half of all web traffic has been from a mobile device since 2017. However, the increase in mobile use has brought new challenges for web and software developers.

Device diversity

There are a lot of different mobile devices with which users might interact with your application. To make matters worse, even seemingly identical devices can behave differently when it comes to real-world app performance. If you wanted to manually test for all the possible end-use cases, you would need a stockpile of thousands of devices and a whole engineering team of testers working around the clock. While crowdtesting can get you more eyes on your product than ever before, it still won’t be able to cover every single configuration.

Keeping pace with updates

To complicate matters further, most phone and tablet users expect their apps to keep pace with updates to their mobile devices. With both Android and iOS rolling out continuous patches and updates to their operating systems, you’ll need to do the same.

For instance, if you’re targeting a market that’s very different from your existing ones, the best course of action might be to rebuild your GTM strategy from scratch. Alternatively, if you’re targeting a market that’s quite similar, you may only need to translate your content and make minor alterations to your sales strategy. Sometimes, it can be even easier than that - for instance, if you’re a US company targeting the UK market, you might only need to find a local banking partner and start accepting orders.

Most common mobile testing challenges

Testing a mobile application comes with a whole host of unique challenges. Many of these challenges come down to what gets referred to as fragmentation — the sheer diversity and variety of actual mobile devices and configurations in use today.

While every mobile development lifecycle will inevitably encounter its own difficulties, most of them can be classified as belonging to one of the following mobile testing challenges:

OS fragmentation

When it comes to mobile operating systems, Apple and Google split the lion’s share of the market. As of June 2024, devices running either Apple’s iOS or Google’s Android system accounted for over 99% of all smart mobile devices, with Android alone making up 72% of the market for mobile operating systems.

As a result of the Android-iOS duopoly, many app developers have chosen to abandon support for other platforms and focus on the two systems that cover the vast majority of mobile usage.

Interoperability

While for some apps it might be enough to simply build two versions – one for Android and one for iOS – if your app has a social element to it, or your users expect it to integrate across multiple devices, you will need to consider how the two versions will communicate and interact. If you plan on building an app that can function outside of Android and iOS, interoperability becomes even more of a challenge.

Screen sizes

Smartphone manufacturers design their devices with varying screen specifications to appeal to a broader range of consumers. These days, even a single mobile device model can come in multiple variants, all with different resolutions and screen sizes. To maintain consistency across a varied landscape of screen size and resolution, app and web developers need to ensure their products resize correctly for all these variations.

OS versions

Developers must consider fragmentation within both Android and iOS operating systems when designing. At any one time, the distribution of different OS versions across devices is in constant flux. With this in mind, it isn’t enough to simply test for the most recent version of iOS.

To cover the majority of use cases, you need to test how your application will perform under the different OS versions with which it will be used. As newer versions of each OS introduce new features, you may want to take advantage of these in your app design. But when it comes to testing, you will also want to ensure that new features don’t inhibit your app’s backward compatibility with older versions.

Ecosystem integration

Whether you are building a mobile web app, a native app to be downloaded from one of the popular app stores, or some kind of hybrid application, modern mobile users expect their apps to integrate seamlessly into the wider application ecosystem. From simple integrations like building a “log-in using Facebook/Google” option into your mobile website to more complex ones that require communication between multiple apps from different developers, ensuring your integrations work properly requires a rigorous testing process.

Different browsers

If OS fragmentation is the primary concern of native app developers, different browsers are more likely to be a challenge for mobile web developers. While there have been attempts to standardize the way browsers render web pages, the success of attempts to implement standards has been limited. This is partly because the popular browsers all use different rendering engines.

Safari and Opera are built on WebKit, but Chrome uses Blink, Firefox uses Gecko, and so on. On iOS, all browsers use WebKit due to OS policy restrictions. These different approaches to graphical rendering affect what your web pages look like on the client side.

Device settings

Because mobile device settings vary according to OS version and device hardware, as well as user preference, it can present a huge challenge for testers. As you implement the final stages of your testing cycle, you will want to keep in mind that any or all of a wide variety of device settings can be changed at any time when your application is in use. Font size, night mode, and accessibility settings can all impact the way your app works.

Language and cultural specifics

For developers marketing their application to an international audience, making sure that different language versions all function smoothly without improper translation can be hard work, especially when it comes to languages not spoken by your core development team. Mistranslations can occur in cultural contexts as well as linguistic ones. Besides different alphabets and characters to think about, there’s also different numerical systems, currencies, date, time, and measurement standards to consider.

To compete in local markets, mobile developers need to implement what is known as localization testing. This type of testing checks how a given mobile application is received on the ground in different geographic locales.

Location awareness

Another reason for the importance of localization testing is that a growing number of apps are dependent on geo-location data to function properly. For example, they might use location-dependent algorithms to provide real-time information about the user's surroundings or to share information between app users in the same area. The challenge of location-dependence means app developers need to negotiate both:

• Infrastructural differences between locations (like divergent network coverage) and 
• Legal ones (like different privacy laws).

Usability

In the field of mobile app testing, the end-user experience of a product is the ultimate measure of whether a testing and quality assurance program has worked. Ensuring usability on real devices is really a meta-challenge that encompasses many of the other difficulties specific to mobile application testing. While usability testing may be a discipline in its own right, the various stages of exploratory and functional testing are also ultimately in place to create an enjoyable experience for the end-user.

Common bugs found during the mobile testing process

In mobile development, a “bug” refers to any error, flaw, or fault in an application that causes it to produce an undesired outcome, or to behave in unintended ways. The process of finding and fixing bugs is known as "debugging" and is a central pillar of all software testing.

While this broad definition of a bug means that the list of things that could count as a bug is impossible to ever fully express, there are certain bugs that test engineers from across the spectrum of mobile development tend to encounter.

High battery usage

When using a mobile app, few things are more frustrating than battery drain. High battery usage can be caused by any one of the critical or non-critical paths in an application. The most efficient apps optimize battery usage for each function and ensure functions don’t run concurrently if they don’t need to.

High memory usage

Another instance of poor resource allocation, mobile testing can indicate when an application is using excessive amounts of ROM or RAM. One common cause of high memory usage is an excessive memory churn rate. This happens when the number of allocations of temporary objects an application makes in a given amount of time is too high. Picking up on excessive memory usage is important because it’s one of the biggest contributing factors to people uninstalling an app.

Button and click frustration

In both native and web apps, unresponsive or poorly designed buttons can be frustrating to users. There are many things that can cause buttons not to work properly. Sometimes the issue is related to different screen sizes, other causes include unexpected interactions between graphical components and having the wrong hierarchy of design elements.

The click-crash

One major bug that can invalidate an otherwise great application is when a specific button causes the app to crash when clicked. This problem can often go overlooked if the button that’s causing the issue is one that rarely gets used, is hidden deep within the app, or only causes the crash when a particular sequence of variables has occurred.

Slow transitions

Abnormally slow transitions between different app pages and tabs are some of the most common bugs that mobile testers need to look out for. Even a delay of a few seconds can introduce friction in the user experience and cause frustration.

Display problems

Mobile users expect their apps to smoothly rotate between portrait, landscape, and auto-rotate options. Other display issues include:

• Distorted dimensions, 
• Unexplained whitespace, 
• Improper rendering, and 
• Resizing issues. 

While software automation tools exist that mean you don’t have to check all possible device configurations manually, there are still numerous configuration options you will need to build into your tests.

Notifications

Mobile users expect applications to notify them of new events, messages, and updates. And yet, bugs in the notification process are common. Notifications that are too frequent, show up at the wrong time, or are just plain broken, can lead to user frustration.

Communication problems

Not all the data on a mobile app is stored on the device itself. Some data is only accessible through API calls to remote servers. The API call process can be interrupted by server outages, traffic issues, and lack of coverage. Bugs in the code handling API calls can introduce friction in the process and cause the app to stall or crash.

Mobile testing strategies

There are many ways to implement a mobile testing program, and no one-size-fits-all strategy will work for every product. That said, there are a number of strategies that all developers should keep in mind as they seek to build the best test program for their specific requirements.

Plan in advance

The first stage of any test process is planning. From a thorough understanding of the test goals, you can develop a strategy that will allow you to maximize test coverage with the available resources.

Having a plan means you can determine what types of testing you will need to perform. The best plans include different types of tests, including:

• Functional testing, 
• Exploratory testing, 
• Compatibility testing, 
• Usability testing, 
• Performance testing and 
• Security testing.
  

Set priorities

While the planning phase should ideally indicate which devices, OS versions, and configurations you need to test for, the sheer volume of possible configurations means that it’s unlikely you will be able to test for all of them. During the planning stage, make sure to note which test cases and configuration options are the highest priority and begin with those. Often, this will mean starting with the device configurations that are most commonly used.

Begin with the basics

Even a brilliant mobile app with lots of advanced functionality will become frustrating to users if basic functions don’t work correctly. Before testing for complex scenarios, it’s usually a good idea to start with simple, everyday use cases. Does your app load quickly? Do all your links work? Do text boxes and forms function properly? These basic tests might seem unglamorous, but if you skip them, your app might not pass the more advanced tests anyway.

Use automation where possible

Manual testing is essential for many aspects of the testing lifecycle, but automated testing has an important role to play too. Automation testing is not suitable for every part of the testing process, but it can speed up and improve tests for some kinds of repetitive tasks.

Automated tools can help you build your basic functionality tests, including checking how your app performs on different OS versions and device configurations. They can also be used for performance testing and to check if your app runs smoothly in an environment with low or intermittent connectivity.

Do security testing

Security testing should be an integral part of your test strategy. Common security tests will include the following:

• Checking that users are logged out of secure sessions, 
• That your app stores and processes sensitive data correctly, and 
• That user authentication mechanisms are robust enough to handle the threat of brute force attacks.

Test early and often

The earlier in the development cycle you begin testing, the easier it will be to find and fix bugs. Once a product is released, however, the work of testing is far from over. In order to keep pace with OS patches, new versions, and new device releases, you will need to keep testing as you go.

Take advantage of crowdtesting

Crowdtesting, where a large number of external testers provide feedback on your product, is especially useful for mobile testing because it can help you get a wider range of user feedback and device configuration coverage than you would with internal testers alone. At GAT, we have more than 90,000 testers in over 190 countries worldwide.

Mobile testing in the future

The challenges of mobile testing today are unlikely to go away any time soon. However, there are several reasons to be optimistic about the future of mobile testing.
The continual development of automated testing tools means that it’s getting easier to incorporate automation into your testing cycle. While automation testing won’t replace the need for manual testing, it will help improve the efficiency and consistency of test programs.

New technologies like 5G and edge computing are set to improve mobile infrastructure and connectivity across the globe. This means more efficient and reliable testing will become possible in a greater range of places.

Finally, the growing importance of security, privacy, and usability issues, and the increase in the number of mobile apps across the globe, mean the mobile testing industry is here to stay.

How can Global App Testing help?

Real life isn’t a controlled test environment, and no test framework will ever be able to fully replicate the real-world conditions a mobile product will encounter upon release. Thankfully, with crowd-testing functional and multi-device usability testing services from GAT, you are well-equipped to run your mobile app through a rigorous test cycle.

So whether you’re building a native Android or iOS app, a containerized web app, or a hybrid application, choose a testing solution from GAT for your mobile development project and create an app your customers will love!
Interested to learn more about our services?
Schedule a call with our specialist today!

Keep learning

My app retention rates are falling: what can I do?
How to write manual test cases for API testing easily
Mobile app deployment: what it is and how to plan for it