10 success factors for mobile testing

So, where are the differences and how should you accommodate them? We have identified ten areas that are highly relevant to mobile testing and require particular attention but if they are handled correctly will mean a successful mobile testing effort.

Success Factor #1:

Connectivity-related testing
Most mobile solutions depend on some kind of network connectivity. Solution design and testing are needed to address variable band-width, offline and flight mode scenarios, and validate user sessions moving between different network conditions.

Both automated network simula-tions and manual testing in real network conditions are required to ensure consistent behavior.

Success Factor #2:

Understanding Physical Characteristics
The physical characteristics in mobile solutions differ significantly from non-mobile solutions. Mobile testing needs to take several of these characteristics into consideration such as: 

• Screen size.
• Touch and gesture capabilities.
• Orientation (vertical or horizontal) and movements in three dimensions.
• Camera.
• GPS.

Some testing scenarios related to these physical characteristics can be automated, for example, orientation changes; but some scenarios do require manual testing, for example, synchronization of gestures and sounds.

Success Factor #3:

Location Simulation
An ever increasing number of solutions utilize location data and GPS integration. In these solutions, features are designed to depend on location or distance to other locations. Successful mobile testing ensures quality across different types of GPS implementations and needs to utilize efficient location simulation.

Success Factor #4:

Test Automation
The ability to automate tests using scripting or recording makes mobile testing significantly more efficient. Smart test automation utilizes parameterized scripts for both device types and user interaction, and is enabled to run on devices in-house and in the cloud. Some sophisticated user interactions involve touch, gestures, and sensors which cannot be fully automated, and, as a result, will require manual testing.

Success Factor #5:

Dealing with Fragmentation
The market fragmentation for both operating systems and device types continues to challenge solution design and testing. Most solutions, both internal and external, need to support hundreds of device types and several versions of operating systems. A relevant mobile testing tool box includes: 

• Physical access to the major device type and operating systems combinations.
• Ability to run manual and auto-mated tests across both physical devices and emulators.
• Access to cloud-based platforms to maximize testing a number of devices.

Success Factor #6:

User Experience
User experience is a main criterion for success. Testing user experience is an art that consists of both objective and subjective measurements. Key elements include: 

• Adherence to platform rules and guidelines, as dictated by app store review guidelines.
• User interaction – Level of intuitiveness and efficiency.
• Navigation – Adherence to platform rules and guidelines.
• Signup and login – Clarity and ease of use. 
• Layout and user interface design – Look and feel, aesthetics, and precision in layout. 
• Exception handling – User friendly messages and graceful exception handling.

Success Factor #7:

Engaging with Third Parties
Most apps are distributed through open and public app stores, each with its own set of guidelines. Apps that fail to adhere to guidelines may be rejected. And, given the third party review process, there is a bug fix latency inherent in updates. Also, when operating systems are updated, the new releases can break existing apps. This means that successful mobile testing must consider:

• Using common testing checklists based on the most recent app store rules and guidelines.
• Using rapid testing cycles for updates in order to minimize impact of bug fix latencies.
• Testing existing apps on beta versions of operating systems.

Success Factor #8:

End-to-End Integration Testing
As mobile solutions and apps become increasingly more business relevant, the scope and depth of transactional features expand and so the need for well-designed and validated system integration solutions has to be addressed as well. When specific systems are needed to remotely manage a fleet of handheld mobile devices (Mobile Device Management or MDM), the mobile solution should bevalidated and tested against the main func-tionalities of the MDM systems: 

• Device technologies supported.
• Range of embedded applications supported.
• IT policy control.
• Device security enforcement.
• Management of connected devices.
• Third-party applications control.

Success Factor #9:

Performance
Mobile solutions can overload servers, through the rapid growth in the number of users and an increase in the average life span of transac-tions due to variations in bandwidth and latency. This can have a performance impact on all users. Also, there are significant performance differences across different types of devices. An app or responsive web may run well on a high-end device but may not be acceptable on a low-end device.

Successful mobile testing focuses on network capabilities, system integration and back end layers, as well as the app itself.

Success Factor #10:

Security
Mobile solutions are implicit in an increasing number of scenarios where sensitive systems are accessed and private data is in transit or at rest, that is, stored in mobile devices. And given the physical nature of mobile devices, they are more easily forgotten, lost, or stolen. Mature mobile testing aims to: 

• Secure confidentiality and integrity of data.
• Validate whether authentication and authorizing is secure.
• Verify that systems are keeping records of events (non-repudiation).

We are mobile testing experts and understand these central success factors and know how to leverage them. Our fine-tuned capabilities are summarized in this document, along with a description of our market-leading testing methodology, TMap NEXT®, which supports the complete testing process. We will explain in detail how our services can help your business achieve its mobile testing goals.