Multi-Dimensional Test Data

June 14th 2019
- by: The Sogeti Digital Assurance & Testing Practice

 

According to the 2018-2019 World Quality Report one of the biggest challenges facing organizations is the development and management of test data. With the advent of AI systems being utilized to aid testing, the Testing sphere will be undergoing a massive shift where the Data used as the source for AI based testing systems becomes critical.

In the book Testing in The Digital Age the authors point out the importance of the link between AI making a difference in Testing when of course the AI is supported by reliable Data and good algorithms which can learn form that data and here a focus is put on the data coming from Test Management tools and other archives which focus primarily on the Test Data from the test design and test execution phases and show a great many ways AI can help make the difference in choosing test cases based on risks identified in the requirements and successive test run data (Pass/ Fail rate) as well as anticipating defects based on the data trends found in the defect tracking data . But What about the synthesis and synergy of these two types of “Test “Data.

Here the potential increases for companies that need to have that connection between not only knowing what test cases to execute to cover the highest risks with the optimal test coverage but the exact combinations of test data sets used as inputs in a test case, which can trigger particular happy paths and also negative testing. Here test data choices become not one dimensional but two dimensional. The Cognitive QA approach takes this consideration into focus. Here the data used , both in terms of process and what test data will be used for test execution is an essential aspect for determining with the correct algorithms what outcomes in testing will lead to and how to be able to “Predict the Unpredictable”. This is a fascinating possibility. Where testing becomes less reactive and more proactive.  

The challenge on how to use this test data, supported by AI or not remains a challenge after many years. And with GDPR considerations, organizations will need to come up with more solid and creative ways to circumvent reuse of production data and even synthesize data needs. SO, these challenges are not new, and many have pointed out the need for test data handling as a serious consideration. One of the gurus of testing from the start was Glenford J Myers ( In the seminal Book : “The Art of Software Testing” ) and he also stressed the importance of having a solid an reliable test data archive and for Quality and improvement process a guru W. Edwards Deming stressed the use of a repeatable cyclic process where process Improvement data needed to be handled in a continuous and structured manner.

There is an ironic quote which also comes to mind and which stresses the importance of Test data in a whimsical way with by stating “In God We Trust… The Rest Send Data”. Whereas there is dispute as to who this quote can be attributed, it most surely shows that the data is the key.