Development of an automated test data generation and execution strategy using combinatorial approach
Loading...
Date
2009
Authors
Jamil Klaib, Mohammad Fadel
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
To ensure acceptable level of quality and reliability of a typical software product, it
is desirable to test every possible combination of input data under various
configurations. Due to combinatorial explosion problem, considering all exhaustive
testing is practically impossible. Resource constraints, costing factors as well as strict
time-to-market deadlines are amongst the main factors that inhibit such
consideration. Earlier work suggests that sampling strategy (i.e. based on t-way
parameter interaction) can be effective. As a result, many helpful t-way sampling
strategies have been developed in the literature.
Much useful advancement has been achieved in the last 10 years particularly to
facilitate the test planning process, that is, in terms of systematically minimizing the
test data to be considered for testing (i.e. based on some t-way parameter
interactions). Despite such a significant progress, the integration and automation of
the strategies from the planning process to execution appears to be lacking. In the
current practice, the sampled test data need to be manually extracted and converted
to some acceptable format before they can be executed (e.g. by a human tester, a
code driver or a third party execution tool). This lack of integration and automation
between test planning and execution can potentially burden the test engineers
especially if the software module to be tested is significantly large.
Apart from integration and automation issues, strategizing to sample and construct
minimum test set from the exhaustive test space is also a NP complete problem (i.e.
nondeterministic polynomial). As such, it is often unlikely that efficient strategy
exists that can always generate optimal test set. Motivated by such challenges, this
paper discusses the design, implementation, and validation of an efficient strategy,
called GTWay. GTWay, unlike other strategies, supports both t-way test generation
and automated (concurrent) execution integrated within the strategy itself. Empirical
evidences demonstrate that GTWay, for some cases, outperforms other strategies in
terms of the number of generated test data. The test generation time is also within
reasonable value considering the fact that some overhead is required to permit the
integration between test generation and execution.
Description
PhD
Keywords
Electrical engineering , Test data generation , Combinatorial approach