Automatically Generating Complex Test Cases from Simple Ones

Staff - Faculty of Informatics

Start date: 15 July 2013

End date: 16 July 2013

You are cordially invited to attend the PhD Dissertation Defense of Konstantin RUBINOV on Monday, July 15th 2013 at 16h30 in room A31 (Red building)
 
Abstract:
While source code expresses and implements design considerations for software system, test cases capture and represent the domain knowledge of software developer, her assumptions on the implicit and explicit interaction protocols in the system, and the expected behavior of different modules of the system in normal and exceptional conditions. Moreover, test cases capture information about the environment and the data the system operates on. As such, together with the system source code, test cases integrate important system and domain knowledge.

Besides being an important project artifact, test cases embody up to the half the overall software development cost and effort. Software projects produce many test cases of different kind and granularity to thoroughly check the system functionality, aiming to prevent, detect, and remove different types of faults. Simple test cases exercise small parts of the system aiming to detect faults in single modules. More complex integration and system test cases exercise larger parts of the system aiming to detect problems in module interactions and verify the functionality of the system as a whole.  Not surprisingly, the test case complexity comes at a cost -- developing complex test cases is a laborious and expensive task that is hard to automate.

Our intuition is that important information that is naturally present in test cases can be reused to reduce the effort in generation of new test cases. This thesis develops this intuition and investigates the phenomenon of information reuse among test cases. We first empirically investigated many test cases from real software projects and demonstrated that test cases of different granularity indeed share code fragments and build upon each other. Then we proposed an approach for automatically generating complex test cases by extracting and exploiting information in existing simple ones. In particular, our approach automatically generates integration test cases from unit ones. We implemented our approach in a prototype to evaluate its ability to generate new and useful test cases for real software systems. Our studies show that test cases generated with our approach reveal new interaction faults even in well tested applications. We evaluated the effectiveness of our approach by comparing it with the state of the art test generation techniques. The evaluation results show that our approach is effective, it finds relevant faults differently from other approaches that tend to find different and usually less relevant faults.

Dissertation Committee:

  • Prof. Mauro Pezzè, Università della Svizzera italiana, Switzerland (Research Advisor)
  • Prof. Mehdi Jazayeri, Università della Svizzera italiana, Switzerland (Internal Member)
  • Prof. Matthias Hauswirth, Università della Svizzera italiana, Switzerland (Internal Member)
  • Prof. Mark Harman, University College London, United Kingdom (External Member)
  • Prof. Gregg Rothermel, University of Nebraska-Lincoln, USA (External Member)