Automatic Test Suite Evolution

Decanato - Facoltà di scienze informatiche

Data d'inizio: 9 Novembre 2012

Data di fine: 10 Novembre 2012

You are cordially invited to attend the PhD Dissertation Defense of Mehdi MIRZAAGHAEI on Friday, November 9th 2012 at 10h30 in room A22 (Red building)

Abstract:
Software testing is one of the most common approaches to verify software systems. Despite the many automated techniques proposed in the literature, test cases are often generated manually. When a software system evolves during development and maintenance to accommodate requirement changes, bug fixes, or functionality extensions, test cases may become obsolete, and software developers need to evolve them to verify the new version of the software system. Due to time pressure and effort required to evolve test cases, software developers do not update test cases often and many test cases become quickly obsolete. As a consequence, the effectiveness of the test suites drops over time.
In this thesis, we propose a new technique for automating test suite evolution aiming to reduce both the developers effort and the costs of software evolution. To understand the evolution of test cases, we conducted an empirical study on real world software systems to gain an insight on how software developers evolve the test cases. Our study shows that many test cases of software systems are similar. These similarities lead developers to follow common activities for repairing and generating test cases by using some common processes. Based on this observation, we identify some Test Reuse Patterns that can be applied to fix many test cases.
In this thesis we define five Test Reuse Patterns: change of method declaration, extension of class hierarchy, implementation of interface, introduction of overloaded and overridden methods. We propose a framework for generating and repairing test cases for evolving software by automating Test Reuse Patterns. The framework, called Test Care Assistant (TCA), repairs not compiling test cases and generates test cases when new classes or methods are added to the software. We evaluated TCA on open source projects and the results suggest that TCA can be successfully adopted in software testing process to help software developers evolve the test cases with less effort and time. The empirical evaluation of the framework shows that TCA can be successfully applied in many cases and that the effectiveness of the test cases generated and repaired by TCA is comparable with the effectiveness of the ones generated by software developers and state of the art techniques.

This thesis contributes to the state of the art by introducing Test Reuse Patterns and proposing a framework,TCA that exploits the Test Reuse Patterns to repair and evolve test cases in the software. Other contributions of this thesis are: an empirical study of software evolution conducted on different test suite repositories, the implementation of TCA prototype, and the empirical evaluation of the applicability and effectiveness of the approach on open source case studies.

Dissertation Committee:

  • Prof. Walter Binder, USI Università della Svizzera italiana, Switzerland
  • Prof. Cesare Pautasso, USI Università della Svizzera italiana, Switzerland
  • Prof. Darko Marinov, University of Illinois at Urbana-Champaign, USA 
  • Prof. Paolo Tonella, Fondazione Bruno Kessler, Trento, Italy