Technical report detail

A Benchmark for Change Prediction

by Romain Robbes, Michele Lanza, Damien Pollet

The goal of change prediction is to help developers by recommending program entities that will have to be changed alongside the entities currently being changed. To evaluate their accuracy, current change prediction approaches use data from versioning systems such as CVS or Subversion. However, as these data sources are not very accurate, they do not provide a valid basis for an objective evaluation of change prediction approaches. We propose a benchmark for an objective evaluation of change prediction approaches based on fine-grained change data recorded from IDE usage. Moreover, the change prediction approaches themselves can use the more accurate data to fine-tune their prediction. We present an evaluation procedure and use it to evaluate several change prediction approaches, both our own and from the literature, and report on the results. Our results show that using fine-grained change data significantly improves the overall accuracy of change prediction approaches.

Technical report 2008/06, October 2008

BibTex entry

@techreport{08benchmark, author = {Romain Robbes and Michele Lanza and Damien Pollet}, title = {A Benchmark for Change Prediction}, institution = {University of Lugano}, number = {2008/06}, year = 2008, month = oct }