Informatics Seminar on Thursday,December 6th at 16.00 - Todd Mytkowicz
Decanato - Facoltà di scienze informatiche
Data d'inizio: 6 Dicembre 2007
Data di fine: 7 Dicembre 2007
The Faculty of Informatics is pleased to announce a seminar given by Todd Mytkowicz
TITLE: Rethinking Performance Evaluation
SPEAKER: Todd Mytkowicz, University of Colorado at Boulder, USA
DATE: Thursday, December 6th, 2007
PLACE: USI Lugano, Room SI - 006, Informatics Building (Via G. Buffi 13)
TIME: 16.00-17.00
ABSTRACT
Reproducible performance evaluation is an integral part of experimental computer science; if researchers cannot recreate the results from published works they will be unable to compare their methods to the state of the art. In this talk we argue that modern computer systems are so complex that small changes in their initial state can have drastic effects on their runtime performance. This sensitivity means that current methods in experimental performance evaluation do not necessarily provide reproducible results. We propose to increase the reproducibility of performance evaluations by deliberately affecting the initial state of the computer system and looking at average behavior, a technique we call intervention. We employ intervention in the context of evaluating compiler optimizations on the SPEC CINT 2006 benchmark suite. Using this preliminary data, we provide a head to head comparison of intervention and the current state of the art in performance evaluation and show how easy it is for current methods to be led astray.
