Empirical Software Engineering as a Science: Challenges and Ways Forward (Take 2)

Speaker:
Robert Feldt, Chalmers University of Technology, Sweden

Abstract:
The Empirical Software Engineering (ESE) community has made great progress in the last 20 years and expanded the field considerably both in scope, volume as well as quality. Nowadays, we have established conferences as well as journals focused on the area, and a majority of the papers published in the top SE conferences (such as ICSE) are empirical. However, while more established scientific fields such as Physics, Biology and Psychology have clear identities, specific schools of thought, and explicated research methods, I argue this is less so in ESE.
In this talk, I propose an updated manifesto for empirical software engineering and discuss some challenges and possible fixes to address them. This, I hope, can give a clearer sense of identity as well as act as a vision for next steps. In particular, I discuss the negative effects of our love for novelty (neophilia) and how it affects publication bias and is a challenge to find truth. I also summarize the ongoing debate among statisticians about how to move beyond p-values as well as some ideas for how to improve empirical studies that use qualitative methods. I will discuss some strategies for how we can improve the reliability and validity of our ESE research and conclude with concrete call-for-actions so that we can be an even stronger science going forward.

Biography:
Robert Feldt (http://www.robertfeldt.net/) is a professor of Software Engineering at Chalmers University of Technology, Sweden, and at Blekinge Institute of Technology, Sweden. He has broad research interests spanning from human factors to hardcore automation and statistics, and works on software testing and quality, requirements engineering, as well as human-centred (behavioural) software engineering. Dr Feldt was an early contributor to search-based software engineering and has recently argued for increased application of psychology and social science to understand and improve software engineering. Most of his research is empirical and conducted in close collaboration with industry partners in Sweden, Europe and Asia, but he also leads more basic research. Dr Feldt received a PhD in Computer Engineering from the Chalmers University of Technology in 2002, has studied Psychology at Gothenburg University in the '90s and has also worked as an IT and software consultant for more than 25 years. He is passionate about empirical research and methods and changing organisations through technological innovation. He is co-Editor in Chief of the EMSE journal and on the editorial board of three other journals (STVR, ASOC, and SQJ).

Hosts: Prof. Carlo A. Furia and Prof. Paolo Tonella