Testing Practices for Plug-in Systems: Confessions
Testing plug-in based systems is challenging due to complex interactions among many different plug-ins, and variations in version and configuration. The objective of this paper is to find out how developers address this test challenge. To that end, we conduct a qualitative (grounded theory) study, in which we interview 25 senior practitioners about how they test plug-ins and applications built on top of the Eclipse plug-in framework. The outcome is an overview of the testing practices currently used, a set of identified barriers limiting the adoption of test practices, and an explanation of how limited testing is compensated by self-hosting of projects and by involving the community. These results are supported by a structured survey of more than 150 professionals. The study reveals that unit testing plays a key role, whereas plug-in specific integration problems are identified and resolved by the community. Based on our findings, we propose a series of recommendations and areas for future research.
Publication
More details of the study can be found in the associated publication.
Test Confessions: A Study of Testing Practices for Plug-in Systems
Michaela Greiler, Arie van Deursen, Margaret-Anne Storey
International Conference on Software Engineering (ICSE) 2012, Switzerland
Download: PDF
