Test Suite Comprehension for Modular and Dynamic Systems
Test suite comprehension became more difficult over the last decade as test suites have grown substantially. Especially for modular and dynamic systems, where the system functionality can change at runtime, testing is aggravated. In addition to enabling dynamic reconfigurations, modular systems are conglomerates of several sub-systems, with different owners. Those two characteristics aggravate testing and test suite comprehension (e.g., understanding which configurations and combinations of the system have been tested) and therefore such systems require further investigation.
My PhD thesis consists of several studies that investigate how abstract models of complex and modular systems can help people understand their test suites.
Research Questions
In this dissertation, I investigated the following research questions:
- What makes testing of modular and dynamic systems challenging?
- What makes integration testing more difficult than unit testing?
- How can we support developers during understanding high-level tests?
- How can we support developers during test maintenance?
All chapters of this dissertations have been rigorously peer-reviewed and have been published in several high ranked publications mostly by IEEE and ACM.
Overview of Content
Chapter one is called “Test Confessions: A Study of Testing Practices for Plug-In Systems”. Here, I present the outcome of a qualitative study including a survey with over 500 developers and testers from the Eclipse community. In this study, I investigated the challenges Eclipse developers face by testing highly dynamic plug-in systems. It also highlights what Eclipse developers and testers think about different types of tests – such as unit, integration or system tests.
In chapter two, I present a tool called ETSE (Eclipse Test Suite Exploration Tool), and an accompanying study of the usefulness of this tool. This chapter also shows several models that help to understand plug-in test suites.
In the third chapter, I introduce another method that can be used to measure the similarity of test suites in order to support the understanding of complex test suites. Also, this method has been evaluated based on two open-source systems.
Finally, in chapter four, I present a technique that allows the detection of test fixture smells. Test smells are antipatterns that can reduce the test execution time of the test suite, hinder maintainability or reduce the readability of the test suites.
