Data-driven: Can we change what we can measure?

This week, I am in sunny Portugal as I am invited as the keynote speaker at the International Conference on the Quality of Information and Communications Technology (Quatic) conference. My keynote highlights my work at Microsoft in the area of data-driven software engineering.

My talk has the slight provocative title “Can we induce change with what we can measure?” and hints at reconsidering the origin of change we might observe due to measurements we take, but also calls for deep understanding of the measurements we take and the context in which we perform our study.

Tom DeMarco states that “You can’t control what you can’t measure”, but how much can we change and control (with) what we measure? This talk investigates the opportunities and limits of data-driven software engineering, shows which opportunities lie ahead of us when we engage in mining and analyzing software engineering process data, but also highlights important factors that influence the success and adaptability of data-based improvement approaches.

In summary, I stress the importance of domain knowledge when engaging in data analytics, but also for any software development improvements we strive for. A deep understanding of the team culture, processes, tools, code organization, but also release cycles and reporting structures are crucial to really understand the phenomena at hand, or to probe the data in the right way.

I also highlighted the role of mixed method research – which is a combination of quantitative methods (e.g., statistical tests, prediction models) and qualitative methods (e.g., interviews, observations, grounded theory) – during engaging in data-driven software engineering.

Updated on February 17th, 2019