How much Quality should we Expect from Software?
Earlier this week, Medtronics bragged about one of their flagship products, a ventilator. Apparently, it depends on over one million lines of code, and the Vice President also adds that this ventilator is a highly complex piece of electro-mechanical equipment.
So, I don’t know about you, but when I heard this, I just stared in shock and disbelieve watching this guy brag about something he should not be too proud of.
Impressing number of lines of code
I understand that this advertisement should impress the public – and not software engineers. Still, the amount of code written is what’s also used to impress business managers, CEOs, investors, politicians, up to presidents.
Well, most software engineers know that more code isn’t something we should be proud of. If anything, more code means more liability and certainly more bugs. And when we hear highly-complex in combination with 1 million lines of code, we know we are in trouble – maybe we just don’t yet know how.
Probably, most of these 1 million lines of code belong to the operating system of the ventilator. But still, what does it mean to have 1 million lines of code?
Do we trust this system?
Are we trusting software systems
And here we are by thinking about trust.
Do we trust the simulation software that influences the measures countries take against COVID19?
Do you trust this software, if you hear that it was mostly written over 13 years ago by one single guy and consists of thousand lines of undocumented C code?
Well, all over the world, governments are using output form simulation and modeling software to make decisions that impact the lives of thousands of people. This time around, we are talking about life or death decisions. The measurements governments take are based on output from such software and influence how many people get sick or die, how many people lose their job, how many businesses close.
So, what is the first thing you would like? I guess it is to have a look at the code, right!
Build trust by open sourcing
So, Neil and others are taking the right step. Open sourcing the system. But before we can see it, the system must be tidied up. Because, let’s face it, software written by one single guy called Neil, looks like…
Well, like software written by a single person. Messy.
So, Neil, announces that several organizations help “to document, refactor and extend the code to allow others to use without the multiple days training it would currently require”.
So, this gets me thinking that changing, refactoring, and extending undocumented code that needs multiple days of training for others to understand, is a really bad idea. Because I assume there is no regression test suite available either. So, how do you know what the impact of your change is for this undocumented, legacy C code?
It’s not a Neil problem
Well, but let’s stop focusing on Neil. Because it’s not a problem with Neil’s code. It’s a problem with code in general.
It’s a problem that we do not admit that code is messy. It’s seldomly beautiful, especially when the software grows and ages.
So, what is one of the best strategies to counteract scenarios like this? Right, code reviews.
Code reviews help counteract code that’s messy, undocumented, hard to understand and change. I’m not saying it is buggy. You are 😉
Well, why should code reviews help?
The effectiveness of code reviews is closely connected to human nature. Simply by knowing that others will see your code helps you improve the code. Because let’s be real, we do not want our colleagues to see the mess we made. What motivation to improve and learn?!
In addition, having another person look over your code, will increase the likelihood of detecting bugs. What a great side-effect.
I stop now writing about the benefits of code reviews because I want to talk about something else.
Research Software
What if it’s just you working on this piece of code? What if this is not commercial software but software from research institutions (like we now see used for determining COVID countermeasures).
What’s a lonely researcher’s strategy to code quality?
To be honest, not much. The constant pressure to produce papers and new “insights” means that there is no time to improve “prototypes”. The incentives for high-quality code and a successful research career are not aligned. Even less than for a company that generates revenue from software.
Open Source to the rescue
I know, because I was part of this system. Open sourcing the software is one strategy that can be used here.
But, what stands against this is the fact that in our society we aren’t allowed to fail. Despite what Silicon Valley and the start-up scene suggest, failure isn’t something we voluntarily sign up for.
We do not want to be the person making mistakes. And we do not want to be the person writing buggy, messy code. Especially if we fear negative consequences from it. So, the incentive for a researcher to open source the software rarely exists.
Let’s assume there is a problem with your code? Let’s assume you made a mistake and your findings aren’t okay. Retracting papers, and publish negative outcomes are possible. In our current scientific community though, it’s as nice as walking naked through the city.
In fact, admitting mistakes, retracting papers, and showing code can be damaging to a researcher’s career and reputation. It all depends on the research field though.
The right to inspect software
But, if software deeply impacts all our lives, do we have a right to see it and to inspect it? And, should it be of the highest quality we can possibly presume?
I believe so.
So, what can we do to improve the situation?
First, let’s be realistic and authentic about human nature and software.
Humans need to feel safe. Learning, growing, being honest and vulnerable all depend on us feeling safe. Showing your code, admitting that mistake, voicing your concern, confess that incentives do not align, acknowledging that you need to learn are all part of this.
One part of feeling safe is known as psychological safety. And psychological safety is crucial to make a teamwork well together. It’s crucial to allow people to grow and learn and become their best selves.
And, what about software? Only toy-software is bug-free, clean, nice, elegant. Real-world software is complex, hard to understand, contains bugs, and gets harder and harder to maintain with each line of code that is added and each year that passes by.
Our goal is to make software as robust, bug-free, understandable, maintainable and so on. That’s a great goal. And, the best way to do so is by not writing and testing software in silos.
You improve code by getting input from others, be it through pair programming or by doing code reviews.
To successfully work together we have to again acknowledge human nature. What a beautiful circle, right?
So, to answer my questions: How much quality can we expect from software?
Well, it depends on many factors. The experience of the people involved, the alignment of incentives, the work style, the development practices and so on. But two highly contributing factors are psychological safety and the incentive structure. Maybe those are the best proxies for software quality out there. Except, that you cannot easily measure them.
But that should not be a reason to revert to measuring simpler things like lines of code or cyclomatic complexity 😉
Remember, we are all part of a socio-technical system. And that’s so fascinating hard.
If you liked that article, have a look at my code review workshops, packed with knowledge on how to improve software quality, and increase collaboration and communication between team members.
Also, maybe you want to hop on to my weekly emails and get access to my code review e-book? It’s a good idea 😉
