The Human Side of Software Development, Agile Methods, and AI Bias
This week in Top Software Engineering Reads you read about the human side of software engineering, how Microsoft switched to agile development and how to handle bias in your artificial intelligence project.
The struggles of an open-source maintainer
A read I want to share with you this week is called the struggles of an open-source maintainer. In this blog post, Salvatore shares very openly how he feels with the enormous amount of pull requests and issues he gets from the community. I admire this post for being so open. It shows many issues an open-source maintainer has to deal with. I think some of the struggles are a bit unique to open source, as Salvatore started this project out of pure interest. As it was an open-source non-commercial project he could take the liberty to work whenever he wants and on whatever really interests him.
Now that Redis is that popular, others want to have a say in the direction. And to a certain extent, that’s okay, or isn’t it? I love the post because it is so human.
It shows what a human-centered activity software development is. Salvatore describes how he feels inadequate in dealing with the sheer amount of pull requests and community feedback. He discusses his fears to lose ownership over his software “baby”. What it means for him to be working every day on the same project and which tolls working a regular job similar to a 9-5 from Monday to Friday has on his productivity. Each section does not go much into detail, but the article paints a very honest and authentic picture. Thanks for sharing, Salvatore.
From waterfall to agile
You are probably familiar with Waterfall and Agile development. If a software project uses waterfall, the project activities are split into linear sequential phases. Each phase depends on the outcome of the previous phase. Agile development, on the other hand, focuses on an iterative process of short incremental phases. For several years now, there is a strong shift from waterfall to agile software development.
Even though agile methods have been around since, what feels ages, many organizations are still not using it. Also, Microsoft has used to develop software in a waterfall-like approach. But over the last years, they changed by embracing agile methods. In this video, Aaron Bjork walks you through what exactly changes and why. For such a large company, this change is impressive. And you will see that changes touch every aspect of the organization, be it how a team operates, which roles exist to the team size, or how to measure success. So enjoy the video or the accompanying blog post here.
If you enjoy hearing about experiences from developers at different companies and which methodologies and best practices they use, you surely also like my soon-to-launch Software Engineering Unlocked Podcast. Have a look and don’t forget to sign-up.
Ethical Debt in AI products
We all have probably already heard of the bias problems using AI products. One of the problems is that, if the training data or the input is biased, so is the output. This means, for example, that if we train face recognition with the current datasets they are much better in predicting white-male faces than faces of the black population or of women. This is also an important issue that is discussed in Congress in order to know how AI technology can be used by agencies such as the FBI or the police. A high-light overview of the problems with facial detection is discussed here. But more importantly, I wanted to share with you a podcast episode of Scott Hanselman where he discusses with Lauren Maffeo how to manage machine bias in artificial intelligence. If you prefer reading, have a look at this short summary of how to manage bias in your AI project.
An Investigation into the Cloudflare Outage
Finally, I want to end with the incident report of Cloudflare. On July 2, 2019, many of the major sites on the Internet were not reachable. Why? Because a popular Content Delivery Network (CDN) had an outage. This is the full incident report. In a nutshell, what happened is that one purely written regular expression backtracked enormously and exhausted CPU used for HTTP/HTTPS serving. But, that’s not what is interesting. The real value of this piece is that it describes the deployment processes and checks as well as the site reliability set-up of such a large company. If you do not want to read the whole piece make sure to read what happened and what went wrong. It is worth your time! So, if you haven’t looked at it yet, I suggest give it a read.
So, that was it again for this week. I hope you found a few interesting articles, and if you haven’t done so, make sure to sign-up for my newsletter.
Each week, I’ll send you my top software engineering reads including my thoughts and reflections on them. And I ping you if a new blog post is out. Also, make sure to check out the code-review e-book I prepared for my subscribers.
