There are many ways to organize software development. Today, the infamous waterfall model has mostly been replaced by some flavor of agile development. However, this change hasn’t been a panacea. Still, software developers are plagued by lack of time, unclear requirements, barely legible legacy code and processes which only get in the way. How can we find the root cause for these problems?

Since the downfall of the waterfall model, it has become out of fashion to think of different phases in software development. In Scrum, for example, there are no more phases: Instead, you have sprints which combine all the old phases of the waterfall model in a single short time span. To recap, the waterfall model had these phases:

  1. Requirement gathering and analysis
  2. System design
  3. Implementation
  4. Testing
  5. Maintenance

All these activities still exist when you’re using an agile development model: Implementation is done within a sprint. The system design can either be done beforehand or during the implementation via continuous design. Requirement gathering and analysis should be done before the sprint starts to make sure that the implementation doesn’t get blocked because of open questions. Testing is done as part of feature development and/or in specialized regression tests. Maintenance is a permanent background task.

The old waterfall phases can be helpful to a software developer when searching for the root cause of (recurring) problems. For example, say that you are implementing a new feature and everything takes much longer than expected because you stumble from open question to open question. Finding answers takes time and suddenly you are not sure anymore that you can deliver in this sprint (or at all). Here, the very first phase didn’t go well: The requirements were too poorly understood to even start development of this feature. Why was this the case? Maybe the product owner (or whoever defines features in your process) didn’t spot all the open questions. Maybe he knew, but decided to start development anyway because the feature was important and things would work themselves out over time. Our maybe the customer didn’t clearly express his ideas when he made the request in the first place. In any way, the cause of the problem is found upstream in the development process. If this happens often, then something is wrong with your requirements engineering process.

Another example for an upstream problem is a lack of focus, i.e., the priorities shift all the time in unpredictable ways. Things that were critical last month are suddenly unimportant even though they were only partially addressed (or not at all). Often these confusing changes in priority are caused by conflicting interests raised by equally powerful stakeholders. I think it is important to have at least a rough idea on what you plan to develop in the next three to six months and to commit to this vision.

Many code bases are in a terrible state and get worse with every line of code which gets changed. To keep code from deteriorating over time, constant refactoring needs to be applied and new code needs to be written properly. It is the developers’ task to do this, but often they are under pressure, badly trained or a combination of the two. Sometimes refactoring tasks are too big to get done as part of a related feature (or bug fix). Then, they need to be planned and executed at an appropriate point in time. Balancing these refactoring tasks and feature development is the job of the product owner in a Scrum process. If refactoring is always pushed back indefinitely, then the code base will never improve. So, if you find yourself working in a messy code base, you might want to check in with your product owner and discuss priorities. Note, that this is not an excuse to write bad code. The majority of blame for bad code clearly lies with the developers.

Last, we briefly have to talk about staffing. I strongly believe that people are more important than processes. With a tight knit and competent team, any process can be done in an efficient manner. But with a dysfunctional team, even the best process will not work properly. The development managers are responsible for staffing and hence, they have a crucial job in software development. I’ve outlined in my developer matrix how big the differences between developers can get. So, hiring the right developers is essential. Of course, a similar spread exists in more bureaucratic roles. Let’s take a product owner as an example. A product owner has a lot of responsibility in a Scrum process and needs to make hard decisions on a daily basis. He needs to know both the product and the team well to make a meaningful plan. A bad product owner will place a huge burden on the development team as his tasks get shifted downstream. Managers should make very sure that the right person is doing this job. Naturally, they also have to hire the right amount of people at the right time. Hiring too few employees is less dangerous than hiring too many. As Fred Brooks pointed out decades ago: Adding people to a late development project only makes it later. So, even when managers are “only” dealing with people issues, they still fulfil a crucial role.

That concludes this article. I hope you get the general idea. When you face your next problem during software development, take a moment and search for the root cause upstream. It might be enlightening. If you liked this blog post, please share it with somebody. You can also follow me on Twitter/X.