Do you know these moments when you read a book, and suddenly, there is this sentence or paragraph that resonates so strongly with you that you wish the author was standing in front of you so you could throw yourself on the floor, humbly weeping, hug his legs, and thank him for his words of wisdom and truth? I experienced such a moment lately when I was reading the quality chapter of Peopleware. Let me quote the words that almost touched me to tears:
“We all tend to tie our self-esteem strongly to the quality of the product we produce […]. Any step you take that may jeopardize the quality of the product is likely to set the emotions of your staff directly against you.”
The authors, Tom DeMarco and Timothy Lister, further describe how, in some Japanese companies, the project team has a power of veto over a product release. If the product does not meet their self-imposed quality standards yet, they can say “no, we have to postpone”. In other words, engineers can force a deadline to be missed when they are deeply convinced that this is better for the product in the long run.
This discussion made me think about Scrum, and an aspect of it that I find to be harmful sometimes. Scrum imposes artificial deadlines on the development team on a regular basis. Only these deadlines are not actually called deadlines, but “end of sprint”. However, in practice, I have observed that the sprint end often has a very similar effect: Everybody hurries and tries to finish things before the sprint closes. Developers put in overtime. Product owners are reluctant to transfer open, unfinished work items over to the next sprint, so they urge developers to do everything they can to finish just this one item “in time”, and to take some short cuts if necessary. Developers, in their willingness to help, and in order to look productive, skip writing tests, corrupt the architecture of the codebase, and add technical debt. As this happens repeatedly, a big mess starts to pile up.
Is the deadline actually real?
I can think of roughly three scenarios where this is justified:
- The deadline is actually not artificial, but real, e.g., you promised a new feature to the customer by a certain date (which coincides with the sprint end), and there will be economic damage or damage to your customer relationship if you do not make that deadline.
- The date is not arbitrary, but critical, and any delay would render the feature worthless. For example, if you are building a “Special Christmas Deal” landing page, you probably don’t want to release it after, say, December 1st. This is something of a sub-case of case 1, just without an external customer.
- You can release only in long intervals, e.g., only once after each sprint. Therefore, you have a strong incentive to release a feature now instead of waiting for another two or three weeks, so you can start gathering data for the next iteration.
These scenarios happen, and they might in fact justify taking some short cuts to gain a short-term benefit. However, in a lot of companies, the majority of sprints does not fall into one of the above categories. In product companies, you don’t usually promise features to customers/users by a certain date, but often release them silently to a fraction of the visitors first (1). Critical dates like Christmas, or the 1st of April, or some anniversary, are rather rare (2). Moreover, companies with a modern build, deployment, and release process can release daily instead of every two weeks (3). This means, a lot of the sprint ends do in fact impose artificial deadlines, not real ones.
What are the dangers that come with these artificial deadlines? I have already mentioned the most obvious one: haste. As the sprint end nears, people start to feel like time is running out. Everybody tends to be ambitious with their sprint commitments, and estimates tend to be imperfect because of hidden complexity. So, sometimes, the sprint will be too short for the agreed-on commitment, and there is a natural desire by all team members to look good and get as much “done” as possible. I’m writing “done” in quotation marks, because, in practice, there are varying degrees of “done” - ideally, there should not be (see Definition of Done in any Scrum glossary), but, in reality, oh yes, there are.
So, instead of a thorough, architecturally sound implementation with good test coverage, your team delivers the 14th case branch to an existing switch statement, which adds to your technical debt. The constructor of EmailService now has twelve instead of ten parameters. You take short cuts, and you “forget” to write tests. Even worse, you had to deactivate some tests because they kept failing with your new feature implementation in place, and you had no time to adapt them. These are obviously things that harm your stability, and can slow you down in the long run. But you take comfort in the fact that your burn-down chart does look pretty nice now.
Peopleware sums up very nicely what happens when a development team is faced with an unrealistic deadline:
“There will be no freedom to trade off resources to make on-time delivery possible. They won’t have the option of more people or reduced function. The only thing to give on will be quality. Workers kept under extreme time pressure will begin to sacrifice quality.”
The knee-jerk justification attempt by the ones calling the shots is usually something like: “I know this is not ideal, but QA has had a look at it and it works, so let’s put it live so that we can start gathering some data on the user behaviour. We’ll add a task to write tests for the next sprint.” However, if time runs out in the next sprint (which, in certain teams, seems to be the norm rather than the exception), guess which tasks will get dropped first. It will be the ones that don’t provide any immediately visible value - or, better, any visible change, because the value of some features is probably also far from proven.
From there, it’s not far until: “About this Add the missing tests ticket: The way I see it, the feature works, it’s been in production for two weeks already, right? And it’s more like a prototype than a real implementation. We’ll throw the code away anyway and build it properly, so I think we can do without the tests for now.” The same argument goes for architectural or stability improvements.
With this kind of working comes another danger, which is more subtle, but possibly even more dangerous. Think back to the opening quote: “We all tend to tie our self-esteem strongly to the quality of the product we produce.” If we keep this in mind, it is easy to imagine the emotional effects this way of working has on the developers. Again, from Peopleware:
“They will push problems under the rug to be dealt with later or foisted off onto the product’s end user. They will deliver products that are unstable and not really complete. They will hate what they’re doing, but what other choice do they have?”
Some people reading this may object: “But it was the developers’ estimates, so they shouldn’t whine when they turn out wrong.” Yes, but it is not helpful or fair to confuse estimates with deadlines. Estimates lose precision quite fast as the underlying time period becomes longer. A lot can happen in two weeks, so unless the work is extremely repetitive, and the team members never change, you will always have to deal with inaccuracies.
After looking at the dangers of tight artificial deadlines, a counter question must be allowed: What good comes from them? If you can release daily, and there is no critical, non-arbitrary date to be met, I simply do not see any tangible benefit. “But the nearing sprint end pushes and motivates developers to get things done.” First, this comes at the cost of slowing you down in the long run, as just demonstrated. Second, this is not sustainable. You cannot have people pushing themselves to the limit every two weeks, and expect to increase productivity for more than a short period that way.
There is a section in Peopleware called “There Ain’t No Such Thing as Overtime”, which explains the psychological effects that condemn such thinking to failure:
“[…] there might be some benefit in a few extra hours worked on Saturday to meet a Monday deadline, but that’s almost always followed by an equal period of compensatory “undertime” while the workers catch up with their lives. Throughout the effort, there will be more or less an hour of undertime for every hour of overtime.”
I have always been suspicious of the term “sprint”, because it’s either a very bad metaphor, or, if it’s really meant that way, it just cannot work. As any runner can tell you, you cannot sprint for an entire race. If you start out sprinting, the more experienced runners will probably laugh at you. But more importantly, you will waste a lot of energy and soon slow down dramatically, until (if you are lucky) you regain your rhythm and find a sustainable pace. So, if “sprint” in Scrum really means “develop at full speed, as fast as you can”, then this will not work in the long run. If it does not mean that, but is supposed to describe some sustainable way of working, then it is a truly bad metaphor.
Even though regular, artificial deadlines seem to be a rather bad thing: Are there still valid reasons to have two-week sprints even though you release every day? Well, yes, there are.
First, dividing long-term work into sprints creates structure and provides orientation. This can be helpful in measuring and assessing one’s progress. A colleague of mine moved from a Scrum team to a “sprint-less” team a couple of months ago, and he told me that he actually misses the structure he had in the Scrum team, and also things like the retrospective meeting. The way he is working now, with very little structure in the development process, makes him feel like he is not accomplishing a lot. There is just one never ending stream of work items in which he sometimes loses track of time and direction.
Second, agreeing on and committing to a publicly visible set of tasks for a sprint provides and protects the team’s focus. Stakeholders cannot change priorities and interrupt the team’s work - and hence destroy their focus - whenever they feel like it. This reduces context switches for the developers, and gives them the chance to maintain their flow, which keeps productivity high.
Third, defining a sprint scope encourages communication and alignment on the team’s direction. “Look, this is where we want to be in two weeks. Does everyone agree, or should our priorities be different?” Especially in large teams of six or more developers, this can be very important and avoid a lot of communication overhead that would occur once people start moving into opposite directions.
Fourth, if there are multiple teams that work on the same product, or otherwise depend on each other’s work, having a common sprint rhythm can be useful for alignment and planning. If everyone has the same mental picture of time, you can easily have agreements like: “We will get the backend endpoints ready in the upcoming sprint, so you can start using them in the sprint after that.”
Best of both worlds?
So can we somehow get rid of the dangers that artificial deadlines pose, without giving up all the benefits that sprints provide? Can we stop sacrificing quality for no good reason while still providing some structure, focus, and alignment?
Getting rid of artificial deadlines means either getting rid of sprints entirely (Kanban comes to mind) or having sprints of variable length. If you drop sprints entirely, you can still have regular retrospective meetings where you, as a team, reflect upon the progress you have made, and talk about possible improvements to your development process. In order to create alignment among the team members, it probably also makes sense to plan ahead in regular intervals and discuss which direction the team should follow over the next couple of weeks.
Scrumban is a method that supports sprints of varying duration. Iterations are typically of similar length, but not necessarily of the exact same length. The planning meeting for the next sprint is not held on a fixed day, but when the number of work items in the “To do” column drops to a certain number. This way, the sprint actually adapts to the situation, which eliminates the deadline aspect from the equation. Nevertheless, there is still structure, because you have regular planning meetings, and you can easily have retrospective meetings if you want to.
No matter if you go with a “no sprint” or “flexible-time sprint” method, giving a regular progress demonstration to stakeholders or other teams can additionally provide structure and orientation. Have the team create short demos of new features or other improvements, so that they can feel and show what they have accomplished. To keep the demo dates from becoming artificial deadlines themselves, it’s probably best to keep them short, but frequent. If you miss one, the next one will be just one week away.
So we see that we can still provide structure and alignment when abandoning sprints, but what about protecting the team’s focus and preventing stakeholders from causing too many direction changes? There is a solution also to that. Both Kanban and Scrumban usually employ a “Ready” or “To do” column, which can serve the same purpose. It should be the only entry point into your process, and have a strict WIP (work in progress) limit. If stakeholders want to have work started on a new item, and the WIP limit has already been reached, they have to select another item that will be removed from the Ready column. Work items that have already been started, i.e., moved past the Ready column, should flow all the way through the process, without being removed or put on hold in between. If you can educate your stakeholders accordingly, and your team sticks to these rules and honours the WIP limits of each column, this will go a long way towards protecting your focus.
Summarizingly, I think sprints of fixed length are a double-edged sword: On the one hand, they provide structure and alignment, and they protect the team from outside diversions. On the other hand, they come with a lot of potential for abuse, in that the sprint end is used as a justification to sacrifice quality in order to slip more features in before it is “too late”. This loss in quality will slow you down in the long run, and worsen developers’ morale.
Being flexible about the sprint duration, or even abandoning the concept of sprints entirely, can mitigate the effects of the artificial deadline. All the while, it is still possible to maintain structure-providing elements like planning meetings, retrospective meetings, or feature demonstrations. Ultimately, agile methods are about experimenting, adapting, and choosing what works best. The possibilities are endless, so let’s encourage each team to find their best fit.
P.S.: After I had put this article live, I realized that I had missed an artificial deadline myself. Usually, I try to publish on Mondays. But between my son’s 3rd birthday and another family celebration last week, I could not find enough time to write, so this article was published on a Tuesday. On Monday, it simply wasn’t complete. I could have taken some short cuts and left some things out, but I did not want to. Publishing the complete, polished article feels much, much better. John Sonmez would probably scold me for not sticking to my plan, but that’s for a different article (link pending, have to write it first) :-)