30 January 2024

“If a sales rep doesn’t make their targets they get fired: why aren’t we firing engineers who miss delivery dates?”

I’ve heard a version of that sentiment many times, usually accompanied by literal table thumping by a senior sales leader 😬.

Business is all about taking bets and managing risk. When creating software products, delivery risk - the risk of “missing a delivery date” - usually sits at the top of the risk register. When I’ve worked in early stage software businesses with high growth expectations, a lot of power rests with the “whales”: prospective deals that might be the difference between a party on a boat with an open bar, or holding a hat to investors for a “down round”. In established B2B businesses where dedicated sales teams carry quotas, someone with a nice watch is always at my door looking for promises to get deals over the line and beat out competitors. When shipping some more product looks like an advantage, it’s suddenly important to predict when the software will be delivered.

Someone in a software business who doesn’t have a software product development background (the ones with the nice watches) might wonder why it is so hard to estimate how long it will take to deliver a new product capability. There are plenty of books written on the subject, but I think it boils down to the inherent uncertainty in creating software and the relationship between that uncertainty and time.

If a business lacks the language to talk constructively about setting expectations for software delivery, fingers are pointed, blame is laid, and ultimately the health of the products and the happiness of the customers suffer.

Inherent uncertainty

I can estimate with a high degree of confidence how long it will take me to drive from my house to the airport at a particular time of day. There is some risk that traffic incidents or car trouble could influence the journey time, but my house is always in the same place, the airport is always in the same place, and I take the same route at the same speed. Unfortunately that level of certainty is not usually possible when building software products.

To say we “build” software is an understatement. Creating software is designing and building. No matter how many standards or conventions are followed, or how many libraries are used within opinionated frameworks, there are myriad design decisions to be made at every turn. Product leaders make design decisions, designers make design decisions, architects make design decisions, and developers make design decisions. Estimating effort for a software project is equivalent to estimating how long it will take to drive a completely unfamiliar route in a strange country. You know that you’ll be driving a car and roughly how fast you’ll be going, but the rest is a bit of a mystery until you are well into the journey!

Product leaders, designers, architects, and engineers also make mistakes. It’s very easy to fail to take into account all the “extras” that come smashing through best laid plans: security reviews, meetings with go-to-market agencies, educational content… the list goes on.

Before the agile revolution took hold there were many attempts to make the design of software a separate set of activities so the building phase was more predictable (aka “waterfall” development). These attempts generally failed because of the difficulty of translating user requirements into a specification that was correct and complete enough to build. That problem hasn’t gone away, but in modern practice we collaborate with users and build in small increments. This improves the ultimate correctness and completeness of the solution, but it doesn’t really reduce the overall uncertainty of the effort required and the elapsed time that effort will take.

Targets and commitments

The most common cause of frustration and source of division that I’ve seen between people in customer-facing roles and product-creating roles is ambiguity in the “flavour” of dates. Whenever dates are discussed, it’s essential to be clear about whether they are targets or commitments:

Target:

• Provides a timeframe to work to when planning
• Keeps a level of flexibility - e.g. “Q1” or “tentatively October”

Commitment:

• The whole team is reasonably confident it can be met
• If it is at risk, mitigations like reallocating capacity from other projects can be used

Barry Boehm popularised the use of the “cone of uncertainty” in software engineering. The gist is that early in a piece of development work, the level of uncertainty is very high and the accuracy of estimates varies significantly. The uncertainty is driven both by what we know we don’t know and also what we don’t know that we don’t know (with apologies to Donald Rumsfeld). As time goes on and completion of the work nears, the uncertainty shrinks and the estimation accuracy improves. Unfortunately that means that when commercial opportunities that depend on product changes are being evaluated and negotiated, uncertainty is at its highest!

Early in a piece of work it makes sense to treat dates as targets. As requirements are resolved, technical solutions are found, and the delivery risk is lowered, it becomes feasible to make commitments. Product and technology leaders have to be absolutely disciplined about reminding stakeholders of which flavour of date is being discussed on each and every date. They also need to be disciplined about not committing until it is reasonable to do so:

The importance of estimating using ranges

I introduced this article with a note on delivery risk. A key delivery risk is that predicted delivery dates might be missed. When an inexperienced developer is asked for an estimate on when a piece of work will be delivered, they will often respond with a date: aka a “point estimate”. Estimating delivery on a specific date is the equivalent of asserting that an estimate has 100% confidence, but it is nearly impossible to do this. The degree of uncertainty attached to any material and potentially valuable piece of work in software is always high - otherwise it’s probably not worth doing! Providing a single date is the equivalent of ignoring the risk of inherent uncertainty altogether.

I remember the “a-ha!” moment of reading Steve McConnell’s “Software Estimation: Demystifying the Black Art” when it was published in 2006. I was in my first software engineering leadership role and a significant part of my duties was estimating when features would be delivered. In the introductory chapters McConnell poses a simple exercise. He asks the reader to estimate a minimum and maximum value, with a confidence of 90%, for ten difficult-to-guess items, e.g. the surface temperature of the sun and the area of the Asian continent. In the stats he has collected from running this exercise with hundreds of software professionals, the average number of correct answers is 2.8 out of 10. McConnell contends that this means on average, despite being asked to estimate with 90% confidence, the average test taker is really only 30% confident! He notes that once this observation has been explained to first-time test takers, on a second attempt they will widen their ranges and score much closer to 90%.

I have seen stakeholders outside a product development team apply pressure to provide point estimates on many occasions. This is usually with the best of intentions: commercial opportunities live and die on dates, and a sales person facing a customer (who in turn is worried about dates in their own business) likes nothing better than to be able to provide a “concrete date” on which a piece of work will be delivered. Product and technology teams yielding to that pressure are asserting 100% confidence in a single date when a date range is the right answer. As Steve McConnell illustrates, to get to even 90% means that range will need to be wide. The middle of the range might be a reasonable guess for a date, but the commercial opportunity needs to be set with the expectation that delivery may not be until the end of that range. Even then, there is a chance that delivery will be later still: a 10% chance in the case that 90% confidence is correct. Until delivery time draws near, it needs to be a target, not a commitment!

Wrapping up

By being disciplined on how we talk about dates, it’s possible to strike a balance between setting timing expectations with prospects and customers, and the realities of uncertainty when creating software. Inevitably there will be times when those dates are disappointing. When that happens, take a moment to remember how tough it is to “just go faster” with a quick read of my earlier article: “A new iron triangle” 😀.

If you would like to tame your dates and separate targets from commitments: I can help! Read more at cronin.nz or drop me a line at gareth@cronin.nz.