Biases create Software Effort Mis-Under-Estimations (part 2)Posted: March 16, 2011
Last week’s post about misunderestimations (part 1) considered the practical difficulty of estimating software effort. Uncertainty of estimates is created by complexity and the infamous, but poorly understood, “unknown unknowns”. The uncertainty is too often ignored.
Consider the best way to estimate…
Team Lead: I need to know precisely how long task X will take.
Programmer: First, I will do the task. Then I’ll get back to you with an estimate of how long it took.
Since we can’t rely on hindsight we are stuck with foresight and this is susceptible to many irrational and emotional biases: pride, competition, leading questions, peer pressure, conditioning, poor communication amongst many other effects. These can wreak havoc on the accuracy of estimates even, in my experience, for rational programmers.
This post explores these biases and how they lead to misunderestimation and its rarer sibling, misoverestimation.
What is a “good estimate”?
First, the objective needs to be clear. What makes a software effort estimate better?
Roughly speaking, there are two complementary perspectives of a “better estimate”. It can mean a “lower estimate” usually because “lower” translates to quicker, faster and lower cost. Alternatively, it can mean a “more accurate estimate” because there can be real costs to either under- or over-estimation of software effort.
For this post, the focus is primarily on estimate accuracy. A good estimate should be realistic and achievable given the stated requirements for scope, quality, computational performance etc.
My conclusion to the earlier post was that being accurate requires providing a realistic range. For example,
“I estimate that Task X will take between 3 and 5 days assuming I can work full-time on it and there are no changes in the requirements.”
The uncertainty will increase with complexity, unfamiliarity and other factors that introduce unknowns.
Less obviously, that estimate range will be shifted by a range of behaviors and personality biases.
Too often, the best-case outcome within the estimation range is used as the estimate. In the example above a 3-5 day range might be watered done to a 3 day estimate.
This is almost always results a misunderestimation: the best possible outcome is not usually the actual outcome. In reality, as discussed in Misunderestimations (part 1) the actual is more likely to be nearer or even above the worst case estimate.
To be clear, I think we should always be looking for quicker and better ways to implement tasks. However, lowering an estimate should be about finding a better way to perform the task, not simply hoping for the best.
Peer Pressure / Perception Bias
How will the following effect software estimates?
(a) Ask a programmer to provide an estimate in front of his/her colleagues
(b) Ask a programmer to provide an estimate in a 1-on-1 discussion
(c) Ask a programmer to provide an estimate by private email (or similar)
Most programmers I ask say that their estimates will be lower in case (a) with a team context and more realistic in case (c) where they are less sensitive to the perceptions of others.
Part of the mental calculus in providing an estimate will include “what will others think of my estimate” and/or “what does it say about me”. Nearly always this will result in a reduced estimate because lower estimates say “I’m a better programmer” or perhaps conveys confidence or pride.
The bigger the audience the greater the potential effect and the more likely that a team member will challenge for a lower estimate. And, in my experience, perception bias can be felt by both junior and senior team members.
Do perceptions matter? Yes, but it is important to work towards estimates that are realistic by working against perception bias. Otherwise, it can create a positive perception in the short term but problems in the longer term by finishing late once or often, by working crazy hours to meet unrealistic deadlines, or through loss of trust or respect individually or for the team.
There is an important “BUT” on estimations in a team context. When the team dynamics are positive, group discussion about the complexities and risks of a task can be a great way of improving estimates by exploring potential problems.
The “Project Pre-Mortem” is a great process for some projects. It involves carrying out a project post-mortem before actually starting the project. This process is based on the recognition that teams are often aware of the potential causes of failure/problems before we start but can be reluctant to discuss them.
Compare the following two estimate requests.
“How long will it take to complete Task X? We need to start tomorrow morning.”
“How long will it take to complete Task X? We will be starting the work in about 6 months.”
Work to be done in the future will almost always be discounted resulting in a lower estimate. If start of work is imminent then there’s a greater incentive to think it through harder, to analyse the task in more depth, identify risks and so on. [There's a psychological term for it that I can't remember. Does anybody know it?]
One way to mitigate this bias is to estimate everything as if it starts today. If you’re the person requesting the estimate then perhaps plant the “starts tomorrow” idea in your request.
Most software estimates involve unknowns so we don’t usually have the luxury of estimating with full knowledge of what lies ahead.
This means it can be necessary to make assumptions:
- Interpretations of incomplete requirements
- Assumptions about existing code that you will be working with
- Guesses about the number and severity of bugs that you’ll stumble across or create
- And many more
As programmers we are specialists doing what we’re good at. It is reasonable to make “educated guesses”. It is worth noting these conscious assumptions as we make them.
But assumptions can be made subconsciously and these might only become apparent in hindsight. Still, hindsight gives insight for learning.
If, for every assumption or unknown you assume the best, then odds are that you have misunderestimated. Conversely, it’s not usually reasonable to assume that everything will go wrong even if you subscribe to Murphy’s Law.
As a sweeping generalisation, the Assumption Bias for most programmers tends towards misunderestimation (again).
Virtual Reality Bias
“The customer needs us to build a Widget. I don’t have much information but I need an estimate so I can tell the customer how much it will cost.”
If you don’t know what you are being asked to do then it can be hard to estimate.
In the absence of specifics it is tempting to invent a Virtual Reality spec. I have no idea whether this is a positive or negative bias or just simply wrong.
The classic tree swing design cartoon comes to mind:
On-the-Spot Estimate Bias
Somebody: I need a rough idea for how long Task X will take. It’s just for initial planning and you won’t be held accountable.
Programmer: OK. It will take about a week
2 weeks later…
Somebody: You said it would take about a week. We’re already a week late! What’s wrong?
Programmer: <<anything said can sound like an excuse>>
Estimates have a life of their own. In my experience estimates (even “wild guesses”) travel further and faster than expected.
Sometimes is it acceptable to decline making a quick estimate. If that’s not an option, be careful of a misunderestimation because quick estimates are particularly vulnerable to the Assumption Bias. It’s also entirely reasonable to follow up the quick estimate with a considered estimate, or a request for more information, or perhaps a CYA email.
It is not unusual or unreasonable for someone asking for an estimate to have a prior expectation. What can cause significant bias is a request like this.
“I think Task X will take 10 days. How long do you think it will take you?”
By stating a clear expectation in the question, the answerer’s thinking is conditioned.
It works like infomercials which will say “Normally the Stair Blaster sells for $200 but for today only you can get it for just $60″. The viewer is supposed to think “Wow! $60 is so cheap” even though the product has probably never sold for $200.
Conditioning works, even for very rational thinkers. In the estimation example the “10 days” sets an anchor point that affects thinking about the estimate. If the programmer is thinking “I thought this would take more like 20 days” then there’s a natural inclination to bring it down. The reverse can happen too, but I find it less common.
Let me be clear, not every instance of conditioning is manipulative or even a conscious decision. But it’s a good pattern to avoid if you’re seeking estimates and best ignored if you are providing estimates.
There is a time for the requestor to talk about their expectation, but that’s best left until later in the conversation.
A customer expectation of time-to-complete is a form of conditioning that can be particularly compelling. If asked:
“The customer wants this change by Friday. Can you do it?”
The thinking might be “it’s possible if the planets align” but the optimistic programmer or the one that likes to please says “possible”. Resist the temptation!
If there’s going to be a delay then it’s almost always better to know earlier not later — pain delayed can be pain increased.
Programmers are not generally hired for their commercial savvy and dealing in dollars is not often part of the job description. But programmers working in business are a cost and when selling software to customers (esp. custom software) effort estimates can be central to pricing.
“We charge your time at $1,000 per day. How much time will Task X take?”
Is the billing rate intended to affect the work time? Do you want a low estimate to keep the price down or a high estimate to make more profit?
These considerations are best separated from effort estimates. How long something is expected to take and how much to charge for it are best treated as separate considerations.
There’s a difference between asking somebody for an estimate when they either love or loathe the task at hand.
Working on something you don’t want to do is part of pretty much every job. The professional thing to do is just to get it done. Uninteresting tasks usually take longer to complete (especially if procrastination kicks in) but just occasionally it gets done faster simply to get it out of the way.
In my experience estimating de-motivational tasks usually produces higher estimates – sometimes much higher in an attempt to have the work assigned elsewhere.
Alas, the reverse can apply. When somebody desperately wants to work on a job they are prone to misunderestimation. This creates an unreasonable expectations (both for the programmer and others) and can turn otherwise exciting work into a stress-fest.
No discussion about estimates is complete without introducing Parkinson’s Law.
Work expands so as to fill the time available for its completion. (C.N. Parkinson, 1955)
Those responsible for the cost of software projects — team leads, project managers, managers etc. — need to find the balance between seeking lower costs (less effort) and on-time delivery (amongst other factors).
With Parkinson’s Law in the back of the mind, it can seem reasonable to seek lower estimates. Looking for faster ways to complete a task can sometimes result in creative solutions when the Procrastinator’s Creed kicks in:
Any job left long enough will result in the quickest method being the only remaining option.
But very often unreasonable deadlines lead to unreasonable stress, long hours and often poorer outcomes of quality and staff satisfaction.
When pushed, a programmer can respond by invoking Hofstadter’s law.
A task always takes longer than you expect, even when you take into account Hofstadter’s Law. (Douglas Hofstadter, Gödel, Escher, Bach: An Eternal Golden Braid 1979, ISBN 0465026567.)
Recursion is a wonderful thing. But I might be biased about that.
Andrew H – Psygrammer