Software Estimation accuracy
In my career as a Software Engineer, there were numerous times when I had to give an estimation on the efforts needed for the project or task ahead. It is one of the skills that distinguish engineers from one another. It is arguably one of the most likely discussions that you would have with your project managers and leadership in general.
And yet, there is very little focus on training people with what seems to be a very nice skill to have. Talking from experience and the discussion I have had with people from the field, it seems to be a common occurrence that in small to mid-sized companies it is very hard to get an accurate estimate when a project is longer than a few months.
Software estimation is hard
There are numerous studies[1][2] that show how companies fail their estimates time and time again. And it is not by a small margin either. Sometimes projects miss their target by 100% or more. The bigger the project the bigger the error margins get.
Also speaking from personal experience I have been involved in projects that have gone way over their initial estimation even with our best intentions. So have some of my peers that are working in the same field for many years.
Software estimation is hard. Then there are a lot of factors that make it even harder. Among them: unknown domain, company growth, employee churn, discipline discrepancy... Those are just a part of the things that influence the outcome of a project.
So what can you do?
There are people that have dedicated a large chunk of their time to this problem. Some companies specialize in estimation consulting. There are also books[4] that have been written on this subject (part of my inspiration for this post) that explain in detail why what and how to deal with software estimation. I highly recommend you check those out if you want to gain more insights into the subject.
I would like to share some of the insights from those resources that I found most interesting and that could help you towards better outcomes:
Always try to overestimate instead of underestimate. The argument here is that overestimation is something that can be fixed with proper project management. There is a knee-jerk reaction to overestimating as in "precious time has been wasted". But the costs of overestimating are much more predictable because there is a limit to them. The limit is the date of delivery. But underestimation has no cost predictability to it. The costs could potentially be endless since there is no guarantee on how bad a project can go.
Sometimes we feel the need to give an underestimation. Before you do, ask yourself why? Quite often a lot of people have a self-imposed pressure to underestimate projects without anyone expecting that of them. That comes from the ego, different setting experiences, the fear of "If I give a large estimation, we won't work on this project" and other factors. This requires some self-work to push aside the ego, adjusting to the current settings of the company and sometimes accepting that some things that we want to work on are probably not the things that we should work on at a particular time.
Very often, even unconsciously, we want to tweak things (feature creep). It's just a thorn in our eyes that we didn't deliver a requirement exactly as it should be so we just want to squeeze in one more thing, color change, a more efficient algorithm available, another language support. What people don't realize is that on a multi-month project those things add up. And if you are not careful with guarding your estimation it can get out of control really fast. Transparency across all stakeholders is important to deal with this issue. As more people are aware of the consequences the easier it will be to weigh in the pros/cons for those so that you can add/remove project requirements.
We have trained our brains with "wrong" defaults. What I mean by this is that sometimes people spend a great deal of their time training their gut feeling to estimate in a certain way, but they have since moved to a different setting. My favorite "wrong" default was when for the "engineer week" unit I would count it as 7 days when instead it should be counted as 5 workdays. That is a big gap in the long run.
Make your colleagues aware that software estimation is hard. Sometimes people just don't know. And when they don't know, their expectations might be off by a lot which might influence you. Be it from a manager by expecting impossible deadlines or a team member giving inaccurate estimations.
Conclusion
Being accurate in software development is a really good addition to a particular business. But we have to be realistic with what can be achieved given the parameters that we work in. Sharing this knowledge with your company will give the opportunity for better outcomes both for the business and yourself.
Links:
https://www.standishgroup.com/sample_research_files/CHAOSReport2015-Final.pd
https://www.amazon.com/Software-Estimation-Demystifying-Developer-Practices/dp/0735605351