Your Estimates Are Wrong
Estimation is a part of your job, right?
Estimates are a huge part of a software engineer’s responsibilities. You won’t find it in most job descriptions that you apply for, but it’s just an intrinsic part of the job. Admittedly, it’s a part of every job, so if you aren’t in software, this probably still applies to you too. Not only are they part of your job, sometimes it feels like that’s all you do. People just keep asking “how long would it take to do X?” and “could we do Y?”. Wait, that second question doesn’t seem like it is asking for a time-frame. Well, if you’ve ever been asked that question, you know full well that just saying “yes” won’t cut it. Either the person will just assume whatever time-frame works for them (like the sales guy promised the client a week before even asking you), they’ll assume that you can do it alongside your current work, or some other crazy situation where you end up in the wrong. It’s best to say something more like, “Yes, it is possible. It should take about two weeks, and if it needs to be done now, here are the tradeoffs for what won’t get done”. But wait, what if the two week estimate is wrong? That’s where our story begins.
That situation has happened to me a ton of times. Sometimes, I was the one responsible for answering it and giving a time-frame, and other times a manager or more senior engineer has taken the wheel. Most times, the estimates have been off. Sure, maybe my own estimates have gotten more accurate over the years, but they have still almost always been wrong. Usually, it comes down to a lack of understanding. When you have to learn about how to do something before doing it, it can really throw off how long something is going to take. What if the documentation is hard to follow? What if the documentation is wrong and you have to find out the hard way that there is an outstanding bug report on GitHub for that package? What if you discover that you need to upgrade to the next major revision of a package that your app already depends on? That’s a breaking change, so the scope of change just got a whole lot bigger. Probably should have waited to give an estimate. There are a myriad of reasons why uninformed estimates can backfire, but what about the times when “we’ve done that before, so it should be straightforward”? Those are the worst.
Even if you are a very experienced engineer and you’ve “solved this before”, that doesn’t mean it is going to be easy. I’ve gotten to a point where “easy” is no longer in my work vocabulary. There are much better words that don’t paint you in a corner like “simple” and “straightforward”. Easy kind of discounts the amount of work it will take, but simple and straightforward still allude to the fact that it will take some work, but it just shouldn’t be complicated (AKA, fewer unknowns and learning that needs to happen). Even then, it’s a dangerous game. Just because something is similar to a solution you’ve built before doesn’t mean that this time will be the same. You will have slightly different requirements, or will be integrating it with a different part of the system. That means it won’t be exactly the same, and there is at least one unknown until you dive in and start the work. That unknown could balloon to an extra week’s worth of work. Engineers talk a lot about patterns and how they make them more efficient, but they aren’t a silver bullet. They will be useful up until they paint you in a corner that is even harder to get out of than if you had just solved the problem a bit differently from the beginning. That’s why no two situations are entirely identical.
Where estimates can help is by putting a deadline on the work. From a productivity perspective, putting a deadline on things can help make you work more efficiently. It also helps you properly scope the work (more on scoping in another blog post, but that is tightly coupled with giving estimates). When you know that something needs to be done “by Friday”, you have an idea of what parts of the work need done on Tuesday to make sure you stay on track. It isn’t a perfect system, but one that you get better at with time. It holds your feet to the fire to get it done on time, and possibly work extra to not break your promise. Again, not perfect, but it can be effective. The problem lies in that it can lead to cut corners just to meet the deadline. That means a crappy quality product, and therefore, what was the point? Sure, it is done on time, but now you’ve lost the trust of the customers asking for it. A more realistic estimate from the very beginning would make everyone happier.
I read a blog post from DHH (look him up) about this about two months ago that touched on estimates a bit, Your Estimates Suck. It isn’t what inspired this post, but I would be remiss to not mention it (I actually wrote the title for my post before remembering his). In it, he basically said to stop thinking about estimates first and think about scope (AKA “appetite” in Shape Up terms), and I’m starting to come around to that way of thinking. That starts with getting the rest of the team on board, which you don’t always get a say in depending on your position in the company. However, that doesn’t mean you shouldn’t try to keep making things better.