Documentation is Hard

How I think about documentation for engineers

Documentation is the first thing to get put off. You work so hard on a feature and you barely get it done on time. Then you have to move on to the next feature. You didn’t have time to document the implementation details. Sure, you have all the context now and remember how it works. What about six months from now? It might take you a little extra time to remember, but you’ll figure it out, right? It’s a problem for another day. That day comes, and the next feature is now so much harder. What makes matters worse is that you aren’t even the one working on it. One of your fellow engineers has been tasked with some improvements to that feature you built. Now you have to communicate how it works, because well, you didn’t have time to document it. It’s a disaster, but is often the reality of software development, especially in a startup.

Startups are famous for moving fast to be first to market, and/or just trying to get product-market fit before running out of money. You put off a new feature for last minute bug fixes for an investor demo, but the due date of your feature doesn’t change, so you just lost a day of development. Now, see previous statement about documentation being the first thing to go. You don’t have time. Hopefully you can at least unit test some of the brittle logic for that complex filter query. The best you can do is comment some parts of the code that are weird while you are writing it and move on. You’ll have to figure it out later, or plead with your leadership team that you need that day back. You may be able to negotiate that, and if that’s the case, that is awesome, and you should make sure the documentation is worth while. Documentation can end up being stale soon after you write it, particularly if it isn’t colocated with the code. Someone changes the function, but forgets that the documentation is in a separate docs folder (not sure why that was ever a “good” pattern, but here we are). Even if the comment block is colocated with the code, if it isn’t kept up to date, then it’s basically wasted time, energy, and space (looking at you webpack).

It all comes down to trade-offs. In a perfect world, you would have all the time to engineer the perfect solution that scales, write extensive test cases, communicate a high level explanation, and thoroughly document it. The reality is much less rosy. You have a tighter timeline than you need. You won’t even have the time to get it fully tested. You were planning for large scale, but the time constraints mean you can really only account for 2-3x the current volume. The bright side is that if the feature isn’t quite what the customers wanted, then you didn’t waste too much time. It’s a trade-off. Sometimes you have to put your business hat on top of your engineering hat. You put aside the perfect solution for “good enough for now”. You’ll pay for it later, and sometimes that’s okay.

I’m not going to say “now here’s how to avoid that”. Honestly, I don’t even think you always need to avoid it. However, I think there are a few ways you can prevent a disaster. The primary way is to document as you go as much as possible. I’m a big proponent of writing readable code, meaning you shouldn’t need to add a lot of comments to it. However, if there is something you wrote that is a bit weird or complicated, you need to document it. This has saved me many times in the past. Using a hashmap to improve the runtime performance of this complicated algorithm that would usually just be a simple couple of loops? Document that shit.

Here’s a real world example. There was a senior engineer I worked with when I was an intern at my fist job. He had a famous block of complicated code that everyone used. It was doing some interesting things with the arguments object in JavaScript to parse function parameters. His comment paragraph started with something along the lines of “so I’m using some deep magic here”, and then continued on the explain the solution in great detail. Fantastic comment, and perfect use case for it (and also giving the engineers that stumbled upon it later a good laugh). Even as a budding engineer, I could understand what it was doing (and the example stuck with me). Documenting a for loop to say “iterating over the array of strings” is completely unnecessary, especially in the days of Typescript, where my IDE tells me the exact structure of that variable.

In my mind, the most important thing about documentation is that it should make your code more readable and save you or your teammates’ time later. If it doesn’t accomplish one or bettter yet, both of those things, then it is a waste of time. Now you know what it’s like to be inside my brain. That’s the business side making an argument against the engineer side. The engineer wants to build the perfect system that is well documented (HOW), but in the same breath, if it needs a ton of documentation then it is probably too complicated. It should be simpler, unless you are inventing something entirely new (looking at you RFCs). The business person really just wants to have something to sell to make money (WHAT). I think considering both when building software is very important, but the ultimate goal is just to solve a problem for someone (WHY). Notice how I tagged those sentences. Both sides are just trying to accomplish the WHY, but they get there a little differently. The engineer is mostly concerned with HOW the product gets built, and the business person wants to know WHAT can be sold/marketed. The goal is the same, and as an engineer, considering the WHY in your day-to-day will result in the best outcome, even in documentation.