I tried GitHub Copilot
An experienced developer's perspective on using AI for writing code
I need to start off by saying that I received the product for free…through work. I wasn’t required to use it or anything, but we are working on AI solutions, so it’s good to see what is out there. I’ll be honest, my expectations were pretty low. When all the recent AI craze started taking off, I was skeptical at best. But I’ll also admit I was being naive. I didn’t really understand the technology yet. I was judging a book by its cover. Shame on me. I’m not going to say that I’ve become an expert since that moment, but I at least have a better understanding of what large language models are offering.
Language is complicated, but our most important invention. It’s evolved over tens of thousands of years and multiple species of humans. We have dead languages like Latin, Sumerian, and Ancient Greek that are no longer spoken. They’ve been immensely important stepping stones, and were spoken by nations of power throughout history. As of today, there are over 7000 spoken languages, but only about 4000 of them have writing systems. That is still an incredible amount given there are less than 200 countries. Most people speak at least 2 languages, unless you are in the US. Some speak upwards of 7 or more. While most large language models have only been trained on some of the more dominant languages, for it to generate language that is legible and coherent is no small feat.
None of those numbers include all of the programming languages that exist, and this blog post is mostly about GitHub Copilot, which helps write code, not just natural language. As we’ve talked about before, code can be a universal language, which adds a whole other layer of complexity.
Enough of the intro, let’s get to the center of the tootsie roll pop. I have mixed feelings about it. When it comes to writing boilerplate, duplicating patterns, or refactoring into a different pattern, the generated code is pretty decent. It is a performance improvement. As an addition for auto-completing code, it can do fairly well, though it will sometimes suggest invalid or just duplicated code. When it comes to rapidly copying console logs for debugging a bunch of variables, it can be slightly faster than copy pasting, but that’s a pretty thin use case. Refactoring or seeing how a chunk of code would look in another language is where it really seems to shine. That is where the workflow improvement comes in.
A prime example where it saved me a ton of time is in a refactor for some React code. I had written a bunch of state management code for updating text on the screen based on the response from an API call. I was having a timing issue with the way the state was getting updated multiple times, because I was streaming back a response. I’ll save you the nitty-gritty details, but really it is just a common React problem with a standard solution. I didn’t realize I was going to run into that problem until I had most of the code written. When it didn’t work, that’s when I realized, “crap, I need to rewrite this to use the reducer pattern”. Not hard, but also not a trivial amount of boilerplate work. I decided I’d try to see how well the AI could do at refactoring it for me. Boy was I blown away. I didn’t have to modify a single line and it “just worked”. My problem was fixed.
I do find the code suggestions distracting sometimes, particularly when I’m writing new code that is still kind of exploratory. It will suggest something based on what I’ve written so far, but that isn’t where I was going. That sidetracks my brain and I’ve actually had to think “wait, what was I writing again?”, which is very frustrating. That could mean that I need to name my function better, but when I’m writing net new code, I don’t always know exactly how it is going to work until I test out a few things. This is also fairly prevalent with configuration options. Whether it is for classes or a options object for a common function, it will suggest a bunch of “default” options for you that may not actually be remotely what you want. It kind of can make you lazy, especially if you haven’t looked at the documentation enough to know which options you need for that use case. You see some stuff in the code suggestion that “looks decent” and tab complete it. Then when nothing works, you are stuck removing each of those config options and reading the documentation anyway. Waste of time.
I will say that I didn’t change my entire workflow to let it write code for me. I still personally want to write code. It is just an installed extension in VS Code. I haven’t gone through all the configurations and setup to make it perfect for my workflow. I’ll admit that maybe I would see an improvement in the types of work I want to use it for. I also haven’t built a bunch of snippets for boilerplate code that I could just use instead. I just never got into that world. If I did, I would probably use Copilot even less.
At the end of the day, I don’t really trust it. If I let it do everything and then there was a critical bug in production, I wouldn’t know how to debug the code. I didn’t write it. I don’t know where the skeletons are hidden. Sure, I could go through a rigorous review process, but that might take me longer than if I just wrote the code myself in the first place. In the reducer example, I have written code for that pattern before, so I knew what the code would need to look like. I knew where the bugs would generally be, and I could properly review the code. This made it worth it. I was comfortable “owning” that code. From time to time, I’ll continue to use Copilot for these refactor use cases and duplicating a bunch of console logs for the variables I want to print out while debugging. But for core development? I’ll pass. I’m personally still learning a lot about software development, and want to continue doing so. I learn best by doing, so letting it do the work for me won’t really teach me as much.