AI isn't a stable building block
How AI technology is moving so fast, it's hard to keep up
If you spend all of five seconds in the tech world right now, you know everyone is talking about AI. AI has been around for decades in various forms, but the specific innovation taking the world by storm is generative AI. It seems to be single-handedly keeping the tech sector afloat, as many software companies both small and large had been struggling of late. With layoffs and lack of funding becoming the norm, just saying you are building with AI could be your saving grace. Everyone is trying to get a piece of the pie though. It’s basically the wild west, and boy are things evolving rapidly. Many of the big players in the space can kill a bunch of startups with a swipe of their hand by just building a single feature (looking at you OpenAI custom GPTs).
From an engineering perspective, it’s almost impossible to keep up. You cannot build anything before the next thing comes in and makes it obsolete. You end up on version 5 of the same feature, just because there is a new pattern that generates better results, or a new LLM (large language model) comes out that is cheaper. There’s Gemma, Llama, Mistral, GPT (OpenAI), Grok, and that’s just what I can name from the top of my head as of today (there are countless others). And each of those has numerous versions of their models with different numbers of parameters, different training data, and different algorithms. Also, that doesn’t count all of the other generative AIs for other media types like images (DALL-E, Midjourney, etc.), video (Sora, VideoPoet, etc.), voice (ElevenLabs, Resemble, etc.) and others. The possibilities are endless, which is really cool (and also terrifying, but that’s a different blog post). The real question becomes, which of these will be around tomorrow? Which ones will be ceremoniously killed by one of the big companies deciding to build a single feature that does everything their startup does?
One of the most popular Python libraries out there for having an abstraction layer around various LLM calls is called LangChain. Even it is moving so fast that by the time you build on one function or pattern in the library, it gets deprecated by the next minor version. Their documentation barely gets beyond simple tutorials before they have to rewrite them because the underlying abstraction had to change. They haven’t even released their first major version (assuming they are following semantic versioning, which seems unlikely based on their deprecation notices of things being removed in minor versions). It also isn’t really the right abstraction. There are functions (that they recommend in their documentation for certain patterns) for interacting with LLMs that have “openai” in their names. That isn’t an abstraction that makes it seem like I can just change out the LLM for one of Google’s new Gemma models. It might still be better than coding everything by hand, but as an engineer I’m always remembering that:
“Duplication is better than the wrong abstraction” — Sandi Metz
“Avoid hasty abstractions” — Kent C. Dodds
And right now, most of the time, the LangChain patterns feel like the wrong abstraction, though I’ll admit that I’m still very new to the AI world (but aren’t most people?).
As an engineer, I don’t know what to build on. By the time I use a pattern or function, our CTO finds something new that yields better results. Sure, that doesn’t always mean that we pivot to the new pattern, but it results in an unstable foundation. You could build something and have a hard time migrating later because everything about the functionality changed. So you end up using really old versions, even though it’s only been a few months (yes, really). The flip side is just as bad. I could try to keep up with the emerging technology and implement all of the new and shiny tools to understand which one is best, but I’ll never actually build anything. I’ll just be stuck in “tutorial mode” and won’t finish a single feature or product. I’ll be constantly learning the surface layer of the technology, but never get deep enough to build something complicated (where I think you truly expand your knowledge of building software). And even when I do learn something new, that could become completely obsolete information in a month. So what was the point?
The only constant in the AI space right now is that it will change. The rate of change might slow down, but it will continue to change and evolve. Models will slowly become commodotized to the point where cost is not much of a factor. Sure, there will probably be subclasses of models that are targeted at certain functionality, but the model will “just work” and be cost effective. You’ll know what to use, and it won’t change out from under you tomorrow. Whether you keep up with the latest tech, or just wait it out to see who “wins”, I think you are making the right decision. No one really knows where this will end up (though we have plenty of sci-fi films from the past half-century to pull ideas from).