Chapter 4: The future of AI

Initially, I didn't want to write this chapter. With how fast and unpredictable everything is moving, it's really easy for any of your predictions to be way off, at least in their timing. But I'm keeping it for two reasons: first, I want to document what I'm thinking right now and reflect on it in the future; second, I want to talk about expectations about the future that can affect our current lives without necessarily being true.

The AI university

Even though I have frequently opposed replacing specialists with AI in this book, I can imagine that I might be wrong and consider what the future could look like with the current pace of AI.

One of the things we discussed in "What should you learn for vibe coding?" is that it's really hard to know what AI is doing, and we don't have any way to verify it. This is one of the things that could theoretically be solved if we introduced highly professional university majors for it. Take a look at fields such as sociology or even graphic design. Even though these fields definitely use some scientific methods (just as vibe coding does), they are mostly based on experience and outcomes. In The outcome bias, I explained why it's logically wrong to judge based on outcomes. But sometimes, you simply can't judge at all using pure logic. In a society, there are too many factors to make everything make sense through mathematical logic. But you need sociology anyhow, so it's justified to resort to personal experiences but in a professional way.

We mitigated this problem by introducing a new method: we find mathematically sound ways of evaluating outcomes (random sampling, for example), gather a ton of them, and then use philosophy, logic, statistics, psychology, and more to form opinions. Since we don't have an objective way to verify our results and can only be so certain, we never treat sociological statements as objective facts. We should never trust a sociologist's statements in the same way we trust the statement that "gravity exists." This by no means undervalues sociology. History will tell us whether we should trust a sociologist or not. In the same way, history might prove my prediction about AI correct, but that doesn't mean I was objectively correct beforehand. I made an educated guess, and it turned out to be true. Sociology, graphic design, and other similar fields are just collections of educated guesses.

This is what could possibly happen to AI engineering. Since we don't have objective and deterministic verification methods, what we should do is invest in this field and make it a university degree. Instead of expecting people to do all these tests on the models, we do them and use mathematics to make sure these tests aren't biased and are being done correctly. Then we make rules from them. Then we shape these rules over the course of humanity to make the smoothest development experience.

This could also be why we—as software engineers—are struggling to adapt to AI. We are expected to suddenly gain years of knowledge while no one knows anything. It's like going back thousands of years and expecting people at that time to explain the common sociological and psychological concepts. The reason why sociology and similar fields work this perfectly today is that they have been perfected over the course of thousands of years. Programming in general is less than a hundred years old.

The false expectations

The thing is, even if AI is inherently unreliable, its extreme resemblance to human beings has given—and will continue to give—many people, managers, and CEOs false hopes and expectations.

1. Lack of seniority

Managers stop hiring juniors and stop investing in them. Seniors retire, and at some point we'll have a shortage of senior developers. As simple as that.

2. Technical debt

Though other forms of debt also play a part, technical debt is more visible. A lack of seniority, hallucinations, and the use of weaker models because of AI costs could accumulate debt and cause a lot of problems in the future.

3. Ruining the industry for a couple of years

Ultimately, we can easily ruin the industry for at least a couple of years. There's no stopping that, because the issues mentioned in this book aren't immediately visible. They show up only after they accumulate. Hopefully we can stop that sooner. That's why I highly suggest that you speak up about these topics.

Software becomes dead

If despite my understanding, the only barrier to SWE becomes the idea and everything else is done by the machine, then the market becomes so saturated that software becomes practically dead. Why would you buy software when you can build it yourself? Intellectual property would mean nothing, because you can take the genius idea that someone made an app for, and make it yourself from scratch, fine-tuned to your own needs as well. If you don't show it to anyone, you haven't done anything immoral or illegal.

Elon Musk posted a tweet a few weeks ago saying that programming languages will be dead in 10 years and AI will write machine code directly. While it caught everyone off guard and he was made fun of, this is what AI maximalism looks like. When you say you offload all coding to AI, you're expressing a similar expectation. I'm not saying that if offloading code to AI turns out to be the correct way, then AI will necessarily write direct machine code in 10 years. But the same prediction could be made about claims such as "in the future, operating systems will become obsolete and everyone will build their own AI-native OS." I actually posted a bunch of these predictions generated by ChatGPT in a tweet. Really funny to read.

Civilization collapse

There's a very thoughtful video by Jonathan Blow called Preventing the Collapse of Civilization. In this video, he talks about how technology decays on its own and requires constant work to actually improve. Thousands of technologies have been lost throughout history for this reason. A lot of advancements in science and technology have been reduced to nothing. It took us many years to understand how the Pyramids of Giza were built, and we're still not sure about many aspects of them.

Software is already decaying. This video is from 2019, way before ChatGPT's public release. One thing he said that I can't get out of my head is that people say, "Yeah, we could make this software better and less buggy, but the market doesn't pay for it," but how do we know whether they actually could? When it has been decades since robust software was made, what makes us think that we still have the knowledge? Knowledge has to be constantly transmitted and honed; otherwise, it rots and decays. I, as a programmer, can say that I can write better code, but the fact is, the more you stop doing things the correct way, the more you stop learning and evolving, and the more your knowledge expires. I may be technically able to write much better and more robust code, but if that takes me 100 times longer than it should, then I probably don't actually know how to do it anymore.

Let's say I learn something and put years of my life into understanding it. Then I explain and teach it in simple terms to younger generations and other people. They can start where I left off. They haven't filled their minds with irrelevant data. They can start much further ahead. The reason you can speak a language so fluently is that it has been perfected and transmitted for many years. You don't have to waste so much time creating new concepts and words just to communicate with the people around you.

In the same way, if I learn and practice how to create better software, the norm changes over time. The knowledge that I have no idea about right now will later become muscle memory. Even if the market doesn't pay for it, that doesn't change anything, because my productivity increases.

That's the paradox of using AI. We think we have increased productivity, but if we have removed all parts of this experience and are just producing, we're eventually going to lose our knowledge. Even if you don't lose it, you'd lose your grip on it; your flaws would increase, you'd stop being sharp, and, when accumulated, this would result in decreased productivity. Terence Tao, perhaps the biggest mathematician alive, has a video where he talks about this paradoxical nature of using AI for work. I highly recommend watching it.

Moore's law

There's something called Moore's law. Gordon Moore said that the number of transistors on a computer microchip doubles roughly every two years. This means that the size of chips and CPUs constantly shrinks. This law continued to hold true until around 2013, when it was estimated to become obsolete. But a brilliant group of scientists managed to pull off the impossible and invent a new technology that saved Moore's law. If it weren't for the constant effort of scientists throughout history and the investment of hardware companies (who needed their products), we wouldn't have had this technology. Imagine if no one had invested in it and the stagnation had continued until the decades-old papers that made this technology possible were forgotten or, even worse, somehow deleted (don't forget the Library of Alexandria). This knowledge could easily have been lost to history, as many other technologies have been.

The apocalypse

Well, we already talked about it. There's no theoretical barrier to an AI apocalypse in my opinion. Who knows? It might happen in our lifetime. It's the ultimate gen z experience at this point.

The good prediction

If the bad predictions don't come true, the golden age of indie development (especially in fields with high barriers to entry, such as game development) will become a platinum age. Good things that simply couldn't be made back then will be made now. And the good stuff that existed before would become even better. AI democratizes the industry in the same way it already has.