When AI Starts Building Itself • Blog by Hafid

A personal note about AI, from a paper published in 2017 to the questions that sometimes keep me awake at night.

A quick note on the visuals below: they are sketches to follow the essay, not proof of anything. Anything labeled hypothetical is exactly that — hypothetical. The one chart is deliberately schematic: no measured values are plotted.

I don’t remember exactly when I started thinking about this.

Maybe it was the first time I casually typed something into ChatGPT and ended up staring at the answer for a few seconds.

Or maybe it was when I started using Claude and realized that, sometimes, the code it wrote was cleaner than mine.

Whatever the exact moment was, something changed after that. I started paying more attention to what AI could actually do, not just what people said it could do.

And the more I looked into it, the harder it became to ignore one question.

What happens when AI becomes good enough to help build the next generation of AI?

It started with a paper

If I trace everything back far enough, I always end up at the same place.

“Attention Is All You Need.”

At the time, the title probably didn’t mean much to me. I didn’t even read the paper when it came out. It was just another research paper somewhere on the internet.

Years later, I finally read it.

That’s when I realized how much of today’s AI ecosystem can be traced back to the Transformer architecture introduced in that paper. The architecture was based entirely on attention mechanisms and removed the need for recurrence and convolutions in the original sequence-to-sequence setup.

It’s strange to think about.

Something that looked like just another research paper in 2017 ended up becoming part of the foundation for the models we interact with almost every day.

And now I keep wondering where the next step comes from.

The development loop is changing

For most of AI’s history, humans were responsible for almost every step of the development process.

We wrote the code.

We designed the experiments.

We ran the experiments.

We looked at the results.

Then we decided what to do next.

That workflow is slowly changing.

Anthropic recently described a progression that I found particularly interesting. It starts with humans doing the work themselves, moves through chatbots and coding agents, and eventually reaches autonomous agents that can run code and delegate work to other agents. The next hypothetical step is an agent capable of building and training its own successor.

I don’t know if we’ll actually get there.

But the direction is interesting enough to make me stop and think.

Figure 01 · Timeline observed emerging hypothetical

Period	Technology / capability	What AI can do	What humans still do
2021–2023 observed	Early AI assistance	Suggest snippets, answer questions	Write, test, and decide almost everything
2023–2025 observed	Chatbots in daily workflows	Draft code, explain errors	Prompt, verify, integrate
2025–2026 emerging	Coding agents	Edit files, run tests, fix errors	Direct, review, set goals
Today emerging	More autonomous agents	Run code, delegate subtasks	Provide goals and oversight
20XX? hypothetical	Recursive self-improvement	Unknown — not observed	Direction, oversight, validation — if the loop ever closes

Fig. 1. Roughly how my own workflow changed. The last row is a question mark, not a forecast.

The last row is still hypothetical.

That’s important.

We’re not there yet, and even Anthropic describes recursive self-improvement as something that is not inevitable.

But the fact that we’re even able to seriously discuss the possibility feels significant.

Humans used to do everything

I still remember how software development felt a few years ago.

You opened your editor.

You wrote the code.

You searched Stack Overflow when something broke.

You stared at an error for an hour.

Then ChatGPT came along.

Suddenly, instead of searching for an answer, you could just ask.

Then the models started writing larger pieces of code.

Then came coding agents.

Now an agent can open files, modify code, run tests, inspect errors, and keep working through a task without me having to tell it every single step.

Each individual change felt pretty normal when it happened.

That’s probably what makes it interesting.

There was never really one dramatic moment where everything changed.

The boundary just kept moving.

And when I look back at the whole timeline, I realize that my own role in the process has slowly changed too.

I’m typing less.

I’m asking more.

I’m reviewing more.

And sometimes, I’m just watching.

The numbers are getting harder to ignore

I’m not working inside an AI lab.

I’m just someone who likes reading about this stuff and occasionally getting lost in papers, benchmarks, and technical reports.

But some of the numbers are difficult to ignore.

METR measures what it calls a “time horizon”. Basically, it estimates how long a task would take a human expert and asks how long a task an AI agent can complete reliably on its own. Their measurements have shown a rapid increase in this capability over time.

Anthropic’s more recent analysis describes a similar trend. According to their data, the length of tasks that frontier models can complete reliably has been roughly doubling every four months. They give an example that moves from roughly four-minute tasks in early 2024, to around 90-minute tasks a year later, and then to tasks lasting around 12 hours with a later model.

That’s the kind of number that makes me stop scrolling.

Not because it means AI can suddenly do everything.

It obviously can’t.

But because the direction is very different from what I was used to seeing just a few years ago.

Every time I think, “Surely AI can’t do that yet,” it doesn’t take very long before I find out that it can.

Eventually, I stopped making that bet.

Figure 02 · Task horizon (schematic) reported direction possible paths

Dataset plotted: none — schematic only, not to scale. Direction discussed in METR's "time horizon" work and Anthropic's analysis of task length (as cited in this essay, 2026). The specific figures I recall from those reports are in the essay text and are not reproduced here as verified data.

Fig. 2. Not data, just the shape of the trend I keep reading about. The right side is deliberately blank.

Coding is changing too

Software engineering is probably where I notice these changes the most.

I can see it in my own workflow.

A few years ago, writing an application meant sitting in front of an editor for hours.

Now I can describe what I want, let an agent implement it, run the application, inspect the result, fix an error, and continue from there.

The interesting part isn’t that AI can write code.

We’ve known that for a while.

The interesting part is that it can increasingly handle longer sequences of work.

Anthropic’s analysis of SWE-bench shows how quickly frontier models have improved on real software engineering tasks. SWE-bench gives models real open-source repositories and real bug reports, then evaluates whether the generated changes actually solve the problem and pass the project’s tests. Anthropic reports that performance went from low single digits to saturation on the benchmark over roughly two years.

And even more interestingly, Anthropic reports that its own engineers are increasingly using Claude to write code, run experiments, debug problems, and explore research ideas.

That creates a strange feedback loop.

AI helps humans build AI.

The better AI becomes, the more work it can take over.

The more work it takes over, the faster humans can build better AI.

And then the cycle starts again.

Figure 03 · Shifting roles human role → AI role per stage

Stage	Human role	AI role	Main bottleneck
Traditional observed	Writes everything	Autocomplete at most	Human time
Chatbot-assisted observed	Asks, pastes, verifies	Answers, drafts code	Verification, integration
Coding agents emerging	Directs, reviews	Edits, runs, fixes	Trust, review bandwidth
Autonomous agents emerging	Sets goals, oversees	Plans subtasks, executes	Oversight, evaluation
Recursive self-improvement hypothetical	Direction, validation, governance	Designs / trains a successor (?)	Unknown — an open question

Fig. 3. Less typing, more choosing. The last row is the one I'm unsure about.

The part that actually bothers me

It’s not really that AI is getting smarter.

It’s the possibility of a closed loop.

Imagine an AI system that can:

Write the training code.
Design an experiment.
Run the experiment.
Analyze the results.
Change the architecture.
Train a new model.
Evaluate that model.
Repeat the whole process.

At that point, the role of the human changes dramatically.

We wouldn’t necessarily be building the model anymore.

We would be telling the system what we want and checking whether what it produces is actually what we wanted.

That’s very different.

And that’s where the phrase “recursive self-improvement” starts to make sense.

The idea is simple. If an AI system becomes capable enough to design and develop its own successor, the successor could potentially be better at doing the same thing.

I don’t know whether that loop will ever fully close.

Maybe there are limitations we haven’t discovered yet.

Maybe the current scaling trends will slow down.

Maybe the hardest part of AI research turns out to be something that models simply can’t automate.

Or maybe we are closer than we think.

I honestly don’t know.

Figure 04 · The development loop follow the rail from top to bottom

1

Human defines the goal

Chooses what is worth building and what good looks like.
2

AI writes the code

Implements training and experiment scaffolding.
3

AI runs the experiment

Executes, logs, and keeps the run reproducible.
4

AI evaluates the result

Checks metrics and tests against the goal.
5

AI improves the approach

Adjusts architecture, config, or plan.
6

New model / system

A candidate successor — still needs human validation.

Repeat — each step already exists somewhere in isolation; the fully closed loop does not.

Fig. 4. The loop I keep sketching in my notebook. Most steps exist separately — the closed version doesn't.

So, what would I do?

This is probably the question I keep coming back to.

If AI can write the code, what am I supposed to do?

If AI can run the experiments, what am I supposed to do?

If AI can eventually evaluate the results too, where do I fit?

For now, I think the answer is still pretty simple.

Choose.

Choose what is worth building.

Choose which problem is actually interesting.

Choose which result is worth trusting.

Know when something isn’t working.

Know when to stop.

Know when to try something completely different.

Anthropic makes a similar distinction in its analysis. Today’s models can increasingly execute well-defined work, but human judgment around choosing goals, deciding which problems matter, and recognizing when an approach is a dead end remains an important gap.

I don’t know how long that gap will last.

Maybe years.

Maybe months.

Maybe much longer.

Three futures I keep thinking about

I don’t consider any of these predictions. They’re just three possibilities that I find myself thinking about.

Figure 05 · Three scenarios none of these are predictions

Scenario	What happens	Human role	Main uncertainty
1Trend slows possible	Progress hits technical or economic limits	Humans keep doing most research and development	Where the ceiling is
2Highly efficient AI emerging	AI handles most implementation and experimentation; humans decide what to pursue	Small teams accomplish work that needed much larger teams	How far efficiency goes
3Self-improvement hypothetical	AI systems design and improve future AI systems	Direction, oversight, validation, governance	Whether the loop ever closes

Fig. 5. Three futures I think about. I wouldn't bet on any single one.

The first possibility wouldn’t surprise me.

The second already feels like it’s happening in some form.

The third is the one I find hardest to imagine.

Not necessarily because it sounds like science fiction.

But because I genuinely don’t know what a world like that would look like.

Maybe I’m overthinking this

There’s also a good chance I’m getting ahead of myself.

AI progress doesn’t have to continue along the same curve forever.

The trends we see today could turn into S-curves. Hardware, energy, compute, data, research bottlenecks, or entirely new technical limitations could slow things down. Anthropic itself presents these as open possibilities rather than settled conclusions.

And that’s probably the part I like most about thinking about this.

There is still a lot we don’t know.

We’re watching something develop while we’re inside it.

That’s a weird position to be in.

Usually, you learn about a technological shift from history books.

This time, we’re living through one.

What I hope

I don’t want this to sound like an argument against AI.

I use AI almost every day.

I build with it.

I learn from it.

Sometimes it saves me hours.

Sometimes it writes something that makes me wonder why I spent three hours solving the same problem myself.

And sometimes it gives me an answer that is completely wrong.

That’s probably why I find all of this so interesting.

I’m impressed by it.

I’m also a little worried about where it could lead.

Both feelings can exist at the same time.

I don’t know where the curve ends.

I don’t know whether recursive self-improvement will ever actually happen.

I don’t know what my own job will look like five years from now.

But I do know that I want to keep paying attention.

Maybe that’s all this post really is.

A note to myself.

A snapshot of what AI looked like from where I was standing in 2026.

Maybe I’ll read this again years from now and laugh at how wrong I was.

I hope I do.

Maybe I’m wrong about a lot of this.

But the feeling of being both amazed and slightly uneasy is too real for me to ignore.

Written one night in 2026, when I stopped writing code and started wondering why I had stopped writing code.