Delete Your AI Setup Every 6 Months. Keep What Isn't Scaffolding.

When Anthropic shipped Opus 5 in July, it deleted more than 80% of the system prompt that had been driving Claude Code. Performance went up slightly.

Boris Cherny, who built Claude Code, then gave the same advice to everyone using it: every 6 months, “delete your claude.md file, delete your skills, and delete your hooks”, see what the model does on its own, and add back only the lines that earn their place. He aimed it at people using Claude Code rather than people building agentic products on top of it, a caveat most of the reaction has dropped.

I sell people the work of writing those files. I think he’s right, and I think the test he’s describing is the most useful thing anyone has said about AI setups this year. It just doesn’t do what most people reacting to it think it does.

I’ve written before about AI setups rotting, from a case where the tool changed underneath one. This is the other way they die, and it’s the more awkward one, because nothing breaks. The model gets better, half your instructions turn into a leash, and no error is ever raised. That post named the pattern. This one is the test that tells you which of your own lines are already rotting.

Two things live in that file

Here is where I part company with the “delete it all” reading.

Your CLAUDE.md is not one kind of thing. It contains two, and they have opposite half-lives.

Scaffolding is anything that exists because the model isn’t good enough yet. Formatting nags, process reminders, do-this-before-that sequencing, routing hints telling it which tool to reach for. Every one of these is a bet that the model will keep being bad at something. It’s a depreciating asset, and it depreciates on Anthropic’s release schedule rather than yours.

Context is anything that encodes a fact about your business the model has no way to derive. Which of your 13 calendars has the client meetings on it. That your version number lives in two files and they must match. That your rate card is the source of truth and three pages import from it. That a particular client hates being called on Fridays.

The test is one line: would a better model make this unnecessary?

If yes, it’s scaffolding, and Cherny is telling you to go and check whether that day has already arrived. If no, it is not going anywhere, because no amount of model improvement will teach a system facts it has never been shown. Opus 6 will be cleverer than Opus 5 at everything. It still will not know your pipeline.

Sorting my own

I ran this on the two files I actually use, because a test you don’t run on yourself is just an opinion.

The one for this website is 141 lines across 11 sections. Sorted honestly, 9 of them are pure context: which files regenerate on every build and therefore must never be hand-edited, the 3 places you have to register a new page or it silently drops out of the machine-readable index, which environment variables the booking form needs, why one lint rule is deliberately switched off in the TypeScript blocks. None of that is the model being bad. All of it is my repo being specific. A better model gets these wrong faster.

One section is scaffolding, and it’s the one that tells the assistant which of my saved workflows to reach for in which situation. That exists because tool selection used to be unreliable. It is exactly the kind of line the 6-month test is designed to catch, and I would not bet much on it surviving the next retest.

One is genuinely on probation. I have a hard rule banning em dashes from everything I publish, because they are the single clearest tell that a machine wrote the sentence. That is a patch over a model habit, so by my own definition it is scaffolding. I retested it this week and it stays, because the habit is still there. It goes in the pile I have to keep checking though, not the pile I get to enshrine.

The second file is the one for my own business, and it runs to 238 lines. It has a section titled “What Claude already knows about Nick, don’t re-ask” and another setting out which workspace is allowed to touch which. Not one line of that is compensating for a weak model. It’s just true things about how I work, written down once so nothing has to interview me again. That file has no expiry date at all.

What this does to the AI setup business

The uncomfortable part, and the reason the advice landed so hard, is that a lot of what gets sold as an AI system is almost entirely the first pile.

If your setup is a long prompt full of tone instructions and behavioural nudges, the honest thing to say is that most of it was already dead and the model was politely reading it anyway. Deleting it costs you nothing, which is precisely why so many people ran the test, lost nothing, and concluded the whole category is theatre.

They drew the wrong conclusion from the right experiment. What that result actually proves is that they never had any context in the system. There was nothing in there a better model could not replace, because nothing in there was about their business. A setup made only of scaffolding is supposed to fail this test. That’s the test working.

I would rather say this out loud than have a client discover it. If you paid someone to write you a beautiful instruction file and Opus 5 does the job just as well with the file deleted, you did not buy an AI system. You bought a temporary workaround at permanent-solution prices.

Run it properly

The 6-month cadence is the useful bit, and it survives the distinction intact.

Take the whole thing out. Not one section, all of it. Then work for a week and write down every point at which the output is wrong, in one list. Now sort that list. Anything the model got wrong because it lacked a fact about your business goes straight back, permanently, and probably deserves to be somewhere more durable than an instruction file. Anything it got wrong because it was being careless goes back on probation with a date on it, because that line is a bet against the next release.

Then diary the next run. It is not a one-off cleanout, it is maintenance, in exactly the way that every AI setup people describe as finished is not.

The one thing I would add to his version: keep the deletions. When a line stops being needed, that is the clearest signal you get that the models moved, and it is worth knowing which of your assumptions expired and when.

The models will keep getting better and keep deleting your scaffolding for you. They will never once write down which of your calendars has the client meetings on it. That half is yours to build, and it’s the only half that compounds.

If you want to build the half that compounds rather than rewrite the half that keeps expiring, that’s what the Build Your Own Brain course does over 6 weeks: your context extracted and written down, your roles defined, all of it in a repo you own and can rebuild from when the tools change underneath you. Join the waitlist.