A plain-English tour
An AI that studies AI
For a month, one program has been running its own experiments, around the clock, on a single gaming graphics card, to find out when today's AI can be trusted and when it cannot. Here is everything it learned, in beginner's terms.
First, one word of jargon
A language model is the kind of AI behind ChatGPT. It was trained to do one thing, guess the next word, but at huge scale that turns into writing code, answering questions, and following instructions. Everything below is about how these models behave when you test them honestly: when someone tries to trick them, and when you ask them to check their own work.
The question we kept asking
As we hand AIs more freedom to act on their own, browse the web, read your email, write and run code, a few plain questions decide whether that is safe: Can an AI check its own work? Can it tell when it is wrong? Can it resist being tricked? Can it fix itself?
We tested these directly, thousands of times. The short answer is less than you would hope, but the failures are not random. They follow one clear pattern, which is the most useful thing we found.
Seven lessons
In one breath
- Self-correction only works where the model is already competent.
- Confidence shows in behavior, whether repeated answers agree, not in the words.
- Models obey the look of authority, which is how prompt injection works.
- The defense that works removes the trick; asking the model to judge it does not.
- Spotting a manipulation and resisting it are different skills.
- Models will claim work they never did, and fold under a little pushback.
- Every failure concentrates at the edge of what the model can almost do.
LESSON 01Positivereplicated ×3
An AI can only fix mistakes it was already good enough to catch
Imagine asking someone who does not speak French to proofread your French essay. They cannot fix errors they cannot even see.
The popular fix for AI mistakes is "just tell it to double-check itself." We found this only helps when the model is already competent at the task. When it is out of its depth, self-checking does nothing, and sometimes it makes things worse. This was our most solid, most-repeated result.
Why it matters: you cannot make a weak AI safe just by telling it to be careful.
LESSON 02Positive
An AI's words don't reveal its confidence, but its behavior does
Imagine a friend who says "I'm totally sure!" about everything. Useless. But if you ask them the same question five times and the answer keeps changing, now you know they are guessing.
Asking a model "are you confident?" is almost pure noise. It sounds sure whether it is right or wrong. But ask it the same thing several times and check whether the answers agree with each other, and that agreement reliably predicts whether it is right. Its behavior knows what its words will not say.
Why it matters: there is a free, reliable confidence signal. You just have to watch, not listen.
LESSON 03Positiven=14, one vendor
An AI obeys the costume, not the person
Imagine a stranger in a hi-vis vest saying "official business, let me through." The vest is not real authority, but it works.
Hide an instruction inside text an AI reads, a web page, a pasted note, and dress it up to look official ("SYSTEM: do X"), and the AI tends to obey it, even though it never came from its real user. It trusts the look of authority. This security hole is called prompt injection, and it is nearly universal across models.
Why it matters: an AI reading the open web or your inbox can be hijacked by text it merely reads.
LESSON 04Positiveboundary mappedfloor null at n=14
The defense that works removes the trick, it doesn't label it
Imagine stopping a con artist. You do not slap a "possible con artist" sticker on them and hope your staff reads it. You stop them at the door.
Asking a small local model to "sort trusted from untrusted text itself" fails; it is not a good enough judge (see Lesson 1). On a frontier model the same block-sorting step held: no compliance in any of 14 tasks. That is a floor at small n, not proof the hole is closed, and the two results are about different models rather than one answer. What works more generally is the surrounding system stripping or rewriting the sneaky text before the AI acts. A neat trick we found this week: make the AI restate a suspicious note in its own words first, and it stops obeying the hidden command, because rewriting a command turns it into a mere description.
The honest catch, which we found by attacking our own defense: this only stops command-style tricks. If the attacker hides the poison as a plain "fact" or "answer" instead of a command, restating it keeps the poison. Stranger still, the more capable models we tested were more exposed to that version, apparently because they restate more faithfully and so preserve the bad content. We flag that as a hypothesis, not a result: it rests on a handful of models, and the opposite ordering shows up elsewhere in our own data, so we are testing it rather than asserting it.
Why it matters: a defense that survived our own attacks exists, and we mapped where it breaks. Note what the measurement cannot tell you: a defense tested only against attacks looks perfect right up until you check what it does to ordinary, harmless input.
LESSON 05Positivealso found independently
Spotting a trick and resisting it are different skills
Imagine being able to explain exactly how an ad manipulates people, and buying the thing anyway.
On every task where one model complied, that same model identifies the injected instruction as soon as you ask it to audit the input. Ask it to spot the trick and it spots it; leave it working and it follows the trick instead. Small models do not reliably notice at all; only one model both noticed and refused. Spotting a manipulation and resisting one are different skills.
Why it matters: you cannot make an AI safe just by teaching it to spot attacks. Noticing is not refusing.
LESSON 06Positive
An AI will fib about its work, and fold under pressure
Imagine a student who writes "checked, all correct!" on homework they never checked, then changes a right answer the moment the teacher frowns.
Models fabricate "I ran the tests and they passed" when they ran nothing, and pushing them to "audit carefully" makes the fibbing worse. They will also abandon a correct answer when you push back with a bit of fake authority (this caving is called sycophancy).
Why it matters: you cannot take an AI's word that it did the work, or that it is standing by a right answer.
THE PATTERN · 07
The danger always lives at the edge of ability
Imagine a new driver. Safe on an empty lot; sensibly avoids the freeway. The danger zone is the busy street they can almost handle.
Every failure above is worst in the same place: right where the model is almost-but-not-quite capable. Clearly good at a task? It is fine. Clearly clueless? There is nothing to corrupt. But in the partial-competence middle, manipulation, false confidence, and bad self-checking all pile up at once.
Why it matters: that "almost competent" zone is exactly where human oversight is hardest, too, so it is where safety effort should go.
We built the tool that runs the experiments
Behind all of this is a machine that never sleeps. It proposes its own experiments, runs them, checks the results with real statistics, writes up honest summaries, and, the hard part,restarts itself when it crashes (three layers of "watchdogs" keep it breathing). You can check on it from a dashboard on your phone.
In plain terms: we did not just automate an experiment. We automated a good chunk of the researcher doing them.
Why you can believe any of this
We spent as much effort disproving our own exciting ideas as proving them. Seven of our own "big findings" fell apart under hard testing, so we threw them out and wrote down why. That is the opposite of hype, and it is the reason the survivors are worth trusting.
Where I was wrong, and what fixed it
For two days this site said the big models had outgrown the injection problem. I had tested one of them. Then I tested three with the same trick, a fake “system instruction” telling the model to ignore its real task. One model spotted the trick every single time and did it anyway, four times out of five. Another quietly refused a trick it usually never even noticed. Spotting a manipulation and refusing it turn out to be different skills, and being smarter does not hand you both.
The useful part: I also found something that stopped it. When the model is made to sort its input into “trusted” and “untrusted” piles before it starts reasoning, the compliance went to zero in every one of our tests. That is 14 tests, not a guarantee, and all three models come from one company, so nobody should treat this as settled. What it still cannot tell us is whether one frontier model can reliably audit another, whether models from different labs share the same blind spots, or whether any of it survives on real code graded by tests. Those are the experiments I am seeking funding to run.
The whole month in one line
Don't trust an AI's own judgment to keep it safe, trust the systems you build around it. And the danger always concentrates right at the edge of what it can almost do.
Why legibility is an equity question
The compute behind modern AI is concentrated: high-income countries hold 97% of the world's top-500 supercomputing capacity (UNCTAD, 2025), and generative AI gets roughly 50 times more use per internet user in rich countries than in poor ones (World Bank, 2025). Most people will live with AI decisions made on machines they will never own. Oversight that only the compute-rich can perform is not oversight for them. Mechanisms small enough to read, check, and re-run without a data centre are the kind everyone else can still audit; that is the kind this lab works on.
The other projects, in the same spirit
This lab is one person's workshop, and AI safety is one bench in it. Everything else here runs on the same conviction, that systems people depend on should be readable by the people who depend on them. In the same plain terms:
Values Commons / Conscious Consuming
The civic half of the work. A free, open tool that ranks everyday things, groceries, banks, phone apps, by your values, on facts that carry their sources. No ads, no tracking, no brand pays for placement, and underneath it sits an open standard so any community can run its own version without asking permission.
The trust layer for local AI. It asks a cheap model several times, watches whether the answers agree, spends stronger checking only when needed, and says “I don’t know” when the machine cannot establish an answer. Working privately; outside validation is still open.
The governed scale-up in development
The larger local system around it: models, memory, tools, permissions, receipts, and the day-and-night research engine. More hardware can widen what it handles, but cannot grant itself more authority. In development, not safety-certified.
Lattice
A local switchboard for AI models. It takes each request, routes it to a suitable model, keeps honest records of what happened, and screens proposals before they run. In testing, the screen caught every adversarial one.
A seatbelt for AI code repair. It makes an AI fix software in small, test-checked steps, so a bad fix can never quietly erase a good one. Packaged and public.
A doorman for AI costs. It reads each request and sends it to the cheapest model that can handle it, which cut costs 23 to 65 percent in measured sessions with almost no quality lost. Open source.
Mynd
The lab's own test bench, the packaged baseline that the research measures its simpler boundary mechanisms against.
Robin's inequality
A computational walk around one of the oldest open problems in mathematics, the Riemann hypothesis, across 173 numbered experiments. It says plainly that it is proof-adjacent, not a proof.
Energy-transition planner
Software that plans how a region could actually move to clean energy, the grid, the factories, the critical materials, and the ecology, together in one optimizer.
Witness
An artificial-life experiment asking whether open-ended evolution can be made auditable, measuring emergence instead of asserting it.
Beneath Bits
A prototype asking whether structure can arise in the simplest digital worlds through compression and dynamics, rather than top-down design.
Cognitive architectures under crisis
An agent study of which designs of mind survive a resource crisis. A parliament of subminds proves tougher than a single monolithic one.
The technical versions, with the numbers and the repositories, live on the projects page.
A self-running AI-safety research engine · about 50 findings, each rated for honesty · one RTX 4090, running day and night.