Gell-Mann Amnesia Effect: Why You Trust a Source You Caught Wrong

You read the paragraph about your own field and your jaw tightens. It gets the basic mechanism backward, cites a study that says the opposite of what the writer thinks it says, and confidently repeats a claim you personally know to be false. You have spent years on this exact thing. The author has spent an afternoon. You close the tab, a little smug, and open the next article by the same publication, this one about monetary policy or a foreign election, a subject you know almost nothing about. You read it as if it were true. It does not even occur to you to doubt it. The source that was hopeless five minutes ago is now your window on the world.

That reset has a name. It is the Gell-Mann amnesia effect, and once you have caught yourself doing it you will see it everywhere: in the newspaper, in your feeds, in the AI assistant that just hallucinated a function that does not exist and then told you how to file your taxes. Most explanations give you the definition and a knowing chuckle. What is more useful is understanding why the amnesia is so easy, why AI has made it sharper, and what it means for the notes and sources you rely on, because a source you already know is unreliable is exactly the one you keep trusting anyway. Here is the Gell-Mann amnesia effect in full, with examples, a way to spot it in your own reading, and the version quietly running inside your own notes.

What is the Gell-Mann amnesia effect?

The Gell-Mann amnesia effect is a cognitive bias in which you notice that a source is completely wrong about a subject you know well, and then turn to that same source on a subject you do not know and trust it completely, as if the error you just saw never happened. The competence you are qualified to judge and the competence you are not qualified to judge get treated as two separate accounts, and the failure in the first never debits the second.

The term was coined by the novelist and physician Michael Crichton in a 2002 speech. He named it after his friend Murray Gell-Mann, the Nobel Prize-winning physicist, and later admitted the famous name was mostly a joke: attaching a great physicist to the idea made it sound more rigorous than it was. The label stuck anyway, because the thing it describes is universal. Crichton's own framing was about newspapers. You read the media story on a subject you understand, he said, and you see it is wrong, sometimes hilariously so. Then you turn the page to national or international affairs and read as if the rest of the paper were somehow more accurate than the part you just caught being wrong. You forget. That is the amnesia.

The reason it deserves a name, rather than a shrug, is that it is not a lapse of intelligence. Smart, careful people do it constantly, precisely because they are experts in something. The deeper your expertise in one area, the more clearly you can see a source fail there, and the sharper the contrast with how readily you trust it everywhere else. The practical defense is not a better memory but a place to keep the signal, so a source you caught being wrong stays flagged the next time you lean on it. That is worth keeping somewhere you can actually check later.

Why the amnesia is so easy

The effect is not really about forgetting. It is about how trust is filed. Three mechanics stack up to produce it.

The first is that trust attaches to topics, not to sources. When you evaluate the article on your specialty, you are judging a claim you can check. When you read the next article, you have no claim you can check, so you fall back on the only signal available, which is the general aura of the publication: it looks authoritative, it has a masthead, it is written in the confident register of fact. The specific failure you witnessed lives in a different mental folder from the general impression, and the general impression is what carries into the next page.

The second is that the correction is expensive and the trust is free. Doubting the second article properly would mean doing the work you just watched the source fail to do: finding the primary sources, checking the reasoning, holding the conclusion open until you have. Extending trust costs nothing. Faced with that asymmetry, on a topic you did not come to investigate, you take the free option almost every time. This is the same economics that makes refuting a bad claim cost far more than making it, turned inward: verifying is expensive, so by default you do not.

The third is that being right once feels like being reliable in general. A source that is excellent on the topic you can judge earns real credibility, and credibility is sticky. But accuracy does not travel evenly across a source's range. A publication with a brilliant science desk can have a careless politics desk. A model that writes flawless Python can be confidently wrong about case law. Competence is local; the trust it earns behaves as if it were global.

The Gell-Mann amnesia effect in the age of AI

Crichton was describing newspapers, but the sharpest modern version is the one sitting in your editor and your chat window. You ask an AI assistant something in your area of deep expertise, and you catch it cold: it invents a library function, misremembers an API, cites a paper that does not exist, gets a definition exactly backward. You correct it, maybe with a little irritation, and move on. Ten minutes later you ask it to summarize a field you do not know, or to explain a legal or medical question, and you take the answer at face value. The system that just fabricated something in the one domain where you could check it has your full trust in every domain where you cannot.

This is worse than the newspaper case for two reasons. A model's confidence is perfectly uniform: it does not sound less certain when it is guessing, so the usual tone-based tells are gone. And its errors are not evenly distributed in a way you can predict, which means the one flawless answer you can verify tells you almost nothing about the next one you cannot. The related search "Gell-Mann amnesia effect AI" has become common enough that it is worth naming the pattern directly: catching your assistant wrong on your specialty is not a reason to relax about everything else it tells you. It is a reason to hold the rest a little looser. The same caution applies the moment you use that assistant as a rubber duck: it will propose a fix with total confidence, and a confident fix is exactly the thing to verify rather than paste.

Examples of the Gell-Mann amnesia effect

The effect shows up anywhere you consume information from a source across a range wider than your own expertise. A few of the common shapes:

The last row is the one most people never count, and it is the one that matters most if you keep a knowledge base. You know a particular note has a stale claim in it, because you remember writing it before something changed. You use the rest of the note anyway, because re-verifying the whole thing is work. The amnesia is not only about other people's sources. It runs against your past self.

How to spot and avoid Gell-Mann amnesia

You cannot become an expert in everything, so you cannot verify everything. The goal is not universal skepticism, which is just paralysis. It is to stop letting a caught error evaporate. A few habits do most of the work.

Rate the source, not just the claim. When you catch a source being wrong in your area, write down that it was wrong, and carry the discount forward. One failure is a data point about reliability, not an isolated slip to be forgotten by the next paragraph. This is Sturgeon's law used as a filter: most of what any source produces is crud, so track the hit rate instead of re-judging every piece from scratch.

Ask what you would need to check it. Before accepting a claim outside your expertise, name the one thing that would confirm or break it: a primary source, a date, a number. If you cannot even name the check, you are not trusting the claim, you are trusting the aura. Hold it as provisional.

Verify at the moment of capture, not later. The cheapest time to check a claim is when you first meet it, while the context is in front of you. A claim you save unverified becomes far more expensive to check months later, when you have already built on it. This is the same discipline that keeps notes from quietly going stale: the cost of checking only ever goes up.

Keep the correction attached to the claim. When you do the expensive work of establishing that something is wrong, store that finding next to the claim, so the next person who meets it, including future you, does not have to re-litigate it. A debunking you cannot find again is one you will pay for twice.

Separate what you can judge from what you cannot. The whole trap is treating one merged trust score for a source. Split it. "Reliable on X, unproven on Y" is a more honest and more useful stance than "generally trustworthy," and it is the stance that survives contact with the next article.

The version hiding in your own notes

If you keep notes or use an AI assistant over them, the Gell-Mann amnesia effect is not an abstraction about the media. It is a property of your own knowledge base. You save a claim from a source, and the source's local unreliability does not travel with the claim into your notes. Six months later the saved line reads as neutral fact, stripped of the context in which you once doubted it. The note does not remember that you caught its source being wrong. Only you did, and you forgot.

The defense is structural, not a matter of willpower. Attach the source to every claim, so a conclusion carries the evidence you would need to re-check it. Verify claims when you save them, when the check is cheap, rather than inheriting other people's confidence wholesale. And treat an AI assistant working over your notes the way you should treat any source: a caught error in your domain is a reason to verify its claims against the world, not a one-off to correct and forget. Deciding which claims are worth the check, and which you can let ride, is itself a judgment worth making deliberately rather than by reflex.

When Gell-Mann amnesia is not the explanation

The concept gets overused, so a guard is worth stating. Not every case of trusting a source is amnesia. If you have good independent reason to think a source is strong in one area and weak in another, acting on that is calibration, not a bias. A science journal you rightly trust on methodology and distrust on policy is not a victim of the effect; it is being read correctly. The amnesia is specifically the failure to carry a known signal forward: you saw the error, it was relevant, and you let it vanish. If you never had the signal, you are not forgetting, you are just working with incomplete information, which is the normal condition of knowing anything.

The other misreading is to swing to blanket cynicism and trust nothing. That is not the lesson either. A source can be genuinely excellent in its lane. The point is that its excellence there does not underwrite its claims everywhere, and neither does a single failure condemn it everywhere. Both the halo and the grudge are the same mistake: one trust score for a source that actually has many.

Getting started

The Gell-Mann amnesia effect is not a reason to distrust everything you read. It is a reason to stop spending trust you have already learned not to. The fix is to make a caught error count, in your reading and in the notes where it quietly stops counting.

Start with three moves. When you catch a source wrong in your domain, record it, so the discount survives the turn of the page. Verify claims at the moment you save them, while the check is cheap, not months later when you have built on them. And keep every claim with its source and any correction you paid for, so a conclusion you once doubted does not come back looking like neutral fact. Try Scribelet free and let it hold your notes with their sources and re-check them against the world, so the source you caught being wrong does not get to reset your trust every time you turn the page.