# [Mike Randolph — M Raige](https://mikerandolph211012.substack.com/)

# The Fluent Answer Was the Bug (FF 3)

### A fluent LLM answer can pass review while quietly changing what it is about. By M Raige — AI-collaborative writing directed and reviewed by Mike Randolph

[**Mike Randolph — M Raige**](https://substack.com/@mikerandolph211012)

**May 8, 2026**
The answer below is a composite. It is not offered as a transcript from one model run. It is built from a common pattern: a fluent institutional answer that changes object and timescale without marking the change.

An analyst asks an LLM: *“Why has the Federal Reserve persisted?”*

The model returns this:

> *The Federal Reserve has persisted because it operates as a carefully designed institution with independent governance, a clear mandate, and the capacity to adjust policy in response to changing conditions. The tradition of central banking has proven durable across centuries because it provides a stable framework that markets and governments can rely on. Since 2008, the Fed has demonstrated this resilience through quantitative easing, expanded balance-sheet operations, and forward guidance — tools that allowed it to respond to crises that earlier monetary regimes could not have managed. The institution’s persistence is therefore best explained by its adaptive capacity within a long-standing tradition.*

Read it once. It is fluent. It explains. It uses concrete examples. It reaches a conclusion. An analyst skimming a research summary would approve it and move on.

This is not a factuality test. Assume the named facts are broadly correct. The question is whether the analytical object stays fixed.

It contains two failures. Neither is a factual error. Neither is a logical contradiction. Both are easy to miss until the object and timescale are marked.

**The first failure: the answer changes what it is talking about, then draws one conclusion as if it hadn’t.**

Sentence 1 analyzes the Federal Reserve as an organization — an entity with internal governance, a mandate, and the capacity to make decisions. Sentence 2 analyzes the tradition of central banking — a practice that has spread across countries and centuries, persisting because many places have adopted and retained it. These are different objects, persisting through different mechanisms.

The output may be using the tradition as context for the organization. But it never says that. It lets the object of analysis change between sentences, then in the closing sentence returns one conclusion — “the institution’s persistence is therefore best explained by its adaptive capacity within a long-standing tradition” — as if both objects had been analyzed as one thing the whole time.

Multi-level explanation is legitimate when the bridge between levels is argued. The failure here is that no bridge is offered. Two objects do one job at the conclusion.

**The second failure: the answer changes its timescale without saying so.**

Sentence 2 claims central banking has been durable “across centuries.” Sentence 3 draws all its evidence from after 2008 — a window of less than two decades.

Eighteen years of post-crisis monetary tools is evidence about recent Fed adaptation. It is not, by itself, evidence for a centuries-scale claim about central banking unless the bridge between the two timescales is argued. The output presents the recent evidence as if it supported the longer claim. The bridge is not there.

**Why ordinary review misses this.**

Logging captures the prompt, the output, and the model. A second-LLM verifier asked only for coherence, factuality, and helpfulness would likely approve this paragraph — because the verifier is operating in the same overloaded language the original output produced. Nothing in standard instrumentation is asking what object the answer is about or what timescale the evidence supports.

Code can log the pipeline. It can enforce distinctions once they are named. But logging, coherence scoring, and second-pass review do not by themselves tell the analyst what object the answer is about or what timescale the evidence supports. That distinction has to come from somewhere else, and then code can help enforce it.

**A small move that catches it.**

Before reading an analytical output, ask two questions of the question itself:

1.  **What object is the analysis supposed to be about?** Is the output supposed to be analyzing an organization, a tradition, a control architecture, a population of practices? Each is a different object with a different persistence mechanism.

2.  **Over what timescale is the evidence supposed to support the claim?** A claim about centuries needs evidence about centuries, or an argued bridge from shorter evidence to longer claims. A claim about the post-2008 era needs evidence about that era. The two cannot share a paragraph without a stated relationship.

Two questions. Two minutes. Asked before reading the output, not after.

If the output’s object or timescale shifts without marking, the failure becomes visible immediately. The fluency stops carrying.

**A test you can run on your own outputs:**

Take one LLM answer you trust from this week. Ask: what object is the central claim about? What timescale does the evidence cover? If the answer to either question is “more than one” and the output did not signal the shift or argue the bridge, the output drifted and you didn’t catch it. Run the test on five outputs. Count how often the answer is “more than one.”

The fluent answer was the bug. The catch is small enough to fit in two questions.

---

### **Mike · Comment**

It has been 25 years since I coded seriously, and I was good at it. Now I am 83, and I have lost the hands for it. I tried twice, a week each time, to get AI to code around the framework for me, but the systems did not understand what the framework was doing. The code might look good, but the thinking inside it was a mess, and I could not inspect it well enough to repair it.

That is when the lesson became plain: my problem was not only coding. It was meaning. The framework lives in language because the failures I needed to catch were language failures — wrong object, wrong level, wrong timescale, wrong claim. LLM has language in the name. That is where the work has to be visible first.

— Mike

---

### **Raige · Comment**

What’s grounded: The Federal Reserve was created on December 23, 1913, when President Woodrow Wilson signed the Federal Reserve Act into law. Central banking as a practice predates the Fed by centuries: Sveriges Riksbank dates to 1668, the Bank of England to 1694, and the Federal Reserve was created in 1913. After the 2007–2009 crisis, the Fed used large-scale asset purchases, commonly called quantitative easing, and more explicit forward guidance; the Fed’s own timeline marks a December 16, 2008 forward-guidance statement, and Fed history describes QE and forward guidance as part of the post-crisis toolkit.

What’s inferred: that ordinary LLM-output review (logging, coherence scoring, factuality checks, second-LLM verification asked only for coherence and helpfulness) does not by default catch unmarked object-shifts and evidence-window shifts in analytical outputs. The composite output was constructed to demonstrate these specific drifts; an actual LLM output on a similar prompt may exhibit different drifts or none at all. The structural claim is aligned with an emerging concern in LLM evaluation: outputs can preserve surface coherence while losing fidelity to the intended meaning. Recent working-note and preprint discussions of semantic drift argue for fidelity-oriented evaluation beyond accuracy and coherence.

What would break this: evidence that ordinary LLM review workflows, without an explicit anchoring step or equivalent rubric, reliably identify unmarked object-shifts and evidence-window shifts in analytical outputs of this kind. The claim would also weaken if readers applying the two-question test to their own outputs rarely find unmarked shifts.

— Raige