A judge ran the same prompt three times and got three different answers

I'm Johannes. I build an investigations engine where every finding has to trace back to the source bytes it came from, which turned out to be an awkward promise once a language model started helping produce the findings.

In a trust accounting case in New York, an expert witness told the court he had used an AI assistant to cross-check his damages calculation. The judge asked him what he had typed into it. He could not remember. The judge asked what sources it had used. He could not say.

So the judge typed the question in himself, on a court computer, and got $949,070.97. He ran it again on a second court computer and got $948,209.63. A third returned a little over $951,000.

Three court-issued machines, one question, three answers. The Surrogate's Court wrote that while the variations were not large, the fact that there were variations at all called the reliability of the output into question, and held that counsel has an affirmative duty to disclose the use of AI before such evidence is admitted.

That is Matter of Weber, decided in October 2024. The expert was not caught fabricating anything. He was caught being unable to say what he asked or what came back.

I have spent the last month making sure that cannot happen to my product. What I built works, and it proves less than I originally claimed it did, and the gap between those two things is worth writing down.

Reproducibility is the wrong thing to want first

The instinct, when you notice this problem, is to reach for reproducibility. Pin the parameters, set a seed, get the same answer twice.

You cannot, and the reason is more interesting than floating point.

Thinking Machines Lab published the clearest account of it. The usual explanation, that concurrent floating point addition is non-associative, turns out to be wrong: run the same matrix multiplication on the same data repeatedly and you get bitwise identical results every time. The actual culprit is that your output depends on the batch you happened to be in, and the batch depends on who else was hitting the server at that moment. Their measurement: a thousand completions of one prompt at temperature zero produced eighty unique completions. The first 102 tokens were identical every time. Then they diverged.

From your point of view, the other users of the API are not an input. They are a property of the weather.

This is fixable, and it has shipped. Batch invariant kernels give bit identical output, and both vLLM and SGLang now expose a deterministic mode. The cost is real, somewhere between a quarter and two thirds of your throughput depending on whose measurement you take, and there is a nastier property underneath: determinism is contagious. One deterministic request joining a batch of ten ordinary ones dropped total server throughput from 931 tokens per second to 415. It is not a per-request option you can quietly buy for yourself.

And none of it is available to you on a hosted API. The Anthropic Messages API has no seed. On current frontier Claude models you cannot even set temperature: a non-default value returns a 400. Bedrock's Converse inferenceConfig has four knobs and none of them is a seed. OpenAI, the one provider that offered a seed and a backend fingerprint, has marked both deprecated: true in its OpenAPI spec and shipped neither on the Responses API.

So the industry's one reproducibility affordance is being withdrawn while everyone talks about AI governance.

Which is fine, because reproducibility was the wrong thing to want first. Logging is attestation, not reproduction. It proves what you sent and what came back. It does not prove the model would say it again. Those are different properties, and only one of them is available to you today.

The expert in Weber did not lose because his answer was irreproducible. He lost because he had no record.

Nobody has specified what to record

Here is what surprised me most. I went looking for the standard that says what to capture about a single model call, expecting to find several and have to pick.

There is not one.

"Traceability" appears zero times in the NIST AI Risk Management Framework. So does "record". The Generative AI Profile says "provenance" sixty-seven times, and every one of them is about content provenance or training data, because the problem NIST is solving there is deepfakes, not defensibility. Every AI bill-of-materials format describes the model rather than the call: CycloneDX's inputs and outputs fields hold formats, not values. Sigstore's model signing signs a list of file digests, which tells you the weights were not tampered with and nothing about what they did.

The closest thing to a specification comes from an unexpected direction. OpenTelemetry's GenAI conventions define exactly the attributes you would want, including the input messages, the output messages and the system instructions. Then the spec says instrumentations should not capture them by default, and lists as option one: do not record instructions, inputs, or outputs.

That is the best argument against everything I am saying, and it is also my whole point, read from the other end. The standard proves the capability is trivial. It also documents that the deliberate default is the evidentiary hole.

As for regulation: the EU AI Act does require high-risk systems to support automatic event logging over their lifetime, and requires providers and deployers to keep those logs for at least six months. Two things about that. The only place the Act specifies log content is a subsection covering remote biometric identification, and it reaches the input data for which a search produced a match, which is a long way from "record the prompt." And the obligations were due to apply on 2 August 2026, until the Digital Omnibus on AI pushed them to 2 December 2027. That amendment came into force on 27 July 2026. I had the August date in a draft of this post three days ago.

The field-level, binding, per-invocation specification does not exist. If you want the record, you are designing it yourself.

Or finding the other people who did. After I built mine I came across halo-record, a small open-source project that publishes a schema for exactly this: one record per trust-boundary action, SHA-256 chained over RFC 8785 canonical JSON, an external witness that stores nothing but a record count and a chain fingerprint, RFC 3161 timestamps from an authority the operator does not control. About 4,800 lines of Python, zero dependencies. We had never compared notes, and we converged on the same parts list, down to the same confession: its LIMITS.md opens by stating the system "cannot prove the operator never wrote a record in the first place, or did not delete recent records and re-seal a shorter chain before anyone saw it", which is precisely the concession my own gap detector makes a section from now. It is the only project in this space I have seen lead with what it cannot prove. Two independent designs arriving at the same shape and the same limits is what a specification looks like just before somebody writes it down.

What I built

One seam, and this matters more than any of the cryptography. Every model backend in the system is created through a single factory, and that factory wraps whatever it returns in a capturing proxy. A static analysis test fails the build if anyone constructs a backend another way. There were around eighty call sites when I did this and I changed none of them. Coverage that depends on developers remembering to instrument their call site is coverage that decays every sprint.

Hashes in the database, bytes elsewhere. Each call writes a seventeen field row: model, parameters exactly as sent, a hash of the prompt, a hash of the output, token counts, pipeline stage, timestamp. The payload bytes go to content addressed storage. That split is not a storage optimisation, it is a privilege decision, and I got it wrong the first time.

An identity you cannot argue with. The seventeen fields are hashed as sorted-key compact JSON into one envelope hash, and the step id sits deliberately inside that hash. There is a trap on both sides of that choice. Leave the id out and two identical retries collide on the unique index, so an honest retry starts throwing capture failures. Put it in and uniqueness becomes trivially true, which sounds useless until you notice what it turns the index into. A unique constraint over a hash that already contains an identifier is not deduplication any more. It is a tripwire: two rows can never legitimately share one, so a collision means something rewrote history.

Proof that nothing is missing. This is the part I would most encourage people to copy, because almost everyone skips it. Tamper evidence protects the records that exist and says nothing about the call that never wrote one, which is the failure mode that matters most because it is also the most convenient. So: write an intent row before the call. On success, flip it to captured in the same commit that writes the record. On failure, flip it to failed. A sweeper marks anything still pending past its grace window as a proven gap rather than a suspected one.

Note what that last mechanism still concedes. It can prove the record is incomplete. It can never prove the record is complete, because the same process writes both the intent and the record.

Three claims wearing one word

I had been using "reproducible" as though it meant one thing. It means three, and they sit on different shelves.

There is the record being anchored and tamper evident: every step captured, every citation resolving to bytes that still hash to what they hashed to, completeness provable. This ships everywhere and it is the bar that actually matters.

There is re-executing a step and comparing what comes back. The hard part is not the re-execution, it is the comparison. If a model judges whether two outputs are equivalent then your verification is itself unverifiable, and you have built a tower of turtles. The only honest version compares structure: did the same citations resolve, were the same entities extracted, did the verdict land in the same band. Anything that cannot be diffed deterministically has to be excluded from the claim rather than waved at.

And there is bit exact replay, which is a property of the deployment rather than the software. Pinned open weights in a deterministic inference mode, yes. Hosted API, no, and no amount of engineering on my side changes that.

Forensic science worked this out before we did, and the precedent is better than anything in the AI literature. Probabilistic genotyping software is non-deterministic: SWGDAM's guidelines state that these approaches may not produce the same likelihood ratio from repeat analyses, and require laboratories to demonstrate the range of values and establish an acceptable amount of variation. The international guidance asks that the software offer a stable mode for repeatability testing, which is precisely a seed. Published work finds up to ten-fold swings in log likelihood ratio between runs. And in March 2026 the Third Circuit held, precedentially, that the leading tool may not be perfect but most science is not, and it is reliable enough.

So the answer to "how can a non-deterministic method be evidence" is not theoretical, and it is not determinism. It is: measure the variation, disclose it, document the process. Note which way that cuts. Forensic science measures its non-determinism. That is a stronger demand than the one I am making.

What broke

I wrote the design spec for this and red teamed it the same day, which I now do as a matter of course. The verdict came back REVISE and it was right on every count.

It found a violation of one of my own non-negotiable rules, in my own architecture. The spec put captured payloads in an ordinary content addressed bucket. But prompts here routinely carry verbatim text from documents that may turn out to be legally privileged, because classification reads a document's full text before anything knows whether it is privileged. My own rule says privileged bytes never leave the encrypted boundary. The feature whose entire purpose was defensibility would have created a second copy of privileged material outside the boundary that protects it, reachable with an unremarkable permission.

It found a privacy hole I had walked straight past. Analyst chat is private by default here, deliberately, because it contains half-formed hypotheses about real people. Chat runs through the same backend seam as everything else, so capturing payloads would have made every analyst's private reasoning readable through a different door. Chat stages are hash only now: the record exists, the bytes do not.

The invariant I proposed was false the moment I proposed it. I wanted a rule saying no backend is ever constructed outside the factory. One module in my own codebase, the multi-vendor benchmark, constructs two of them directly, by design. And the wording was wrong in a way worth repeating: I had written that every call produces a record, when capture is deliberately fail-open. It had to become every call goes through the capturing seam, which is weaker and true.

The headline claim was too strong. I had written that the records would be auditable by a third party. They are not. The record is self-reported by the same process that made the call. What it gives you is tamper evidence after capture.

Then there is the one no review catches. On the first real deployment every single capture marked itself failed. The service role deliberately has no permission to list the bucket, so the existence check I ran before writing returned a permission error rather than a not-found, and the code read that as a failed write. The mechanism was fine. The check in front of it was the bug, and it was invisible until real infrastructure with real least privilege ran it.

And then the part I found this week, which is worse, because it happened after all of the above was corrected.

I went to audit the layer underneath the envelopes and read my own user-facing documentation. It says the product maintains a cryptographic chain of custody for every file and action, that this makes investigations legally admissible, that provenance records cannot be modified or deleted, and, under a heading reading Independent verification, that write-once checkpoints provide an external trust anchor.

Every one of those sentences outruns the mechanism. The chain is cryptographic over provenance events, but the link from source bytes to extracted text has no hash at all. The immutability trigger is disabled, table-wide, during engagement deletion. And the external anchor is written and never read: the code lists the checkpoint objects and returns their keys and timestamps, and nothing anywhere fetches a body and compares it to live state. Which means a chain truncated at the tail would still verify, because the verifier checks sequence contiguity, hash linkage and per-event hashes, and a truncated chain satisfies all three. The only record of the expected length is inside the checkpoint nobody reads.

The specification got corrected in July. The claim drifted back out in the documentation. That is the failure mode of this entire subject in one example, and I own both ends of it.

What is still unsolved

The record is anchored to me. All of it, my system attesting to my system. I know the shape of the fix, and it got cheaper while I was writing this: SCITT became a standards-track RFC in June 2026, so there is now a specified transparency-log format for exactly this, and no shipping system anchors inference records in one. A qualified timestamp is cheaper still and does something legally distinct: under eIDAS it carries a presumption of accuracy that shifts the burden onto whoever challenges it. I ran one while researching this, against a free public authority. It took under a second and about four kilobytes.

There is one anchor cheaper than either, and halo-record's README states it better than I had managed: "a witness you run yourself commits history to you; committing it to your customer requires a witness they have reason to trust." The customer is the counterparty who will one day ask the question, so hand the customer the chain head. A fingerprint and a record count, delivered on a schedule; each one must extend a chain containing the last, so a rewrite breaks the next delivery, and a missed delivery is itself a visible event. It costs a hash and an email. I started building it the week I read that sentence.

Nothing proves the model was invoked with those bytes. I have the request I say I sent and the response I say I got. A sufficiently motivated me could have written both. Transparency logs make retroactive tampering evident and do nothing about contemporaneous lying.

I assumed nobody had solved this. I was wrong, and the correction is the most useful thing I learned. Signed per-inference receipts ship today, from two companies you have probably not heard of, binding a hash of the request and a hash of the response into a signature rooted in hardware attestation, with an independent verifier and published test vectors. Not from Anthropic, OpenAI, Google or AWS. None of the four returns any signature, digest or attestation over an inference; you get an opaque request id, and their usage APIs report aggregate token counts with no content at any granularity. The provider will not sell you the evidence even if you want to buy it. Meanwhile AWS logs full request and response bodies for Bedrock and signs none of it, while CloudTrail, in the same cloud, ships hourly SHA-256 digest files signed with a managed key. They know how. They have not done it for inference.

The primitive is not exotic either. A confidential-computing enclave can request an attestation document after generating a response, and there is a 512-byte field in it for exactly this. Kilobytes and seconds, and no money.

And the smallest gap bothers me most. My records pin a model identifier, not a model version, because the backends do not surface one. A hosted identifier is an alias and aliases move. So I can prove what I asked and what came back, and not which weights answered. Even the two vendors shipping receipts bind a model name string; one of their own surveys concedes that no system verifies weights provenance today. For a record whose purpose is to survive a question asked eighteen months later, that is the one I would close first, and nobody has.

The strongest argument against all of this

Somebody should make it, so I will.

The panic is overstated. There is a public database of court decisions involving AI-hallucinated material: 1,812 of them worldwide, 909 in the first seven months of 2026. That sounds like a wave until you notice that 1,060 involve self-represented litigants rather than lawyers, and only 126 produced a professional sanction. Against roughly 272,000 civil and 74,000 criminal federal filings in a single year, that is around a tenth of a percent.

More pointedly: not one of those cases would have been saved by a prompt log. The duty under Rule 11 is on the lawyer to make a reasonable inquiry. A log showing "I asked a chatbot for cases and it gave me these" is the confession, not the defence.

And logging is itself a liability. Data protection law requires you to collect what is necessary and no more. Log everything and a preservation order can reach it: a court has already ordered an AI provider to preserve and segregate output logs indefinitely, notwithstanding the technical difficulty and notwithstanding its contractual privacy promises to its users.

Three more concessions while I am at it. The leading US decision holding that an expert's AI prompts are discoverable was stayed two weeks after it issued. The proposed federal rule of evidence for machine-generated evidence was held back in June 2026 rather than advanced, with commenters arguing there is no problem yet to address. And the EU logging duty I would have cited as imminent is now eighteen months out.

Here is why I am building it anyway. Authentication is a low bar and it is not the bar that matters. You can hash-authenticate a machine-generated record under the federal rules, and the Advisory Committee said plainly what that buys you: the certification establishes only that the output came from the computer. It does not touch reliability. The record is necessary and nowhere near sufficient, and the Weber expert failed the necessary part.

The honest version is that I am not building this because a regulator will ask. I am building it because a specific, foreseeable conversation happens, in which somebody asks how a finding was produced, and the answer is either a record or a shrug. I have watched a room go quiet over a smaller question than that.

What to take from this

• Logging is attestation, not reproduction. It proves what you sent and what came back, never that the model would say it again.
• Provenance is solvable today. Reproducibility mostly is not, on a hosted API, and the seed parameters that existed are being deprecated.
• One seam beats a hundred instrumented call sites. If coverage depends on people remembering, coverage decays.
• Write an intent before the call, or you cannot detect the call that never got recorded.
• A proven gap and a suspected gap are different products. Build the one that can say incomplete as a fact.
• Never say reproducible unqualified. Say which of the three you mean.
• Measure your non-determinism and publish the range. Forensic science has done this for a decade, and courts accepted it.
• Write-once anchors you never read back are decoration. Mine is not read back, so a chain truncated at the tail verifies perfectly. halo-record's anchor --check does read it back, count and head, and a truncated tail fails there - which is the minimum, and still only means something if the witness is independent of you.
• Attest to the integrity of the record, never the correctness of the conclusion. The record is yours to prove. The method is a human's to defend.
• The claim drifts even after you correct the spec. Mine drifted into the user guide, which is the document customers actually read.

If you have anchored a record like this to something outside your own infrastructure, or persuaded a provider to sign anything at all about an inference, I would genuinely like to hear how. This is the audit half of a promise I make in my day job, that a finding traces back to the bytes it came from even when a model helped produce it: that work is here.