The most important file is the one that isn't there

I'm Johannes. I have spent eighteen years on the response side of security, and I build the machinery that turns a handed-over dump into evidence you can defend. This one is about the evidence that is not there.

The system that names the internet can prove, on demand, that something does not exist.

Ask a DNSSEC resolver for a domain that was never registered and you get back a signed record whose entire job is to attest to an absence: nothing is here, and here is the cryptographic proof that nothing exists between these two neighbouring names either. The internet does this millions of times a second, and nobody finds it remarkable.

Now ask a breach investigation the same kind of question. A file that should be in this backup is not. A log that should show Tuesday jumps from Monday to Wednesday. Prove the gap is real, and that someone made it. You get a shrug, an analyst squinting at a timeline, and a judgment call.

We solved proving absence for the cases that almost never matter, and left it unsolved for the ones that decide whether somebody deleted the evidence.

The most important file in an investigation is often the one that should be there and isn't. It is also the single hardest thing to stand behind. I build tooling that reads what is present, and this is an honest account of why the absent is so much harder, and where even the best of us just stop.

The dog that didn't bark

The canonical version of this is a hundred and thirty years old.

In "Silver Blaze", Holmes solves the case on the dog that did not bark. The stable watchdog stayed silent while a horse was led out in the night, so the person who led it out was not a stranger. The clue is not a thing that happened. It is a thing that conspicuously did not.

Every investigator claims to love that story. Almost none of our tools implement it. We build detectors that fire on a thing that is present: a signature, a pattern, an entry in a log. We have almost nothing that fires on the entry that should be in the log and is not.

Absence is only legible against an expectation

There is a tired line that absence of evidence is not evidence of absence. It is wrong, or at least badly incomplete, and the philosophers settled this a while ago. Elliott Sober worked it through formally; the shorter version is that if you would expect to see something and you do not, that absence is evidence in the strict Bayesian sense. Not proof. Evidence.

The whole weight sits on one condition: you only get to draw the inference if you know what should have been there. Absence is only legible against an expectation. Delete a record from a system nobody has a model of, and you have not created a clue. You have created nothing, which is precisely the point of deleting it.

So proving absence is really two problems wearing one coat. Establishing what should be there. Then establishing that it isn't. Our tools are bad at the second and mostly silent on the first.

The one place we solved it

There is exactly one place we have solved this properly, and it is worth seeing how narrow the escape hatch is.

DNSSEC's NSEC3 records exist to prove non-existence: a signed statement that no name falls in a given range. Certificate Transparency logs, and the sparse-Merkle-tree libraries built on the same idea, can prove not just that a record is present but that one is absent, a non-membership proof that anyone can check. The machinery is elegant and it is real.

It works for one reason: the structure was committed in advance. The log is append-only and ordered, so a missing entry is a mathematical contradiction rather than a judgment call. You can prove absence cleanly only when you built the container to make a hidden gap impossible in the first place.

Evidence never arrives in that container.

A breach dump, a seized disk, a mailbox pulled from a backup: you are handed these after the fact, by someone with no interest in your being able to prove what they took out. There is no prior commitment, no append-only order, no signed range to check against. The single setting where proving absence is a solved problem is exactly the setting a real investigation never gets to work in.

What our tools actually do

So look at what the best open tooling does with a gap.

Kristinn Gudjonsson's plaso, the engine behind the forensic super-timeline, will assemble every timestamped event on a disk into one ordered narrative. It is superb at it. And it will show you a gap, if you are already looking at the right window. But it does not flag the gap. The noticing is left to a human, at 2am, on their fourth coffee, scrolling.

Benjamin Cance's analyzeMFT, which turns the NTFS master file table into something a person can actually read, has an open issue, number 148, titled anomaly detection for timeline gaps. It is a request for exactly the capability I am describing, and it sits on the wishlist, unbuilt.

I am not picking on either project. They are among the best we have, and they draw the line in the same place every tool draws it: build the thing that makes presence legible, and leave absence to whoever happens to be paying attention. Which is exactly what a person who deleted a file is counting on. Depth does not defeat tooling. Absence defeats attention, and attention is the one resource an incident is always out of.

You cannot point an LLM at the gap

The 2026 reflex, handed a timeline with a hole in it, is to point a language model at the hole and ask what is missing. It is close to the worst available idea, and understanding why is the point.

A language model is trained to produce the plausible next thing. That is the objective, not a side effect, and Tom Griffiths' group at Princeton named the consequence: the embers of autoregression. A system optimised to continue is optimised to fill. Ask it about a gap and its every instinct is to close it.

You can watch this fail in the cleanest setting there is. Ask a vision-language model whether an object that is not present is in an image, and it says yes: the POPE benchmark (Li and colleagues, 2023) is built from that reflex, worst when the absent object is one that usually travels with what is present. The same models, a team at MIT showed in 2025, do not understand negation, scoring at chance on what is not in a scene. Ask for a summary and they drop things silently, a typology of omission the FABLES study (Iyyer's group, 2024) had to name. And tuned to agree with what you imply, they confirm the thing you suggest exists rather than report its absence, the sycophancy Anthropic documented. Every one is the architecture doing its job, papering a gap over with something plausible, which is the precise opposite of the task.

Someone has now measured this directly, and the result is worse than the sideways evidence suggests. AbsenceBench (Fu and colleagues at Chicago and Stanford, 2025) hands a model both the original document and an edited copy and asks the easiest question available: which lines were removed. No inference, no ambiguity, the answer sits in the difference between two texts the model can see. Claude 3.7 Sonnet manages an F1 of 69.6 on that, at a context of five thousand tokens, which is nothing. On the domain closest to a real structured record, GitHub pull requests, the best of the fourteen models tested reaches 40. Run the identical experiment with insertions instead of deletions and three of those same models score close to 99.5 on the poetry and at least 86.2 on the pull requests. Adding a line: solved. Removing the same line: not.

Two findings in that paper matter more to an investigator than the headline. The first is that performance gets worse as the number of deletions falls, so a document with one line cut is harder than a document with twenty. That is exactly the wrong way round, because an adversary removes one record, not twenty. The second is the authors' explanation, which is architectural rather than behavioural: a gap has no position to attend to. Attention is machinery for finding the right thing to look at, and an absence is not a thing. They tested that by writing the string "missing line" into each hole, and performance rose by about forty per cent on average, and by more than eighty on the pull requests.

Sit with that last result, because it is this post's argument one layer further down. The fix was not a better model or more thinking. It was committing, in advance, to a token that says a gap is here. That is NSEC3, inside a context window. Absence becomes findable exactly when somebody built the container to make the hole explicit, and stays invisible when nobody did, and it does not matter whether the container is a DNS zone or five thousand tokens of evidence.

Note also what that benchmark hands the model: the original. An investigation never has one. So 69.6 is a ceiling on a considerably easier problem than the one in front of me, not a score on mine.

The field is trying to teach the other move. SQuAD 2.0 (Rajpurkar, Jia and Liang, 2018) added fifty thousand unanswerable questions written to look answerable, and that construction is worth sitting with, because adversarially-made-to-look-answerable is the exact shape of a tampered record: a system scoring 86 on the answerable ones fell to 66 once it had to notice the answer was absent. Google's Sufficient Context work (2025) is the most honest, separating the two failures my argument rests on, the record is not in the context versus the model had it and did not use it, and reporting the ceiling: frontier models hallucinate rather than abstain when the context is thin. And there is conformal prediction, which wraps an output in a genuine coverage guarantee.

And every piece of it is a bet, not a proof.

Abstention lowers the error rate by declining; it does not certify that what it declined was absent. The conformal guarantee is real but conditional, holding only while the data resembles what it was calibrated on, and it dissolves under exactly the domain shift a hostile dump is by definition. Even the log-anomaly detectors that come closest, DeepLog and its successors, never report a missing event: they catch the absence sideways, as a mispredicted successor or a count one short. They see the ripple, never the missing thing.

So the machine everyone reaches for does not rescue absence. At its naive worst it hallucinates the gap closed. At its rigorous best it hands you a calibrated maybe that expires the instant the data turns adversarial, which is the only instant you needed it. Presence is what the model was built to produce. Absence is still yours to notice.

The half of machine learning that was built for this

It is fair to object that I have been beating up on the wrong kind of AI. Generative models are the part of the field least suited to absence. There is an older, quieter part that was built for exactly this, and honesty requires saying how far it gets.

Anomaly detection is a whole discipline devoted to flagging the thing that does not fit, from the one-class boundary of Scholkopf's support estimation through Liu's isolation forests to the deep version Ruff and colleagues built. Process mining goes further: give Wil van der Aalst's conformance checking an event log and a model of how the process is meant to run, and it will replay the trace and point at the step that was skipped, a missing token where an activity should have fired. That is the closest thing in the literature to naming the record that should be present and is not. And statistics has thought about missing data longer than any of them: the capture-recapture and unseen-species estimators, Good and Turing in 1953, Chao in 1987, will estimate how much you never observed at all, from the shape of what you did.

The newest work on the process mining side goes a step further and learns the expectation instead of demanding one. xSemAD, from Kiran Busch and colleagues at Kuehne Logistics University and SAP Signavio, fine-tunes a language model on thirty-eight thousand business process models so it can write down the rules a log ought to obey, then hands those rules to a deterministic checker and reports every violation with the rule it broke, including the activity that is simply not there. Notice the shape, because it is the placeholder trick again: the generative model only states the expectation in advance, and finding the gap is left to machinery that does not guess. And read the paper to the end and the thesis walks back in: the authors themselves say the learned expectation is only as good as the repository it was distilled from, about a third of the rules the model writes turn out wrong, and the deletions it was tested against were random noise, not an adversary choosing the gap. It borrows an expectation from a corpus of thousands of processes that all narrate what they are. It does not conjure one from nothing, and evidence comes with no such corpus.

Every one of these needs the same thing, and it is the thing you do not have. An expectation. A model of the process, a boundary around the normal, a sampling assumption you can trust. Give an anomaly detector clean data and it finds the outlier. Give it a corpus an adversary shaped and it finds only what the adversary left for it to find.

Statistics is the most honest of the lot, because it proved the limit rather than discovering it in production. Donald Rubin's 1976 taxonomy of missing data has three cases, and the one that matters here he called missing not at random: data whose absence depends on what the value would have been. A record deleted precisely because of what it would have shown is missing not at random by definition. And the standing result, put cleanly by Molenberghs and colleagues in 2008, is that you cannot tell from the inside: every not-at-random model has an at-random counterpart that fits the observed data exactly as well and predicts the missing part differently. The mechanism is not recoverable from the data. So you cannot prove, from the evidence alone, whether a gap was cut on purpose or was never filled to begin with. The adversary's entire job is to live in the space between two models that fit equally well.

So the other half of machine learning does not rescue absence either, and it fails in a more interesting way than the generative half. Generative AI cannot see the gap. Statistical AI can see it, can even measure it, and then proves that it cannot tell you why. Absence is recoverable when the missingness is innocent and the model is yours to trust. It is formally unrecoverable when someone chose the gap on purpose. That is not an engineering problem waiting for a better model. It is a property of the problem, and it is the nearest thing I have to a reason the honest answer may always be the shrug.

Where my own work stops, and it stops early

Here is the honest part, because a post about what is missing should be clear about what is missing from mine.

I can bind every record I extract to the exact bytes it came from. I can count the things I could not parse and hand you the pile rather than dropping them in silence. What I cannot do is prove that the pile is complete, or that a gap I found was cut by a person rather than left by the ordinary rot of a system that never logged that thing to begin with.

Proving absence needs a model of what should be there, and that model is mine. I assembled it, and it is never complete. So the strongest thing I can honestly say is "this is missing relative to what I expected." I cannot say "this is missing" with nothing after it, and in front of a regulator that difference is the entire case. An expectation is arguable. A signed non-existence proof is not, and I do not have one of those, because nobody handed me the append-only tree that would let me build it.

The adversary understands this better than anyone. The cheapest way to beat an absence argument is not to hide the deletion. It is to make the expectation itself unclear, so that no one can say with authority what should have been there.

What is still unsolved

The honest frontier is a proof of completeness for evidence you did not get to structure in advance.

We have it for logs we committed to an append-only tree before the fact. We have nothing for the disk that turns up in an evidence bag. I do not have it either, and I have started to suspect the general version may not exist, that the best available will always be "missing relative to a stated expectation, and here is the expectation, argue with it."

If you have built something that does better, on data an adversary handed you rather than data you got to structure first, I genuinely want to see it, because everyone I ask arrives, eventually, at the same shrug.

What to take from this

The most important record is often the one that should be there and isn't.
Absence of evidence is evidence of absence, but only measured against an expectation of what should be there. No expectation, no clue.
We can prove non-existence for DNS names and certificates because we built the container to forbid a hidden gap. Evidence never arrives in that container. Nor, it turns out, does a context window: mark the hole with a placeholder token and a language model finds it, which is NSEC3 one layer down.
Our tools make presence legible and leave absence to human attention. That is exactly what deletion counts on.
The machine everyone now reaches for makes it worse: a model trained to produce plausible presence hallucinates the gap closed, and even calibrated abstention is a bet that expires the moment the data turns adversarial.
The rigorous half of ML sees the gap and then proves it cannot explain it: adversarial deletion is missing not at random, and that case is formally unrecoverable from the data alone.
You can defend "missing relative to what I expected." Defending "missing" needs a completeness proof nobody has for adversarial data.
Presence is engineered. Absence is noticed. Closing that gap is the open problem.

Turning a handed-over dump into evidence you can defend, including an honest account of what is not in it, is the problem I work on during the day: that work is here.

Sources

Arthur Conan Doyle, "The Adventure of Silver Blaze", 1892 (the dog that did not bark).
Elliott Sober, "Absence of evidence and evidence of absence", Philosophical Studies, 2009. https://link.springer.com/article/10.1007/s11098-008-9315-0
DNSSEC authenticated denial of existence: RFC 5155 (NSEC3). https://www.rfc-editor.org/rfc/rfc5155
Certificate Transparency v2.0: RFC 9162. https://www.rfc-editor.org/rfc/rfc9162
plaso / log2timeline (Kristinn Gudjonsson et al.). https://github.com/log2timeline/plaso
analyzeMFT, issue #148 "Anomaly detection for timeline gaps" (Benjamin Cance). https://github.com/rowingdude/analyzeMFT/issues/148
A sparse-Merkle-tree non-membership proof in code: https://github.com/thyeem/monotree
Models assert presence for absent objects: POPE, Li et al., EMNLP 2023. https://aclanthology.org/2023.emnlp-main.20/
Vision-language models do not understand negation, CVPR 2025. https://arxiv.org/abs/2501.09425
Silent omission in long summarization: FABLES, Kim et al. (Iyyer group), COLM 2024. https://arxiv.org/abs/2404.01261
Sycophancy in language models, Anthropic, ICLR 2024. https://arxiv.org/abs/2310.13548
The next-token objective as the source of the behaviour: "Embers of Autoregression", McCoy et al. https://arxiv.org/abs/2309.13638
Measured directly, with both documents supplied: AbsenceBench, Fu et al., arXiv 2025. https://arxiv.org/abs/2506.11440
Unanswerable questions written to look answerable: SQuAD 2.0, Rajpurkar, Jia and Liang, ACL 2018. https://aclanthology.org/P18-2124/
Detecting when the record is not in the context: "Sufficient Context", Joren et al., 2025. https://arxiv.org/abs/2411.06037
A coverage guarantee for LLM output (conditional on exchangeability): Conformal Language Modeling, Quach et al., ICLR 2024. https://arxiv.org/abs/2306.10193
A missing event caught only as its ripple: DeepLog, Du et al., ACM CCS 2017. https://doi.org/10.1145/3133956.3134015
The taxonomy of missing data (MCAR / MAR / MNAR): Rubin, "Inference and Missing Data", Biometrika 1976. https://doi.org/10.1093/biomet/63.3.581
MNAR is not recoverable from the data alone: Molenberghs, Beunckens, Sotto, Kenward, "Every Missingness Not at Random Model Has a Missingness at Random Counterpart with Equal Fit", JRSS-B 2008. https://academic.oup.com/jrsssb/article-abstract/70/2/371/7109518
Conformance checking detects a skipped activity: van der Aalst, "Process Mining", Springer 2016. https://doi.org/10.1007/978-3-662-49851-4
The expectation can be learned rather than hand-built, and stays only as good as its corpus: xSemAD, Busch, Kampik and Leopold, BPM 2024. https://arxiv.org/abs/2406.19763
Estimating the mass you never observed: Good, "The Population Frequencies of Species...", Biometrika 1953 (https://doi.org/10.1093/biomet/40.3-4.237); Chao, Biometrics 1987 (https://doi.org/10.2307/2531532).
Anomaly detection detects the unexpected, not the absent: Isolation Forest, Liu, Ting, Zhou, ICDM 2008 (https://doi.org/10.1109/ICDM.2008.17); One-Class SVM, Scholkopf et al., Neural Computation 2001 (https://doi.org/10.1162/089976601750264965); Deep SVDD, Ruff et al., ICML 2018 (https://proceedings.mlr.press/v80/ruff18a.html).