How EnemoAi is measured

What we test, how we pick what we test it on, what we do with the misses, and what we haven't tested at all. Written and checked against our code on 5 November 2026. The backtest sections were updated on 7 November 2026 with a second pre-registered test.

How a claim gets measured

We use four kinds of test. Each answers a different question, and none of them stands in for another.

• Backtests: would EnemoAi have seen a company failure coming, using only what was public at the time?
• Forward calls: does a warning recorded today turn out to be right later?
• The answer-quality test: do answers hold up against their sources and a strict grading rubric?
• Reading tests: do rules that pull figures from filings get those figures right?

Backtests

A failure is a bankruptcy petition, a receivership of the company or its bank, or a wind-down of its main business, within 24 months of the replay date. Failures are replayed 24, 12, 6 and 3 months before the event. Survivors are replayed on the day their distress was disclosed and six months later. That makes 138 replays: 108 of failures and 30 of survivors.

Two predictors are written to run on those dates:

• Live Threat Alerts. The alert grading runs over the company's SEC filings. A company counts as flagged when a critical or high alert fired in the 12 months before the replay date. No AI is involved.
• Mentor. It answers a supplier's or lender's question as of the date and ends with a stated chance of failure within 24 months. 50% or more counts as a warning. An answer without that line counts as no call, neither a hit nor a miss.

Every proportion carries its count, its sample size and a 95% Wilson interval. For Mentor, the Brier score gets a 95% range from resampling whole companies, and it's compared with always stating the set's own failure share. Lead time is the time from the first critical or high alert to the event, looking at any day in the 24 months before it, so it's a look back and not something a replay on a given date could have seen.

Later sources void the run. Every passage shown to Mentor, in every research round, and every source it cites must be dated on or before the replay date. One that is later or undated stops the run and nothing is written. Alert replays are checked for later or undated filings again at publishing. A limit we can't test away: the model behind Mentor was trained on text from after these dates and may remember the ending. Its sources can be checked but its memory can't.

Forward calls

A daily job is written to read the SEC's form index and record, on the filing day, each company that filed a late-report notice (NT 10-K, NT 10-Q, NT 20-F), plus an equal number of companies that filed on time that day, chosen by a fixed hash and not by hand. A call resolves as a failure on an 8-K Item 1.03 within 24 months, or as no failure if the company is still filing after 24 months. A company that stops filing without one stays unresolved and is counted as such. Customers' Watchlists, alerts and decisions are never used. We quote no forward score here: the first ones arrive when a flagged company fails, and the rest about 24 months after each call.

The answer-quality test

An answer passes only if all of these hold:

• Every quoted citation is found word for word in a passage the model was shown, no cited claim is unsupported, and at most 10% are only partly supported.
• Where the question has a known answer, pulled from its source, the answer gets it right.
• All seven criteria score at least 4 out of 5, and all three seats are impressed. The seats are finance professors, strategists and investors.
• The freshness date and disclaimer are shown, the answer isn't cut off, and compute stays inside the 10.78% ceiling.

The panel isn't made of people. The same model that writes the answers, Claude Opus 5.5, grades them against a written rubric, playing all three seats. It also judges whether cited claims are supported. No professor, strategist or investor has graded an answer. A high grade would mean better grades from this rubric, not better outcomes for a business.

What the live gate tests, and what it doesn't

Passing the bar and passing the gate are different things. The gate stops a code change from merging when answers measurably get worse than the last accepted run. It compares question by question and allows for the noise of regrading. Its latest recorded result is a pass, because nothing dropped, on a run in which 0 of 6 answers met the bar. A gate pass means "no worse than before". It doesn't mean "good".

The gate has real limits today. The set holds 64 questions, but every run on file used the same 6, so they aren't unseen. With 6 pairs it catches only large, systematic drops. It also needs a live run by EnemoAi's founder, and a passing result applies only to the exact code it measured.

Reading tests

Rules that read merger filings and credit agreements were measured on filings drawn by a salted hash from SEC form indexes and full-text search results, with the rules frozen before the held-out filings were read. One reader made the answer key, with no second reviewer. Neither rule set feeds answers yet. On first scoring, the second held-out group of 12 merger filings gave 31 right, 15 wrong and 24 missed on fairness ranges, and found 0 of 2 stated equity values. The second held-out group of 12 credit-agreement exhibits gave 4 right, 0 wrong and 9 missed on covenants. Both samples are small.

How samples are drawn

• Backtest companies: named by hand. 27 failures and 15 survivors of comparable distress, not a random or rule-based draw. The build script checks each one against its own SEC filing or FDIC page and stops if a check fails. Two banks are listed as excluded because they reported to the FDIC, not the SEC. All 42 are SEC filers, and failures are far more common here than in real life, so this set can't give a false-alarm rate or precision for companies in general.
• Answer-quality questions: chosen by us. 64 questions cover every mode and use public companies and agencies only. Known answers are pulled from their sources by a script.
• Forward calls: by rule. Every late filer on the day, and same-day on-time filers by a fixed hash.
• Reading tests: by rule. Salted-hash draws, as above.

What is and isn't pre-registered

• Backtest companies, dates and thresholds: not pre-registered. The set was picked after the outcomes were known, and the 12-month window and 50% threshold were written into code before the first run with no outside timestamp. The code is in a private repository, so you can't check its history yourself.
• The alert grading isn't tuned to this set. We fixed one reading bug the backtest found, in which a form that isn't an 8-K was read using 8-K item numbers. We left the rest alone on purpose, because tuning it to these companies would make the test meaningless.
• Forward calls are prospective by construction. Each call is recorded before its outcome exists, which makes these the cleanest test, along with the alert replay.
• Written but not in use. A held-out split fixed by a hash of each company's SEC number, and a larger set of companies drawn by rule, are both drafted and not merged. Because the 42 companies have already been scored, a split of them could never be a fully clean test.
• A rules-only score was pre-registered and passed the test we set, but that shows little. On 5 November 2026 a distress score built only from public SEC data, with no AI, was frozen and then scored once on 113 failures and 98 matched survivors from 2011 to 2025, none used in development. It flagged 86 of 113 failures (76%, range 67–83%) and 45 of 98 survivors (46%, range 36–56%). That separation of +0.30 (range +0.17 to +0.43) meets the test we set for it, but the test came from a baseline on a different set of companies. It doesn't clearly beat filing alerts on the same companies: +0.07, range −0.08 to +0.22. It fell from +0.51 on the development set, its stated probabilities are off, and it says nothing about the quality of EnemoAi's answers or about any population rate, since about half this set failed. It missed 27 failures and wrongly flagged 45 survivors. This score isn't offered to customers.
• A second model was pre-registered and met the same test on a small sample, without clearly beating simpler signals. On 7 November 2026 a model weighting the same kinds of filing signals, fitted only on the companies of the two earlier tests and checksummed, was registered before any test company was drawn. The draw took every eligible bankruptcy left after excluding all companies used before, and survivors were matched on industry, size and date as well as distress. Only 19 failures and 16 survivors were left. It flagged 13 of 19 failures (68%, range 46–85%) and 2 of 16 survivors (12%, range 3–36%): separation +0.56 (range +0.28 to +0.79). Its stated shares ranked well (AUC 0.87) but were too cautious at both ends. With so few companies, it doesn't clearly beat the first score (+0.14, range −0.19 to +0.49) or filing alerts (+0.21, range −0.19 to +0.58) on the same companies, and one fewer failure flagged would have missed the hit-rate part of the test. It missed 6 failures and wrongly flagged 2 survivors. It isn't offered to customers, and it says nothing about the quality of EnemoAi's answers.
• An earlier pre-registered test failed. On 2 November 2026 three candidate risk scores built from filings were registered and then tested on a fresh draw of 128 companies. None was confirmed, and none is offered to customers. The report isn't merged. It also records that the log of an earlier attempt showed some rates before the run was repeated unchanged.

How misses are published

The misses from the alert replay are below, because no run is on the track record yet. We re-ran it on 5 November 2026 on GitHub, with no AI, and it matched the first run of 1 November 2026.

The median warning came 389 days before the event, and 23 of 27 failures had some warning, some of them only days ahead. Read plainly, 43% against 40% is very little separation, and the ranges overlap.

• Failures with no flag on any replay date (9): SVB Financial Group, Silvergate Capital, Rite Aid, Hertz Global Holdings, Diamond Offshore Drilling, Intelsat, Windstream Holdings, PG&E, Washington Mutual.
• Survivors wrongly flagged (8): Carnival, American Airlines Group, Avis Budget Group, Kohl's, Ford Motor, Occidental Petroleum, Transocean, GameStop.

These are errors by EnemoAi's alerts. They say nothing about the companies.

What the code does. Publishing refuses a results file whose checksum shows it was edited, a run that stopped at its budget, and any run with a later or undated source. The track record is built to show every company and date, with hits, misses and false alarms together.

What the code doesn't do. Publishing is a command EnemoAi chooses to run, and the page shows only the latest published run. Nothing stops us from running several times and publishing the one we prefer, or from not publishing at all. The checksum catches accidents, but someone editing a file on purpose could recompute it. An outside reviewer would need to check this. The detailed test reports are in our private repository, so what you see here is what we've chosen to copy out.

What is not measured yet

• Whether EnemoAi helps a business. We have no measure of outcomes for businesses that follow, or ignore, its advice.
• Answer quality on unseen questions. Only the same 6 of the 64 questions have been run live.
• Human judgment of answers. No outside expert has graded any.
• Mentor's predictions. There's no completed backtest, so there's no hit rate or calibration.
• Fate Radar and Path to the top. Neither has been backtested.
• Out-of-sample prediction at scale. The two pre-registered tests drew 211 and 35 companies, and the second used up nearly every eligible bankruptcy left. Neither was large enough to show a clear edge over filing alerts. A larger test needs smaller companies or new failures as they happen.
• Forward calls. No score is quoted here.
• Private and smaller businesses, and companies outside the U.S. Every company in the backtest files with the SEC, so results may not carry over.
• Every other public source. Court, agency and non-U.S. records aren't scored as predictors.
• Translations. Checks catch formatting and numbers, not meaning, and answers in other languages haven't been run through the quality test.

Independent review

None has taken place. When one does, the reviewer's terms will let them publish what they find, including anything against us, and we'll link to it from this page. Until then, please read every number here as our own, unaudited.

EnemoAi provides information from public sources, not legal, investment, tax or accounting advice. Your business makes the final decision. Past results do not promise future ones.