Terminal X's AI "matches senior investor judgment" 88.6% of the time. It is a grader agreeing with a grader

The headline and the sentence underneath

Headlines are a compression format, and this one compresses a great deal. "Terminal X matches senior investor judgment 88.6% with GPT-6.1," reads the customer story OpenAI published on October 20.¹ You picture a machine seated at the investment committee, voting the way the partners vote. Then you scroll to the third section and find the sentence the number actually belongs to: "Across 500 scored judgments, its Eval Agent on GPT-6.1 Sol agreed with the senior human reviewer 88.6% of the time, vs. 86.4% for the next-best model."¹

That is a different animal. The Eval Agent does not invest. It grades. It reads the output of other agents, the memos and theses Terminal X builds for buyout and credit desks, and checks each one against a 54-criterion pass/fail rubric: right sources, right comps, a narrative that holds, a recommendation tied to the house view.¹ The 88.6% measures how often that grader agreed with a human grader. A referee agreeing with another referee is useful information. It does not mean either of them can play.

What the 88.6% leaves out

Start with the room. "The senior human reviewer" is singular on the page. How many reviewers there were, at how many firms, over what period, on what kinds of deals: not disclosed. Nor do we know how many of the 500 outputs were sound to begin with, and that base rate matters more than it sounds. If most memos deserve a pass, a grader that stamps everything "pass" also posts a handsome agreement score, and nothing on the page rules that out.

The 11.4% of disagreements come without a direction either.² A grader that fails a good memo costs an analyst some rework. A grader that passes a drifting thesis costs whatever the deal costs. Only the second kind should keep anyone up at night, and the split is not disclosed.

Then the gap. 88.6% against 86.4% is 2.2 points, which the page dresses as "+220bps," a unit usually reserved for credit spreads.¹ What the 2.2 points are points of is less clear than it looks. The page ties "500 scored judgments" to the GPT-6.1 Sol run; it never says the rival was scored on the same 500 items, by the same reviewer, so we will not convert the gap into a head count.² Nor can the gap be called noise or signal from the outside. If both models graded the same items, the honest comparison counts only the cases where they disagreed with each other, and that table is exactly what the page leaves out. A single rate's sampling margin, roughly ±3 points at this size, describes each score alone; it settles nothing about the difference between two paired evaluations.² The "next-best model" is never named. Neither is the size of the pile behind the claim that GPT-6.1 Sol caught "about 25% more" departures from a fund's mandate: 25% more than a model that caught how many, at what false-alarm rate?³

The money figures are softer still. One private equity firm, unnamed, "reported" $2.4 million in annualized savings from automated due diligence and memo workflows, according to Terminal X.⁴ Annualized from what period, against which baseline, net of what fees: not disclosed. The "approximately 30% lower cost per query" compares two price points nobody prints.⁵

Who wrote this, and who else they wrote for

This is a customer story on the model maker's own website, under a button that says "Start building with OpenAI." That does not make it false. It makes it advertising, with the usual property that both parties had a reason to like the result.

The tell is in Terminal X's own archive. On July 27, its blog announced that Claude Opus 5 was live on the platform and, "after internal evaluations," stood out "as a state-of-the-art model for financial workflows," citing a roughly 9-point average accuracy gain in internal testing.⁷ Twelve weeks later, a different internal evaluation crowns a different vendor's model. The homepage, for good measure, shows a demo eval panel for a fictional firm with Claude at 91% and GPT at 87%.⁷ A mockup proves nothing. It does suggest the leaderboard is furniture that gets rearranged when a new guest arrives.

The serious objection

Here is the best case for Terminal X, and it is a real one. The company says it is model-agnostic by design and routes each task to whichever model scores best on the client's own evals.⁸ On that view, praising Claude in July and GPT-6.1 in October is not inconsistency; it is the product working. Models leapfrog each other every quarter, and a router that switches beats a loyalist that doesn't. Nor does anyone claim the grader replaces the investment committee. Used as triage, a model that flags most of what a senior reviewer would flag is worth having.

We accept the triage argument. It is the honest version of the story, and it is not the version in the headline. Routing by evals also cuts the other way: if the evals are what decide, the evals are what the client is really buying, which is exactly why the client should see them, rubric, answer keys and misses included. Here a client's evals have become a model vendor's marketing, with the denominators removed.

Who owns the miss

One quieter design choice deserves attention. Every fail the Eval Agent records "becomes a Memory," encoded as a new workflow instruction or test case for that desk.¹ When the grader is right, the system learns. When it is wrong, a model's mistake gets written into the house rules in the same ink as a partner's correction. The page says analysts correct the system; it does not say how often anyone audits the grader rather than the memo.

Terminal X's pitch is that firms should "own their judgment as data," in agents deployed on-premise or in their own cloud.¹ Ownership is a sound principle, and agents a firm runs on its own servers are a fair answer to the black-box rent much of the AI trade charges. But owning the agents is half the deal. A firm buying this should also own the scorecard. Until it is published, 88.6% is a number about a referee, printed by the company that sells the whistle.