Agents for earnings season

Before a public company reports earnings, every number in every draft gets traced back to a source. Finance people call it tick-and-tie. It's careful, repetitive, and high stakes, and every figure has to be checked again whenever the draft changes.

I spent the first part of my career doing this kind of work. That's why I trust this problem to an agent only when the system can show its evidence, and only when we've measured how often it's wrong.

The platform serves Caterpillar's 50-person core Finance and Investor Relations teams, and expansion into Legal and Communications workflows is planned.

A toy version of the verification loop, running on a made-up draft. Run the check, then click any figure to see its evidence.

Each draft is split into individually checkable claims, and every claim is searched two ways at once. Financial documents are full of exact tokens like segment names, fiscal periods, and figures, where keyword search beats embeddings. Narrative restatements need semantic search. Fusing and reranking both is what gets the right source chunk back 99.9% of the time.

A verifier agent then compares each claim to its evidence and passes it, or flags it with a citation for a reviewer. Around that, a LangGraph multi-agent system on Azure AI Foundry runs the validation, tick-and-tie, and drafting workflows.

I built the evaluation framework before tuning the system. A missed error is the expensive failure. A false alarm is cheaper, but enough of them and people stop trusting the tool. So the framework tracks catch rate, false-alarm rate, and retrieval hit rate separately, and every change to prompts, chunking, or models runs against it.