金融研究Hugging Face Daily Papers28 Oct, 04:00 am SGTSIGNAL A67D95

Investor's Last Exam: Evaluating AI Agents on Equity Research Under a Frozen Information Set

ORIGINAL / Investor's Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?

This work introduces Investor's Last Exam (ILE), a benchmark that evaluates AI agents across the full equity-research process using a versioned, timestamp-governed financial database and an expert-led scoring framework under a strictly frozen information set. Its novelty lies in shifting the unit of evaluation from isolated capabilities to the evidence–assumption–judgment chain and in jointly measuring research quality, forecast accuracy, and simulated investment performance,

01 ABSTRACT

The paper builds ILE, an equity-research benchmark with a versioned financial database spanning 3.43 million document versions across nine source categories plus structured market data; retrieval and computation are governed by validated timestamps. Tasks are organized around professional analyst deliverables, and an expert-led framework links scoring criteria to sources, calculations, and assumptions via financial dependency graphs, with evaluation spanning 86 assignments. GPT-6 Astra and Claude Opus 5.5 lead overall at 68.1 and 66.9 out of 100, yet even Astra scores only 52.5 on Forecast & Model; the authors conclude that even state-of-the-art models fail to translate overall research capability into predictive performance.

02 KEY FINDINGS

ILE uses a versioned financial database (3.43 million document versions across nine source categories) plus structured market data, with validated timestamps governing retrieval and computation to form a strictly frozen information set.
The task taxonomy is organized around professional analyst deliverables, and an expert-led framework links scoring criteria to sources, calculations, and assumptions via financial dependency graphs, with evaluation spanning 86 assignments.
Evaluation jointly assesses research quality, forecast accuracy, and simulated investment performance; GPT-6 Astra and Claude Opus 5.5 lead overall, but Astra scores only 52.5 on Forecast & Model.

03 RELEVANCE

The benchmark shifts evaluation from isolated capabilities to the full evidence–assumption–judgment chain, helping researchers locate where analytical advantages fail to translate into forecasts and investment judgments and providing a traceable, diagnostic foundation for training and evaluating financial agents.

04 EVIDENCE BOUNDARY

Evidence boundaries: the findings rest on the paper's own ILE benchmark and 86 evaluation assignments, and the input does not establish whether its market, period, and asset coverage represents broader equity-investment settings. Scores reflect performance under this frozen information set and expert scoring framework and cannot be directly extrapolated to real investment capability or returns. The contrast between overall and Forecast & Model scores is the authors' interpretation that research capability fails to translate into predictive performance, an inference from a limited benchmark. The benchmark is expert-led, and the input provides no quantitative assessment of potential design or scoring bias.

Return to the primary source

AI GENERATED SUMMARY / DISCOVERED BY HUGGING FACE DAILY PAPERS

Read original ↗