LLM Financial Forecasts: Momentum and Commentary Tone Influence Earnings Surprise
ORIGINAL / Momentum, Commentary Tone, and Earnings Surprise in LLM Financial Forecasts
The study independently manipulates 20-day momentum, commentary tone, and whether actual EPS beats a manipulated consensus using synthetic earnings announcements. It provides 2,016 prompts each to six instruction/chat checkpoint models across different families and sizes. The novelty lies in systematically identifying input reliance in LLM financial forecasts, helping understand model sensitivity to non-fundamental information.
01 ABSTRACT
The study constructs synthetic earnings announcements to test how multiple language models respond to inputs such as momentum, commentary tone, and earnings surprise under a direct-answer protocol. The authors report matched changes in conditional answer-prefix scores and find that tone contrasts exceed EPS contrasts in most models, and momentum contrasts often exceed EPS as well. These results measure observable input reliance and numerical recognition, not internal reasoning mechanisms or actual return predictability.
02 KEY FINDINGS
- The primary EPS contrast across six checkpoints ranges from -0.19 to 86.80 percentage points.
- In five models, the primary tone contrast magnitude exceeds that of EPS; momentum exceeds EPS in five as well.
- Natural-word numerical diagnostic accuracy ranges from 51.0% to 95.8%.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.CL · ARXIV CS.LG · ARXIV Q-FIN