Design-Based Intervals for Downsampled Cross-Sectional Market Aggregates in Bandwidth-Constrained Financial Data Pipelines
ORIGINAL / Approximate Design-Based Intervals for Downsampled Cross-Sectional Market Aggregates: A Randomized Design for Bandwidth-Constrained Financial Data Pipelines
To address the inability of deterministic Top-k selection to assess aggregation error in downsampled cross-sectional options panels, a design-based interval estimation method using probability sampling is proposed to improve aggregation accuracy and provide error quantification.
01 ABSTRACT
Financial institutions often use deterministic Top-k selection by open interest for downsampling, but this rule provides no aggregation error estimate, making risk metrics unauditable. The authors prove that under such target-blind rules the target is not identified, only bounded intervals are possible; they propose probability sampling with inclusion probabilities proportional to open interest, paired with the Hájek ratio estimator and design-based variance estimator to construct approximate intervals for every aggregate. Experiments show the method reduces mean absolute error of OI-weighted implied volatility by 35% at 10% retention and provides narrower intervals.
02 KEY FINDINGS
- Deterministic Top-k selection (e.g., by open interest) provides no aggregation error estimate; the target is not identified, only bounded intervals are possible.
- Probability sampling with inclusion probabilities proportional to open interest, combined with the Hájek ratio estimator and linearized variance estimator, generates approximate design-based intervals for each aggregate.
- On 80 US trading sessions and 83 Hong Kong options sessions, the randomized design reduces mean absolute error of OI-weighted implied volatility by 35%, with intervals significantly narrower than the worst-case intervals of Top-k.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV Q-FIN