Interaction-Constrained Offline RL for Autonomous Driving Distribution Shift
ORIGINAL / Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving
This work argues that offline reinforcement learning for autonomous driving must control not only policy action-space distribution shift but also degradation in joint support between ego and surrounding agents' futures, termed interaction distribution shift (IDS). The authors propose ICDP, a framework that recovers a residual interaction-support component via contrastive density-ratio estimation from the joint data distribution, suppressing high-value yet interaction-unsuppor
01 ABSTRACT
The authors contend that in interactive environments such as autonomous driving, constraining distribution shift only in the policy's own action space is insufficient, because a candidate ego trajectory may be well supported marginally but poorly supported jointly with surrounding-agent behavior in the logged interaction. They introduce interaction distribution shift (IDS) and show that joint-support degradation decomposes exactly into an ego-support component and a residual interaction-support component. ICDP recovers the latter via contrastive density-ratio estimation, avoiding explicit joint-density modeling, surrounding-agent prediction, or reactive simulator rollouts.
02 KEY FINDINGS
- Introduces interaction distribution shift (IDS): insufficient joint support between ego trajectories and surrounding agents' futures, beyond policy action-space shift.
- Joint-support degradation decomposes into an ego-support component and a residual interaction-support component.
- ICDP recovers interaction compatibility via contrastive density-ratio estimation, requiring no joint density modeling, surrounding-agent prediction, or simulator rollouts.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG