Replanning-Guided Visual Memory for Long-Horizon VLA Policies
ORIGINAL / When Plans Change, Remember: Replanning-Guided Visual Memory for Vision-Language-Action Models
Proposes RePlanMem-VLA, which uses large disagreement between action predictions during policy replanning as an online memory-write signal to selectively store key visual frames, addressing the need for historical visual information in long-horizon VLA policies without expensive and redundant full history storage.
01 ABSTRACT
Long-horizon multi-stage VLA policies often require visual information no longer present in the current observation, and retaining complete history is computationally expensive and redundant, motivating selective visual memory. RePlanMem-VLA treats unusually large disagreement between temporally aligned predictions from consecutive action chunks as evidence of task-relevant events, triggering updates to a bounded visual memory. Compared 35 disagreement measures on single-task autoregressive decoder-only and Flow-Matching Transformers, validated on OpenVLA-OFT and π0.5 policies over MIKASA-Robo-VLA tasks, achieving ROC-AUC of 0.981-0.997, and preserving task-relevant observations until later decision points in 100% of episodes in memory-retention experiments.
02 KEY FINDINGS
- Uses large disagreement between action predictions during replanning as an online memory-write signal.
- Validated event detection across multiple VLA architectures and pretrained policies, achieving ROC-AUC of 0.981-0.997.
- Preserved task-relevant observations until later decision points in 100% of episodes in memory-retention experiments.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG