TARSphere: A Full-Stack Framework for Long-Horizon Embodied Agents

01 ABSTRACT

TARSphere is a robot-oriented full-stack framework for long-horizon embodied agents that runs policies in a physics simulator with embodied NPCs and constrained dialogue. Its event-driven finite-state machine updates task states from completed actions and delivered information, while an embodied brain harness lets a frozen MLLM act through multi-level executable skills and revise decisions using observations, hierarchical memory, and execution feedback. The authors build TARSphere-Bench with 360 episodes across 30 indoor scenes to assess perceptual robustness, disruption adaptation, task update, human coordination, and intent understanding via five paired conditions; evaluation of 8 MLLMs shows general-purpose models can outperform embodied models, task updates and human interaction remain challenging, and frequent collisions and falls indicate the need to evaluate execution safety beyond task success.

02 KEY FINDINGS

1. The TARSphere framework runs robot policies in a physics simulator with embodied NPCs and constrained dialogue to support long-horizon tasks under physical and social constraints.
2. An event-driven finite-state machine updates task states from completed actions and delivered information; the embodied brain harness lets a frozen MLLM act through multi-level executable skills and revise decisions using observations, hierarchical memory, and execution feedback.
3. TARSphere-Bench contains 360 episodes across 30 indoor scenes and assesses perceptual robustness, disruption adaptation, task update, human coordination, and intent understanding via five paired conditions; evaluation of 8 MLLMs shows general-purpose models can outperform embodied models, task updates and human interaction remain challenging, and frequent collisions and falls indicate the need to evaluate execution safety beyond task success.