Tactile Foundation Model: A Large-Scale Framework for Dexterous Manipulation
ORIGINAL / $\mathcal{N}_0$-Foundation: Towards the Age of Tactile Intelligence
This work proposes a comprehensive framework for tactile-enabled embodied manipulation, integrating hardware, datasets, representation models, and benchmarks, aiming to address the lack of tactile information in existing robotic manipulation datasets and provide new data and model foundations for precise manipulation tasks.
01 ABSTRACT
The work presents a tactile manipulation framework, including tactile sensors, data collection systems, large datasets, visual-tactile representation models, and evaluation benchmarks. The authors constructed over 30,000 hours of synchronized visual-tactile data, released an open-source subset, trained representation models, and validated in both real and simulated environments.
02 KEY FINDINGS
- Proposes a tactile manipulation paradigm integrating sensing hardware, data, representation, and evaluation.
- Constructs a dataset with over 30,000 hours of synchronized visual-tactile demonstrations covering 450 tasks.
- Releases a 5,000-hour open-source subset to promote open research.
- Proposes a visual-tactile representation model that learns transferable representations across sensor designs.
- Provides real and simulated benchmarks, showing policies benefit from physical contact state.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG