RoboDojo

RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies

Affiliations

Tianxing Chen ^{1 * §} , Yue Chen ^{4 * § †} , Zixuan Li ^{3 *} , Junyuan Tang ^{3 *} , Kailun Su ^{3 *} , Haoran Lu ^{4 *} , Weijie Wan ^{3 *} , Baijun Chen ^{1 *} , Songling Liu ^{4 *} , Haowen Yan ³ , Honghao Su ³ , Zhiyang Dou ⁶ , Kaixuan Wang ¹ , Dandan Zhang ¹³ , Yunze Liu ³ , Yan Qin ¹⁶ , Qiwei Liang ¹⁶ , Qiwei Wu ¹⁶ , Zijian Lin ³ , Wenwei Lin ³ , Yuran Wang ¹⁰ , Minghua He ⁴ , Tianshu Wu ⁴ , Ruihai Wu ⁴ , Jingquan Zhou ¹⁸ , Kai-Chong Lei ³ , Haibao Yu ¹ , Yuanfeng Ji ⁵ , Weiyang Jin ¹ , Guanyu Lin ⁹ , Xiaofan Li ¹⁷ , Qi Xiong ³ , Renjing Xu ¹⁶ , Zhongyu Li ¹² , Wenhao Chai ⁸ , Enze Xie ¹ , Ziwei Wang ¹¹ , Yao Mu ¹⁴ , Hao Dong ⁴ , Wojciech Matusik ⁶ , Mingyu Ding ^{7 †} , Wenbo Ding ^{3 †} , Ping Luo ^{1 †} , Masayoshi Tomizuka ^{2 †}

* Co-first authors · § Co-project leaders · † Corresponding authors

Published

Jul 6, 2026

Report

arXiv:2607.04434

Repository

RoboDojo-Benchmark/RoboDojo (RoboDojo) , XPolicyLab/XPolicyLab (XPolicyLab)

Leaderboard

View leaderboard

Email

RoboDojoCommittee@gmail.com

Community

WeChat Discord Feishu QQ

Share

Coming Soon

Generalist robot manipulation policies have made substantial progress, yet existing benchmarks remain limited in their ability to systematically and comprehensively evaluate policy capabilities. Many benchmarks rely on simple, short-horizon, or skill-narrow tasks that often share similar manipulation patterns and cover only limited capability dimensions.

Moreover, evaluations are commonly conducted either in simulation or in the real world alone: simulation provides efficient and scalable feedback but cannot fully capture physical deployment challenges, whereas real-world evaluation offers direct evidence of deployment performance but is costly, time-consuming, and difficult to reproduce.

Overview of RoboDojo

To address these limitations, we introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. RoboDojo includes 42 simulation tasks and 18 real-world tasks designed to cover diverse, challenging, and complementary manipulation capabilities. The simulation benchmark evaluates five capability dimensions: generalization, memory, precision, long-horizon execution, and open-vocabulary instruction following, while the real-world benchmark exposes policies to challenging physical-world deployment conditions.

To support large-scale evaluation, RoboDojo implements heterogeneous parallel simulation in Isaac Sim, substantially improving evaluation throughput. For real-world evaluation, RoboDojo provides RoboDojo-RealEval, a reproducible real-world evaluation system with remote cloud access. By standardizing the hardware setup, scene reset procedure, evaluation protocol, and deployment interface, RoboDojo-RealEval enables policies to be tested under consistent physical conditions.

In parallel, XPolicyLab provides a unified infrastructure for policy development and deployment, allowing policies to be integrated once and evaluated across RoboDojo simulation and real-world settings with minimal policy-side adaptation. We integrate 30 policies into XPolicyLab and evaluate them on RoboDojo, establishing a public leaderboard with systematic analysis of current policy performance. The website is available at robodojo-benchmark.com .

Citations

Cite our work

RoboDojo

Copy

@article{chen2026robodojo,
  title={{RoboDojo}: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies},
  author={Chen, Tianxing and Chen, Yue and Li, Zixuan and Tang, Junyuan and Su, Kailun and Wan, Weijie and Chen, Baijun and Lu, Haoran and Yan, Haowen and Su, Honghao and others},
  journal={arXiv preprint arXiv:2607.04434},
  year={2026}
}

XPolicyLab

Copy

@article{community2026xpolicylab,
  title={{XPolicyLab}: A Unified Standard and Open Ecosystem for Robot Policy Evaluation and Deployment},
  author={Community, XPolicyLab and Chen, Tianxing and Chen, Yue and Nian, Tian and Cai, Zijian and Chen, Guangyu and Lin, Wenwei and Liang, Qiwei and Xiang, Peicheng and Su, Kailun and others},
  journal={arXiv preprint arXiv:2608.09892},
  year={2026}
}

Future Extensions

Expanding the RoboDojo benchmark family.

Upcoming benchmark lines for mobile, dexterous, tactile, and humanoid manipulation, built on the same unified evaluation pipeline as RoboDojo.

• 

  Mobile Manipulation

  Benchmarking generalist mobile manipulation across diverse scenes and embodiments.

• 

  Dexterous Manipulation

  Benchmarking generalist dexterous manipulation with high-DoF end effectors.

• 

  Tactile Manipulation

  Benchmarking touch-driven policies that rely on contact-rich feedback.

• 

  Humanoid Robotics

  Benchmarking whole-body humanoid manipulation in unified sim-and-real settings.