Install & Download

This guide walks through the standard native RoboDojo setup path: check the system requirements, install the codebase and dependencies, download assets and datasets, and update local asset paths.

1. System requirements

Supported platforms

RoboDojo is built on Isaac Sim 5.1 and Isaac Lab. We recommend Linux (Ubuntu 22.04 x64) with an NVIDIA GPU.

Most platform constraints come from Isaac Sim and Isaac Lab, so check their official requirements if you are using a different driver, CUDA version, or workstation setup.

Hardware and software

Item	Requirement
OS	Ubuntu 22.04 (Linux x64) recommended
RAM	32 GB or more
GPU VRAM	16 GB or more (extra VRAM may be needed for rendering-heavy workflows)
NVIDIA driver	570 or 580 recommended on Linux
CUDA	12.8 recommended (installed with PyTorch during setup)

For the latest Isaac Sim hardware notes, see the official system requirements.

2. Install Vulkan

Vulkan is required for Isaac Sim rendering. Skip this step if vulkaninfo already reports a working Vulkan installation.

sudo apt install libvulkan1 mesa-vulkan-drivers vulkan-tools
vulkaninfo | head

3. Clone & install RoboDojo natively

3.1 Clone the repository

git clone https://github.com/RoboDojo-Benchmark/RoboDojo.git

3.2 Run the install script

The native install path creates the RoboDojo conda environment and installs Isaac Sim 5.1, Isaac Lab, CuRobo, and the remaining Python dependencies:

bash scripts/install.sh -i

When finished, activate the environment:

conda activate RoboDojo

3.3 Resume from a failed step

If installation stops midway, resume from a named step instead of restarting from scratch:

bash scripts/install.sh --from STEP_NAME

Available step names:

Step name	Resumes from
`system`	System package setup
`conda`	Conda environment creation
`base_deps`	Base Python dependencies
`submodules`	Git submodule initialization
`isaacsim`	Isaac Sim installation
`isaaclab`	Isaac Lab installation
`curobo`	CuRobo installation

Example - retry after an Isaac Sim failure:

bash scripts/install.sh --from isaacsim

3.4 Verify optional tools

ffmpeg is required for some data-processing workflows. On Ubuntu, the install script tries to install it automatically; you can verify it manually with:

ffmpeg -version

If missing, install from ffmpeg.org or via sudo apt install ffmpeg.

4. Download assets

Simulation assets, including objects, textures, and embodiments, are hosted on ModelScope. Git LFS is required before downloading them.

# Install git-lfs once (Ubuntu)
sudo apt install git-lfs
git lfs install

# Download assets into ./Assets
bash scripts/init_assets.sh

After download, the assets directory should look like:

Assets/
├── Background/
├── Eval_Layout/
│   └── ...
├── Material/
└── ...

5. Update embodiment paths

After the assets are in place, point robot configs to the local asset directory:

python utils/update_embodiment_config_path.py

Run this command again if you move the repository or re-download assets to a different location.

6. Download data

RoboDojo datasets are hosted on Hugging Face and ModelScope. Git LFS is required.

sudo apt install git-lfs
git lfs install

Run the downloader without arguments to list the available formats, sizes, and destinations:

bash scripts/RoboDojo/download_data.sh

Examples:

# LeRobot v3.0, 120 GB → ./data/RoboDojo_lerobot_v30_video
bash scripts/RoboDojo/download_data.sh huggingface lerobot_v3.0

# LeRobot v3.0 with end-effector values, 120 GB
# → ./data/RoboDojo_ee_lerobot_v30_video
bash scripts/RoboDojo/download_data.sh modelscope lerobot_v3.0_ee

# LeRobot v2.1, 64 GB → ./data/RoboDojo_lerobot_v21_video
bash scripts/RoboDojo/download_data.sh huggingface lerobot_v2.1

# Simulation HDF5, 523 GB → ./data/RoboDojo
bash scripts/RoboDojo/download_data.sh huggingface hdf5

# Simulation HDF5 with depth, approximately 4.5 TB → ./data/RoboDojo_depth
bash scripts/RoboDojo/download_data.sh modelscope depth

# Real-world HDF5, 273 GB → ./data/RoboDojo_real
bash scripts/RoboDojo/download_data.sh huggingface real

Notes:

Format availability may differ between Hugging Face and ModelScope. The downloader checks the selected mirror before pulling LFS objects.
depth, RoboDojo_depth, and the compatibility alias hdf5_w_depth all select the RoboDojo_depth folder.
The depth export is approximately 4.5 TB. Check free disk space before downloading it.
Runtime observation/action dictionaries used by policy adapters are documented in XPolicyLab → Standard Data Formats.

HDF5 data format

Directory layout

data/RoboDojo/
└── <task_name>/                 # for example, stack_bowls
    └── <embodiment>/            # for example, arx_x5
        ├── data/
        │   └── episode_XXXXXXX.hdf5
        └── preview_video/
            ├── episode_XXXXXXX_cam_head.mp4
            ├── episode_XXXXXXX_cam_left_wrist.mp4
            └── episode_XXXXXXX_cam_right_wrist.mp4

The other HDF5 releases use the same task/embodiment layout:

data/RoboDojo_depth/: simulation episodes with depth observations
data/RoboDojo_real/: real-world episodes; embodiments include arx_x5 and piper_x

Simulation episode structure

episode_XXXXXXX.hdf5
├── data_format_version                        scalar string, "v1.0"
├── instruction                                scalar string
├── additional_info/
│   └── frequency                              scalar int64, 25
├── vision/
│   ├── cam_head/
│   │   ├── colors                             (T,), JPEG bytes, dtype |S...
│   │   ├── approximate_depths                 (T, 480, 640), uint16, RoboDojo_depth only
│   │   ├── intrinsic_matrix                   (3, 3), float64
│   │   ├── extrinsic_matrix                   (T, 4, 4), float64
│   │   └── shape                              (3,), int64, [480, 640, 3]
│   ├── cam_left_wrist/                        same fields as cam_head
│   └── cam_right_wrist/                       same fields as cam_head
├── state/                                     current state at timestep t
│   ├── left_arm_joint_states                  (T, 6), float64
│   ├── left_ee_joint_states                   (T, 1), float64
│   ├── left_ee_poses                          (T, 7), float64
│   ├── left_delta_ee_poses                    (T, 7), float64
│   ├── right_arm_joint_states                 (T, 6), float64
│   ├── right_ee_joint_states                  (T, 1), float64
│   ├── right_ee_poses                         (T, 7), float64
│   └── right_delta_ee_poses                   (T, 7), float64
└── action/                                    target state for timestep t + 1
    ├── left_arm_joint_states                  (T, 6), float64
    ├── left_ee_joint_states                   (T, 1), float64
    ├── left_ee_poses                          (T, 7), float64
    ├── right_arm_joint_states                 (T, 6), float64
    ├── right_ee_joint_states                  (T, 1), float64
    └── right_ee_poses                         (T, 7), float64

End-effector poses use [x, y, z, qw, qx, qy, qz] and are expressed in the world base frame, not the robot base frame. To convert an EE pose into the robot-base frame, use the robot spawn pose from robot_config (default_root_pos and default_root_rot in env_cfg/robot/*.yml) as the world-to-robot-base transform.

State and action alignment

At timestep t:

state[t] is the current robot state.
action[t] is the absolute target state for the next frame.

For shared joint and gripper fields:

action[t] = state[t + 1]          for t = 0 ... T - 2
action[T - 1] = state[T - 1]     final-frame padding

End-effector pose targets follow the same next-frame alignment. action is an absolute target, not a delta. For imitation learning, use (vision[t], state[t]) as input and action[t] as the target.

Loading an episode

Use XPolicyLab’s image decoder because it handles both simulation and real-world JPEG storage layouts:

from XPolicyLab.utils.load_file import load_hdf5
from XPolicyLab.utils.process_data import decode_image_bit

episode = load_hdf5(
    "data/RoboDojo/<task>/<embodiment>/data/episode_0000000.hdf5"
)
rgb = decode_image_bit(episode["vision"]["cam_head"]["colors"][0])

With this setup, you can test data conversion, model loading, training scripts, and debug-mode evaluation before connecting to a simulator-backed benchmark.

7. Download ckpt

RoboDojo policy checkpoints are hosted on Hugging Face and ModelScope. Git LFS is required before downloading them.

sudo apt install git-lfs
git lfs install

Run the script without arguments to print the command format:

bash scripts/RoboDojo/download_ckpt.sh

Choose a download source and the policy whose checkpoints you need. For example, download the Pi_0 checkpoints from Hugging Face:

bash scripts/RoboDojo/download_ckpt.sh huggingface Pi_0

Or download the same checkpoints from ModelScope:

bash scripts/RoboDojo/download_ckpt.sh modelscope Pi_0

The corresponding policy adapter must already exist under XPolicyLab/policy/<POLICY>. The downloader will not create a missing adapter directory; install or initialize the adapter before requesting its checkpoints.

If the remote repository has no checkpoint folder matching the requested policy name, the script exits with a non-zero status instead of silently succeeding. It also validates that the remote ckpt/RoboDojo root exists for the selected source and revision before sparse checkout.

The script downloads only the selected policy into a local cache, then creates the checkpoint path expected by XPolicyLab:

XPolicyLab/policy/<POLICY>/checkpoints

When you download checkpoints for a second policy, sparse checkout adds the new remote directory instead of replacing the previous selection, so existing checkpoint symlinks under other policies stay valid.

If ROBO_DOJO_CKPT_CACHE (or the default cache path) is relative, it is resolved from the directory where you run the script.

If that checkpoints path already contains a different file, directory, or symbolic link, the script preserves it with a timestamped .partial.<timestamp> suffix before creating the new link.

8. Optional: Install with Docker

Docker installation details

Docker is useful when you want a reproducible simulator/evaluation container. The RoboDojo image contains Isaac Sim 5.1, Isaac Lab, CuRobo, the RoboDojo Python stack, and the lightweight XPolicyLab client used by the eval client.

The image does not include Assets/, policy dependencies, or checkpoints. Download assets on the host, mount them into the container, and run your policy server outside the container over WebSocket.

8.1 Docker prerequisites

Need	Details
NVIDIA GPU + driver	Driver 570 or newer is recommended for CUDA 12.8. Check with `nvidia-smi`.
OS	Linux, tested on Ubuntu 22.04.
Disk space	The image is large because it includes Isaac Sim and cached extensions. Keep at least 300 GB free for the image and build cache.
Assets	A host-side `Assets/` directory downloaded with `bash scripts/init_assets.sh`. Assets are mounted at runtime, not baked into the image.

Prepare host-side assets before running container evaluation:

bash scripts/init_assets.sh
python utils/update_embodiment_config_path.py

8.2 Install Docker and NVIDIA Container Toolkit

If Docker and the NVIDIA Container Toolkit already work with --gpus all, skip to the image build step.

sudo bash docker/install_docker_nvidia.sh
newgrp docker

Verify that Docker can see the GPU:

docker run --rm --gpus all nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04 nvidia-smi

If you are on a restricted China network, the installer can use TUNA/USTC apt mirrors and Docker Hub registry mirrors. It auto-detects this by default; force it with USE_CN_MIRRORS=1 or disable it with USE_CN_MIRRORS=0.

8.3 Build the image

From the repository root:

docker build -t robodojo:cuda12.8 .

The first build can take a long time because it downloads Isaac Sim extensions and compiles CuRobo kernels. Later builds should reuse Docker layer cache.

8.4 Smoke-test Docker

Run the end-to-end Docker smoke procedure:

bash docker/smoke_docker.sh run

This builds the image if needed, starts the bundled demo_policy server on the host, runs one container evaluation, and checks that a result file is written. The demo policy outputs zero actions, so a success_rate of 0.0 is expected; the smoke test is checking the simulator, rendering, WebSocket connection, and result writing path.

Monitor a running smoke test from another terminal:

bash docker/smoke_docker.sh monitor

The smoke script launches the demo server on the host using the native RoboDojo conda environment. If you do not have the native environment and only want Docker, skip the demo smoke step and connect the container to your own policy server.

8.5 Evaluate an external policy server

Start your policy server outside the container and bind it to 0.0.0.0, not only 127.0.0.1, so the container can reach it. Then run the simulator client in Docker:

docker run --rm -it \
  --gpus all --network host --ipc host \
  -v "$PWD/Assets:/workspace/RoboDojo/Assets:ro" \
  -v "$PWD/Assets:$PWD/Assets:ro" \
  -v "$PWD/eval_result:/workspace/RoboDojo/eval_result" \
  -v "$HOME/.cache/warp:/root/.cache/warp" \
  -v "$HOME/.local/share/ov:/root/.local/share/ov" \
  -v "$HOME/.cache/ov:/root/.cache/ov" \
  robodojo:cuda12.8 \
  bash scripts/robodojo.sh client \
    --task stack_bowls \
    --policy-name <POLICY_NAME> \
    --policy-host 127.0.0.1 --policy-port 9999 \
    --ckpt <CKPT_NAME> --eval-num 1

Single-task mode requires --task, --policy-host, --policy-port, and either --policy-name or --policy-dir. Omit --task to run batch client mode against multiple tasks (optional --only, --dimension, --gpu-ids, and comma-separated ports for multiple policy servers). Common options include --ckpt, --eval-num, --action-type, --env-cfg, and --env-gpu. Add --dry-run to inspect the resolved simulator command without launching Isaac Sim.

8.6 Required mounts

The two Assets mounts are intentional:

Mount	Why it is needed
`-v "$PWD/Assets:/workspace/RoboDojo/Assets:ro"`	Gives the container access to robot, object, material, and layout assets.
`-v "$PWD/Assets:$PWD/Assets:ro"`	Preserves absolute host paths baked into CuRobo configs by `utils/update_embodiment_config_path.py`.
`-v "$PWD/eval_result:/workspace/RoboDojo/eval_result"`	Persists result artifacts on the host.
Cache mounts under `~/.cache/warp`, `~/.local/share/ov`, and `~/.cache/ov`	Reuse CuRobo kernels and Isaac Sim extension/shader caches between runs.

The second Assets mount prevents errors such as:

ValueError: /abs/host/.../X5A.urdf is not a file

CuRobo robot configs may contain absolute host paths generated by utils/update_embodiment_config_path.py, so the container must be able to access the same absolute path.

8.7 Networking

Policy server hostnames depend on Docker networking:

Policy server location	Docker mode	`--policy-host`
Same host as the container	`--network host`	`127.0.0.1`
Same host, bridge network	`--add-host=host.docker.internal:host-gateway`	`host.docker.internal`
Another machine	default networking	The machine’s IP or hostname

If the client cannot reach the policy server, first check that the server is listening on the expected port and is bound to 0.0.0.0.