Open Reality

How it works

Raw demo video in, policy-ready episodes out

OpenReality runs every demo through one pipeline: reconstruct it in metric 3D, ground and QA the trajectories, then export clean LeRobot episodes your policies can train on.

  1. input · capture

    Demo video

    Your teleop or human-hand demo. Ordinary RGB video, no rig or depth sensor.

    mp4 · rgb frames
  2. reconstruct · slam

    3D reconstruction

    VGGT-SLAM recovers metric, gravity-aligned geometry and per-frame camera pose.

    point cloud · SE(3) pose
  3. ground + qa · repair

    Grounding + QA

    Objects get labeled and anchored in 3D. Trajectories are extracted, checked, and flagged where a take fails.

    trajectories · labels · flags
  4. export · dataset

    LeRobot dataset

    Clean, scale-consistent episodes in LeRobot format, ready to train a policy.

    lerobot · episodes
preview · experimental
Experimental / preview

Isaac sim preview

Optional preview of episodes replayed in Isaac. Unvalidated, with no sim-to-real claims yet.

usd · sim preview

pipeline · monocular video → metric 3D → grounded + QA trajectories → lerobot episodes · isaac preview (experimental)