Open Reality

Before / after

A raw demo in, a trainable episode out

Start from the demonstrations your team already has. OpenReality returns the same episode with metric geometry, trajectories, grounded labels, and quality flags, in LeRobot format your policies train on directly.

beforeRaw teleop / human-hand demo

  • RGB framesmp4 · rgb
  • Action / state logcsv · raw

afterRefined LeRobot episode

  • RGB framesmp4 · rgb
  • +Metric depthdepth · per-frame
  • +Camera + gripper trajectorySE(3) · per-frame
  • +Grounded object labelslabels · 3D anchored
  • +Per-frame quality flagsflags · drift · occlusion

Load it like any LeRobot dataset

from lerobot.common.datasets.lerobot_dataset import LeRobotDataset

ds = LeRobotDataset("openreality/your-demos-refined")
ep = ds[0]
ep["observation.images.cam"]   # rgb frames        (kept)
ep["observation.depth"]        # metric depth      (added)
ep["observation.state"]        # SE(3) trajectory  (added)
ep["annotations.objects"]      # grounded labels   (added)
ep["quality.flags"]            # failure flags     (added)

Validation in progressWhat we are measuring next

  • Policy liftSame demos, same model, baseline vs refined. Metric = task success and data-efficiency.
  • Dataset salvageDemos flagged for drift, occlusion, missing contact, or bad calibration, then repaired or rejected.