Kitting Data Samples

Twelve episodes from our bimanual kitting cell: four expert demos and eight on-policy rollouts of a fine-tuned π0.5 policy, labeled frame by frame. Raw data is in the download at the bottom.

Watch an Intervention ("Put NUt in the bag")

put the bolt in the bag — 19.0 s. The policy runs 4.5 seconds and stalls; the operator takes over, recovers it, and hands back twice before the policy finishes. Every frame carries this flag, plus a per-frame time-to-success target. Click the timeline to seek.

The Episodes

collect expert demos → fine-tune → run rollouts autonomously → capture interventions, label failures and recoveries → retrain → repeat

A Value Function Trained on This Data

We trained a value function on this data’s time-to-success labels. Across 215 episodes, its predictions correlate 0.86 (median) with ground truth. Here it reads one rollout:

The prediction climbs on approach, crashes at the failed grasp, stays low through the takeover, and climbs to done once the policy finishes.

What’s in an Episode

Files

Per-frame trajectory (parquet), three camera streams, and an H.264 overhead preview.

Trajectory

state and actions: float32[14] = [left q0–q5, left gripper, right q0–q5, right gripper]. Joints absolute, in radians; grippers 0–1 (0 = closed). Actions are absolute poses — leader arms during human frames, policy output otherwise.

Labels

is_intervention per frame; episode_outcome per episode (autonomous success / success after intervention / failure); time_remaining per frame (value target); a language instruction per episode.

Cameras

3 streams per episode, 224×224 at 50 fps, frame-synced with the trajectory. ELP 170° overhead + 180° gripper cameras.

Platform

2× YAM arms, 26.50″ apart; state and actions at 50 Hz on the camera clock.

Format

LeRobot format; dataset metadata included.

Full Package