Ongoing project · Robot Teleoperation Junho Lee

Teleop Demonstration Accumulation for Diffusion Policies

Diffusion policies need a lot of demonstrations. This is a WebXR-based browser teleoperation system that drives a real Piper arm and a PyBullet twin, so demonstrations can be collected from any headset or phone browser without extra software.

The end effector follows the twist of the handheld device: twist is sampled at 10 Hz and the robot tracks it at 200 Hz.

Control link

Teleoperation.

Teleoperation system with Piper (Agile-X) and RealSense L515

Teleoperation system in PyBullet with Franka Panda

Policy rollout

Deployment.

Success

Fail

Sensors

Captured input.

RGB input captured from the PyBullet teleoperation environment

RGB input

Depth input captured from the PyBullet teleoperation environment

Depth input