Teleoperation#

Our system supports Oculus Quest 2 controller and keyboard for teleoperation.

Teleoperation

Setup Oculus Quest 2#

You need to set up Oculus Quest 2 on its first use.

  1. Turn on Oculus Quest 2.

  2. Download the Oculus app (Meta Quest) on your phone.

  3. Create an Oculus account.

  4. Pair Your Headset.

  5. Go to headset settings and select “Developer Mode.”

  6. Wear the headset and go to “Settings > System > Power” and set “Auto Sleep Headset to 4 hours”.

  7. Go to “Settings > System > Developer” and disable “Guardian.”

  8. Connect your Oculus Quest 2 to the client computer with USB-A to USB-C cable.

If you are not using Docker, you need to install oculus-reader package following this instruction. If you already have ADB and git-lfs installed, you can simply install oculus_reader:
pip install git+https://github.com/rail-berkeley/oculus_reader.git

Start Teleoperation#

  1. Before starting teleoperation, check whether the server daemon is running (see Run Server).

  2. In the client computer, execute the following script:

python furniture_bench/scripts/collect_data.py --furniture <furniture> --out-data-path </path/to/save/data>

Note

When you use Docker, make sure to mount the storage to save your data (Run Client). Otherwise, the collected data will be deleted once you close the docker container.

  1. (Optional) If you want to use only a keyboard, add an argument --device keyboard:

Note

Oculus Quest 2 sometimes asks permission for connection. So, if the controller does not work, wear the headset and press the “Allow” button.

  1. The command above will launch a GUI, as shown below. To start data collection, place furniture parts in their reset positions shown in the tool.

GUI Tool

  1. After completing an episode, press “A” (“B”) to save the trajectory labeled as “success” (“fail”).

Tip

  • The data will be saved at <out-data-path>/<furniture_name>/<current_timestamp>/.

  • Each episode is stored as follows:

|- 2023-01-16-10:48:51
  |- 2023-01-16-10:48:51.pkl                  # Demonstration data         (224x224 images, actions, rewards, etc.)
  |- 2023-01-16-10:48:51_color_image1.mp4     # Wrist camera RGB images    (1280x720)
  |- 2023-01-16-10:48:51_color_image2.mp4     # Front camera RGB images    (1280x720)
  |- 2023-01-16-10:48:51_color_image3.mp4     # Rear camera RGB images     (1280x720)
  |- 2023-01-16-10:48:51_depth_image1         # Wrist camera depth images  (1280x720)
  |- 2023-01-16-10:48:51_depth_image2         # Front camera depth images  (1280x720)
  |- 2023-01-16-10:48:51_depth_image3         # Rear camera depth images   (1280x720)