Hand Tracking Devices

Goal of this chapter: The hands are tracked with the device you chose - TRACIN's camera, gloves or a Meta Quest.

In Tracking Options, the MOVIN / Third-Party Glove dropdown sets what tracks the wrists and fingers. TRACIN's camera works on its own; gloves and a Meta Quest keep tracking the hands even when they're hidden from TRACIN's camera.

If you are using a different glove system and would like support, please share your request via the MOVIN discord server.

MOVIN Hand Tracking#

With MOVIN, TRACIN tracks wrist and finger movements using its camera - no gloves or extra devices needed.

1. Turn on Hand Tracking and select MOVIN#

   

   • In Capture, go to Tracking Options in the bottom panel.
   • Check Hand Tracking (1), then select MOVIN in MOVIN / Third-Party Glove (2).

2. Calibrate the actor#

   • Calibrate the actor as usual. Once calibration is done, the actor's wrists and fingers move with the performer in real time.

Tips for MOVIN Hand Tracking#

• Distance: keep the hands 1-4 m from TRACIN. For the best accuracy, stay within 1.5-3.5 m.
• One performer: tracking works best with a single person in the camera's view.
• Face the camera: keep the hands and wrists clearly visible, and stand facing TRACIN.
• Clothing: sleeves shouldn't cover the wrists. Dark clothing makes the hands harder to tell apart - wear something that contrasts with your hands.
• Keep TRACIN still: if the camera shakes or vibrates, hand tracking may become unstable.
• Lighting: see Fix the lighting in Set up your Camera.

Keep both hands in the camera's view#

TRACIN's camera sits about 95 cm above the ground, so the hands are tracked only while they're inside its field of view (FoV) and not hidden by the body. Check the Camera View in MOVIN Studio: body parts outside the FoV appear gray.

These poses lose the hands:

Integrate with Rokoko Smartgloves#

1. Select 'Rokoko' in the 'MOVIN / Third-Party Glove' dropdown.#

• In MOVIN Studio, go to Tracking Options in the bottom panel of the Capture screen.
• Turn Hand Tracking on, then select Rokoko in the MOVIN / Third-Party Glove dropdown.

2. Install Rokoko Studio and pair Smartgloves.#

• Install Rokoko Studio on your PC.

• Launch the app and create a new project. Next, create an actor named "MOVIN".
  • Be careful to write the name in capital letters. Integration may not work if "MOVIN" is not written in uppercase.
• Pair your Rokoko Smartgloves to Rokoko Studio.

• Drag and drop the gloves from the 'Capture devices' section onto the 'MOVIN' actor.
  • If the glove emojis appear beside the 'MOVIN' actor, the gloves are connected.
• Please refer to Rokoko's official documentation for the details.

3. Enable Unity Streaming in Rokoko Studio.#

• In Rokoko Studio, Click the 'Streaming' tab on the right side, then click the gear icon (Settings) under the Unity section. Set the IP and port as shown below, and check the 'Include connection' box.
  • IP: 127.0.0.1(localhost)
  • Port: 14043

• Activate the Unity stream by checking the 'Activate' box.
• Return to MOVIN Studio and confirm that wrist and finger motion is now reflected on the skeleton.

4. Click 'Calibrate ROKOKO' if the calibration is needed.#

• MOVIN Studio allows you to calibrate your Rokoko gloves directly within the app—no need to switch back to Rokoko Studio.
  • In Tracking Options, click Calibrate ROKOKO below the MOVIN / Third-Party Glove dropdown.
  • Hold the Rokoko calibration pose for 3 seconds to complete the process.

Integrate with MANUS Smart Gloves#

1. Select 'MANUS' in the 'MOVIN / Third-Party Glove' dropdown.#

• In MOVIN Studio, go to Tracking Options in the bottom panel of the Capture screen.
• Turn Hand Tracking on, then select MANUS in the MOVIN / Third-Party Glove dropdown.
• If a hand looks rotated, adjust MANUS left / right Y Rotation Offset below the dropdown.

2. Install MANUS Core and pair the gloves.#

• Download and install MANUS Core on your PC.

• Launch the software and pair your MANUS gloves with the system.
  • Refer to MANUS’s official documentation for full setup instructions.
• Once connected, check MOVIN Studio to ensure that wrist and finger movements are mirrored on the skeleton.

3. Calibration the glove in MANUS Core.#

• For full details, refer to MANUS’s documentation on calibration procedures.

Integrate with Meta Quest#

With the MOVIN Quest Companion app, a Meta Quest headset streams its hand tracking to MOVIN Studio over Wi-Fi. The performer wears the headset during capture, and its cameras track the wrists and fingers.

1. Turn on Developer Mode#

   The companion app isn't on the Meta Store, so the Quest needs Developer Mode to run it.

   1. Create a free Meta developer account at developers.meta.com/horizon.
   2. Sign in to the Meta Horizon app on your phone.
   3. In the app, go to Menu → Devices → select your Quest → Developer Mode → ON.
   4. Restart the Quest.

2. Install ADB on your PC#

   ADB installs the app from your PC onto the Quest. You only need it once, and any PC works - it doesn't have to be the one running MOVIN Studio.

   • Windows: open PowerShell and run winget install --id=Google.PlatformTools -e. Then open a new PowerShell window and run adb version to check.
   • Ubuntu: run sudo apt install adb android-sdk-platform-tools-common. The second package lets your account access the headset over USB.

3. Connect the Quest to your PC#

   1. Connect the Quest to your PC with the USB-C cable.
   2. Wearing the Quest, find the Allow USB debugging? dialog, check Always allow from this computer and select Allow.
   3. On the PC, run adb devices. Your Quest should be listed as device.

   Listed as unauthorized? Unplug and replug the cable, then allow USB debugging in the headset again.

4. Install the MOVIN Quest Companion app#

   1. Download MOVINQuestCompanion.apk - this link always gets the latest version.
   2. In the folder you downloaded it to, run adb install MOVINQuestCompanion.apk.

   To update to a newer version later, run adb install -r MOVINQuestCompanion.apk.

5. Select Quest in MOVIN Studio#

   1. In MOVIN Studio, go to Tracking Options in the bottom panel of the Capture screen.
   2. Turn Hand Tracking on, then select Quest in the MOVIN / Third-Party Glove dropdown.
   3. Choose Quest Input:
      • Hand Tracking: the Quest's cameras track your wrists and fingers.
      • Controllers: hold the Quest controllers. They drive the wrists, and pulling the trigger closes the fingers. Fingers aren't tracked individually.

6. Launch the app and connect#

   1. Make sure hand tracking is on in the headset: Settings → Movement Tracking → Hand Tracking.
   2. In the headset, open App Library, choose Unknown Sources from the dropdown at the top right, and launch MOVINQuestCompanion.
   3. On first launch, allow the Hand Tracking permission.
   4. Your PC appears in the list as hostname ip:port. Select it to connect.
      • Not in the list? Type the PC's IP address under Or type IP manually and select Connect.
      • To click, point with a controller and pull the trigger, or point with your hand and pinch.
   5. When the status shows ● Connected with ● L ● R, your hands are being tracked.

   MOVIN Studio lines the Quest up with TRACIN automatically - keep both hands in view and steady for a moment after connecting.

Network and firewall#

• Some office or public Wi-Fi networks block traffic between devices. If so, use a separate router with only the Quest and the MOVIN Studio PC on it.
• If Windows asks for network access when MOVIN Studio first starts, select Allow access.
• If your firewall blocks the connection, allow these UDP ports:

Troubleshooting#

When the app can't reach MOVIN Studio, it shows a hint under the status:

• Connected, but ○ L ○ R: your hands are out of the headset's view, or hand tracking is off in the headset. The connection itself is fine.
• No PC in the list: automatic discovery is blocked on your network - type the IP address instead.

For more installation and troubleshooting details, see the MOVIN Quest Companion repository.