Hand tracking

Hands are input sources, not a separate API

A tracked hand does not arrive through some parallel channel. It is an XRInputSource like any other, and if the hand-tracking feature was granted it carries an extra property: inputSource.hand, an XRHand. Everything you already wrote against inputSources, handedness and the select event keeps working — a pinch fires select the same way a trigger pull does.

This matters more than it sounds. The portable way to handle hands is usually to not handle them specially at all: let select drive your interactions, and reach for joint poses only when you actually need the skeleton. Applications that branch on "is this a hand" at the top of their input code end up maintaining two interaction systems that drift apart.

Hand tracking is an optional feature, requested by name. Put it in optionalFeatures rather than requiredFeatures unless your app is genuinely useless without it — a required feature that the device cannot grant fails the whole session request, and most headsets let the user turn hand tracking off in system settings.

Requesting hands and reading a joint

Joint poses are only available inside a frame callback, and only against a reference space. There is no way to ask "where is the index tip right now" outside the loop, because outside the loop the question has no well-defined answer.

const session = await navigator.xr.requestSession('immersive-vr', {
  // Optional, not required: the user can disable hand tracking in system
  // settings, and a required feature that cannot be granted kills the session.
  optionalFeatures: ['hand-tracking'],
});

function onFrame(time, frame) {
  for (const source of session.inputSources) {
    // No hand property means this input is a controller, or the feature
    // was not granted. Both are normal.
    if (!source.hand) continue;

    const indexTip = source.hand.get('index-finger-tip');
    const thumbTip = source.hand.get('thumb-tip');
    if (!indexTip || !thumbTip) continue;

    // getJointPose can return null on ANY frame: occlusion, the hand leaving
    // the tracking volume, or the runtime simply losing confidence.
    const a = frame.getJointPose(indexTip, referenceSpace);
    const b = frame.getJointPose(thumbTip, referenceSpace);
    if (!a || !b) continue;

    const dx = a.transform.position.x - b.transform.position.x;
    const dy = a.transform.position.y - b.transform.position.y;
    const dz = a.transform.position.z - b.transform.position.z;
    const distance = Math.hypot(dx, dy, dz);

    // a.radius is the runtime's estimate of the joint's thickness in metres.
    // Scaling the threshold by it adapts to small and large hands for free.
    const threshold = (a.radius + b.radius) * 1.4;
    if (distance < threshold) onPinch(source);
  }
}

Note the two separate null checks. get() returns undefined for a joint the runtime does not expose at all; getJointPose returns null for a joint it cannot locate this frame. Conflating them produces code that crashes the first time a hand passes behind the other one.

The 25 joints, and how they are named

Joint names are strings from a fixed vocabulary. There are 25 per hand: the wrist, four joints for the thumb, and five for each of the other four fingers.

wrist

The single root joint. Everything else is conceptually downstream of it, though the API gives you each pose independently rather than as a hierarchy.

thumb-metacarpal … thumb-tip

Four joints: metacarpal, phalanx-proximal, phalanx-distal, tip. The thumb is the exception — it has no intermediate phalanx, so loops written for five joints per finger break on it.

{index,middle,ring,pinky}-finger-metacarpal … -tip

Five joints each: metacarpal, phalanx-proximal, phalanx-intermediate, phalanx-distal, tip. The metacarpal sits inside the palm, so it is useful for orientation and almost never for contact.

XRJointPose.radius

A per-joint thickness estimate in metres, and the one piece of the API most people ignore. It is how you write a pinch threshold that works for a child and an adult without a calibration screen.

Why pinch detection is not a distance check

The naive version — distance between thumb tip and index tip below some constant — works on your own hand and misbehaves on everyone else's. Three things go wrong. Hands vary in size by more than the threshold itself, so a value tuned on a large hand never triggers on a small one. Tracking noise makes the distance jitter across the boundary, producing a burst of pinch and release events instead of one. And the moment a hand is occluded, the poses stop arriving and a held pinch silently ends.

The fixes are all small. Scale the threshold by the joint radii instead of hardcoding metres. Use hysteresis: a tighter distance to start a pinch than to end one, so the gesture cannot chatter. And treat a lost pose as "state unchanged for a few frames" rather than "released" — a pinch that drops the instant the hand tilts away is the most common complaint about hand-tracked interfaces.

Better still, when the interaction is simply "the user selected something", do not detect pinch at all. The runtime already does it, better than you will, and reports it as select. Custom gesture detection earns its cost only for gestures the runtime does not provide.

What the demo above is doing

The skeleton is built to the real joint layout — count them and you will find 25, with the thumb correctly one joint shorter than the fingers. The thumb and index curl toward each other while the other three hold a relaxed rest pose, which is roughly what a real pinch looks like to a tracker.

The distance driving the highlight is measured between the two fingertip joints in world space every frame, not baked into the animation. That is why moving the threshold changes the result rather than just changing a label: at 5 mm almost nothing counts as a pinch, and at 6 cm the gesture is "detected" while the fingers are still visibly apart — which is exactly the failure mode a too-generous threshold produces on real hardware.

Turning on the joint radius draws each joint's thickness estimate as a translucent sphere. When two of those spheres touch, the fingers have made contact — which is a far more robust definition of a pinch than any constant you could pick.

Where hand tracking actually works

Support is real but uneven, and it is a user-toggleable setting on most of these. Design for the case where it is simply off.

Checked 2026-09. Support changes with firmware; verify on the WebXR hand input spec and caniuse before relying on any row.

Mistakes that cost the most time

These share a shape: they all assume the hand is always there and always the same size.

Requiring the hand-tracking feature

Putting it in requiredFeatures means the session request rejects on any device where the user has hand tracking switched off. Ask for it optionally and degrade.

Assuming getJointPose is non-null

It returns null whenever tracking is lost — a hand behind the other hand, at the edge of the camera view, or moving fast. This happens constantly, not rarely.

A hardcoded pinch distance

Hand sizes vary by more than the threshold. Scale it by the joint radii and the same code fits every user.

Looping five joints across every finger

The thumb has four. Code that indexes a fixed five-element array per finger reads past the end and either throws or silently uses the wrong joint.

Reimplementing select as pinch

The runtime already fires select for a pinch, with its own tuning and hysteresis. A hand-rolled version is worse and does not work with controllers.

Further reading

The joint name vocabulary is worth having open the first time you write this code — the names are long and a typo silently returns undefined rather than throwing.

• W3C — WebXR Hand Input Module — The normative joint list, XRHand, and the radius semantics.
• MDN — XRHand — Practical reference for get() and the joint name strings.
• MDN — XRFrame.getJointPose() — What the returned pose contains, and when it is null.
• VR controllers — The input source model that hands plug into, and why select is the portable action.
• WebXR sessions — Requesting optional features, and the reference space joint poses resolve against.