Depth sensing

What the module actually hands you

Depth sensing gives you, once per frame per view, an image in which every value is a distance to the nearest real surface along that direction. That is all. Everything people associate with the feature — occlusion, placing objects on things, physics against the room — is something you build on top of that one image.

The image is small. On current Android hardware it is around 160 by 90, against a camera feed of two megapixels or more. It is also noisy, and it has holes: transparent surfaces, dark surfaces, and anything beyond the sensor range come back as garbage or as nothing at all. Treating it as a reliable depth image of the room is the fastest way to ship something that looks broken in a kitchen with a glass door.

What it is good for is exactly the thing that makes AR stop looking like a sticker on the screen. A virtual object that disappears correctly behind a real chair is read as being in the room. The same object drawn on top of everything is read as being on the display. The difference is one comparison per fragment.

Requesting it, and reading a frame

The request carries preferences, not requirements, and the session may hand you something you did not ask for. Reading back what you actually got is mandatory — the CPU and GPU paths are different APIs, and the data formats have different unpacking.

const session = await navigator.xr.requestSession('immersive-ar', {
  requiredFeatures: ['depth-sensing'],
  depthSensing: {
    // Both are ordered preferences. You may get the other one.
    usagePreference: ['gpu-optimized', 'cpu-optimized'],
    dataFormatPreference: ['luminance-alpha', 'float32'],
  },
});

// Read what was actually granted. Branching on what you *asked* for is
// the first bug in most depth-sensing code.
const usage = session.depthUsage;        // 'cpu-optimized' | 'gpu-optimized'
const format = session.depthDataFormat;  // 'luminance-alpha' | 'float32' | ...

function onFrame(time, frame) {
  const pose = frame.getViewerPose(localSpace);
  if (!pose) return;

  for (const view of pose.views) {
    if (usage === 'cpu-optimized') {
      const depth = frame.getDepthInformation(view);
      if (!depth) continue;   // No depth this frame is normal, not an error.

      // Normalized view coordinates, not pixels. Returns metres.
      const centre = depth.getDepthInMeters(0.5, 0.5);

      // Reading the raw buffer instead: values are in the buffer's own
      // units and must be scaled.
      // const raw = new Uint16Array(depth.data)[y * depth.width + x];
      // const metres = raw * depth.rawValueToMeters;
    } else {
      const depth = glBinding.getDepthInformation(view);
      if (!depth) continue;

      gl.bindTexture(gl.TEXTURE_2D, depth.texture);
      // This transform maps normalized view coords to depth-buffer coords.
      // The buffer is NOT aligned to the viewport. Skipping this is the
      // single most common reason occlusion is offset or rotated.
      setUniformMatrix('uDepthFromView', depth.normDepthBufferFromNormView.matrix);
      setUniform('uRawValueToMeters', depth.rawValueToMeters);
    }
  }
}

getDepthInformation returns null whenever the runtime has no depth for that view this frame — during startup, on a sudden move, or when the sensor is saturated. Draw the previous frame's result or skip occlusion for a frame; do not tear anything down.

The names you have to get right

The module is small but every one of these has bitten somebody. Four of the six are about not trusting what you asked for.

usagePreference

cpu-optimized gives you an ArrayBuffer you can read values out of; gpu-optimized gives you a WebGL texture you can only sample in a shader. Ask for what you will actually use — reading back a GPU texture per frame stalls the pipeline.

dataFormatPreference

luminance-alpha packs a 16-bit value across two channels; float32 gives metres directly. The packed format is more widely available, so code written only against float32 fails on the majority of devices.

rawValueToMeters

The scale from buffer units to metres. It is a property of the frame, not a constant, and hard-coding a value that happened to work on one device is how a project acquires a mysterious per-device scale factor.

normDepthBufferFromNormView

The transform from normalized view coordinates to depth buffer coordinates. The depth buffer has its own aspect ratio and orientation. Omit this and occlusion is stretched, offset, or rotated ninety degrees — and it will look almost right in portrait, which is why it survives review.

getDepthInMeters(x, y)

The convenience accessor on the CPU path. x and y are normalized view coordinates in 0..1, not pixel indices, and it throws when they are out of range rather than clamping.

XRWebGLBinding.getDepthInformation

The GPU-path entry point. It lives on the binding, not on the frame, which is easy to miss when porting code between the two usage modes.

Why the edges look like stairs, and what to do about it

A depth buffer around 160 pixels wide, stretched across a full-resolution viewport, gives you roughly one depth sample per eight or ten screen pixels. A hard comparison against it produces exactly what you would expect: a blocky, aliased silhouette that shimmers as the user moves. This is not a bug in your shader.

The usual fix is to soften the comparison rather than sharpen the data. Instead of a binary test, fade the virtual fragment out over a few centimetres of depth difference. The edge becomes a short gradient, which reads as a soft shadow rather than as jagged geometry, and it costs one smoothstep. The demo above does this — switch between the hard and soft modes at a low resolution and the difference is obvious.

The second half of the fix is knowing when not to trust the buffer at all. Depth is unreliable at silhouette edges, where a single sample straddles a near and a far surface. Some implementations expose a confidence signal; where there is none, a common approach is to compare neighbouring samples and skip occlusion where they disagree sharply, accepting a virtual object that overlaps a little rather than one whose outline boils.

What the demo above is doing

The floor, the pillar and the low wall are the real world. They are rendered once per frame into a small offscreen buffer that stores nothing but distance from the camera — the same information a real depth sensor provides, produced here by geometry the demo happens to know about. The panel on the right shows that buffer directly, nearer surfaces brighter.

The spheres are the virtual content. Their material samples that buffer at the fragment's own screen position, converts the sample to metres, and compares. Behind means discarded. This is the same shader you would write against a real XRWebGLDepthInformation texture, minus the alignment transform — which the demo does not need because its buffer is already aligned to the viewport, and which is exactly the step that trips people up on hardware.

The depth buffer is stored at one byte per pixel over an eight-metre range, so it is quantised to about three centimetres — deliberately, because a real sensor is quantised too. Drag the resolution down to its minimum with hard occlusion selected and the silhouette becomes a staircase. Switch to soft and the same data becomes usable.

Feature availability

Depth sensing is the least portable of the AR features here, and the usage mode you get is not always the one you asked for.

Checked 2026-09. Read session.depthUsage and session.depthDataFormat at runtime; a granted feature can still deliver the mode you listed second.

Mistakes that cost the most time

The first one is responsible for more confused bug reports than the rest combined, because it produces occlusion that is wrong in a way that still looks deliberate.

Skipping normDepthBufferFromNormView

Sampling the depth texture with raw viewport UVs. The result is offset or stretched, and on a device whose depth buffer happens to share the viewport aspect it looks almost correct — until someone rotates the phone.

Hard-coding rawValueToMeters

It varies by device and by format. A hard-coded scale gives occlusion that is consistently too near or too far on every phone except the one it was written on.

Assuming float32

Most devices deliver luminance-alpha. Code that reads the buffer as Float32Array produces meaningless distances rather than throwing, so it fails silently.

Treating a null result as fatal

Depth is missing for whole frames during startup and fast motion. Falling back to no occlusion for that frame is correct; ending the session is not.

Binary occlusion at full resolution

A hard test against a 160-wide buffer aliases badly and shimmers under motion. Fade over a few centimetres instead — it is one line and it is what shipping apps do.

Trusting depth at silhouette edges

A sample that straddles a near and a far surface returns something in between, which belongs to neither. That is where occlusion artefacts concentrate, and where backing off is better than believing the data.

Further reading

The spec is short and the two usage paths are described separately — read the one you intend to use rather than skimming both.

• W3C — WebXR Depth Sensing Module — Usage modes, data formats, and the normative definition of the alignment transform.
• MDN — XRDepthInformation — The shared base interface, and where the CPU and GPU variants diverge.
• MDN — XRCPUDepthInformation.getDepthInMeters() — Coordinate conventions, and the exceptions it throws when they are wrong.
• AR hit testing — The other way of asking about the room, and why it answers with a pose.
• Anchors — Keeping the object you placed on the point you placed it on.
• Environment blend modes — Whether the display can show a real surface in front of your content at all.