Design7 min

Spatial computing UX: designing for mixed reality and AR

A practical guide to spatial computing UX for mixed reality and AR: placing content in space, eye and hand input, comfort, accessibility and prototyping.

Saurabh YadavFounder, Design Studio by Saurabh
DesignSpatial computing UXAr UX designVisionos design tipsXr interface design
Share

The first time you design for a headset, you realise how much a flat screen was doing for you. A phone has edges. It has a top and a bottom, a thumb zone and a fixed distance from the eye. Take all of that away and suddenly every decision you made on autopilot needs a reason.

That's what spatial computing UX really is: designing interfaces that live in the room with the user instead of behind a sheet of glass. Mixed reality headsets, AR glasses and phone-based AR all fall under it, and the design problems overlap more than the hardware does.

I come from animation, where I learned to guide someone's eye through a frame. Spatial design feels like a cousin of that skill, except the viewer can turn around. Here's how I'd approach it if you're a product designer moving from screens to space.

What spatial computing actually changes

On a screen, the interface is the whole world. In mixed reality, the interface shares space with a kitchen, an office or a factory floor. That shifts a few fundamentals:

  • There's no fixed frame. Content can be placed anywhere, so where you put it becomes a design decision with comfort and safety consequences.
  • Depth is real. Distance, scale and layering carry meaning, and getting them wrong causes eye strain.
  • Input is the body. Eyes, hands and voice replace the mouse and the thumb, and none of them are as precise.
  • The room is part of the UI. Lighting, clutter and other people all affect legibility and attention.

Most of the good guidance out there agrees on the same core ideas: choose the right level of immersion, place content deliberately, keep interactions natural and keep cognitive load low. The hard part is applying them.

Start with the right level of immersion

Not every task needs a fully virtual world. Before designing anything, decide how much of the real world the user should still see.

Immersion level What it looks like Good for Risk
Phone or tablet AR Digital objects over a camera view Try-before-you-buy, quick measuring Arm fatigue, small viewport
Passthrough with windows Flat panels floating in the real room Productivity, media, browsing Clutter if too many panels open
Anchored 3D objects Models placed on real surfaces Training, product demos, design review Objects drifting or misaligned
Fully immersive The room is replaced entirely Focus, simulation, entertainment Disorientation, losing awareness of surroundings

My honest view: start lower than you think. Most productivity tasks work well as a few well placed windows in the user's real space. Full immersion should be something the user chooses, not where they land.

Note

Platforms like Apple's visionOS distinguish between windows, 3D volumes and fully immersive spaces, and let apps share space with others by default. Even if you're designing for another headset, that layered model is a useful way to think.

Placing content in space

This is where screen habits fail you. A few rules I'd follow:

Keep the important things in front

Put primary content roughly in front of the user, at a comfortable distance and slightly below eye level, much like a monitor on a desk. Make people turn their head for secondary things, never for the main task. Constantly looking up or far to the side is tiring.

Respect personal space

Content that appears too close feels invasive and is hard to focus on. Content too far away becomes tiny and hard to target. Let users move and resize windows, and remember where they put them.

Anchor things predictably

Decide whether each element follows the user (head-locked), stays in the room (world-locked) or sits on a surface. Head-locked content is fine for brief notifications but uncomfortable for anything you read for long. World-locked content feels more natural for most panels.

Watch out

Don't glue large panels to the user's head. Text that moves with every small head movement is one of the fastest ways to cause discomfort.

Designing for eyes, hands and voice

On current headsets, the common pattern is to look at something to target it and use a hand gesture, like a pinch, to select it. Some systems also allow direct touch when content is within reach, plus voice.

What that means for your UI:

  • Targets need to be bigger and better spaced than on a phone, because eyes and hands are less precise than a fingertip on glass.
  • Show what's targeted. A subtle highlight when the user looks at a control tells them the system understood. Without it, people second guess every pinch.
  • Avoid accidental activation. Looking at something is not the same as wanting it. Never trigger actions on gaze alone.
  • Limit long arm-raised interactions. Holding your arms up to tap floating buttons gets tiring quickly. Indirect gestures with hands resting in the lap are kinder for long sessions.
  • Offer voice, but don't depend on it. Voice is great for search and dictation, less great in shared spaces or for people with speech differences.

Comfort and accessibility are not optional

Spatial interfaces can make people physically unwell, which is a different level of "bad UX".

  • Keep motion gentle. Avoid moving the user's viewpoint without their input.
  • Don't flash or pulse large areas of the field of view.
  • Make text larger than you'd expect, with strong contrast against varied real-world backgrounds. A translucent panel over a bright window needs extra care.
  • Support seated use and one-handed use.
  • Respect system accessibility settings, such as reduced motion, larger text and alternative input methods.

Many of the same principles from accessibility is UX apply, they just carry more weight in a headset. For motion specifically, my thinking on accessible motion design translates directly.

Tip

Test with people who don't use headsets every day. Experienced users have adapted to discomfort. A first-timer will tell you within five minutes what feels wrong.

How to prototype spatial UX without a big budget

You don't need a full 3D pipeline to start.

  • Sketch the scene from above: where the user stands and where each panel sits.
  • Mock up flat panels in Figma first. Most spatial apps are still mostly 2D panels.
  • Block out 3D objects in a tool like Blender to check scale and placement.
  • Use a game engine or the platform's own tools when you need to test it on a headset.
  • Test in different rooms and lighting. A design that works in a dim studio may fail in a sunny office.
  • Record sessions and note where people hesitate, squint or rub their eyes.

Work with us

Want a team to design it with you? Our UI/UX design agency for startups and SaaS takes products from research to developer handoff.

FAQ

What is spatial computing in UX?

Spatial computing UX is the design of interfaces that exist in three-dimensional space around the user, rather than on a flat screen. It covers mixed reality headsets, AR glasses and phone-based AR. The focus shifts to placement, depth, body-based input and physical comfort.

Is designing for AR different from designing for VR?

Yes. AR and mixed reality keep the real world visible, so your interface competes with real lighting, clutter and people, and must keep users aware of their surroundings. VR replaces the world, which gives you more control but raises the risk of disorientation and motion sickness.

Can I design spatial apps in Figma?

You can design the 2D panels, which is most of the work for many apps. For depth, scale and placement, you'll need a 3D tool or the platform's own prototyping tools, and you'll need to test on a real device before trusting anything.

Key takeaways

  • Spatial computing UX means designing for a room, not a rectangle: placement, depth and comfort become core decisions.
  • Start with the lowest level of immersion that does the job. Let users choose more.
  • Keep primary content in front at a comfortable distance and avoid head-locked panels.
  • Eyes and hands are imprecise, so use bigger targets, clear highlights and never act on gaze alone.
  • Test on real devices, in real rooms, with people who don't use headsets daily.

What to do next

If you're exploring a spatial product, start with one task and design it as a few panels in a real room before reaching for 3D. Motion and depth only help once the basics are clear. If you're planning an AR or mixed reality experience and want a second opinion on the UX, tell me about your project.

Saurabh Yadav's Memoji

Written by

Saurabh Yadav

Founder of Design Studio by Saurabh, a UI/UX design studio in India. We design apps and websites that feel effortless, from research to shipped pixels. Most recently: Pukaar.ai, as seen on Shark Tank India, and the Demerara Bank app.

Keep reading

All articles →