A beginner’s guide to AR/VR design 👾


Welcome back. It’s officially Config week and I’m so amped. Each year, Config reminds me why I love this industry so much—and reaffirms my techno-optimism. Beyond the anticipation of new Figma feature announcements, I’m excited to be meeting all of the designers, tool builders, and educators in the community. I want to know: What are you hype to learn at Config 2024? Hit reply and let me know.

P.S. If you see me there, come say hi. But be warned: I’m famously bad at putting names to faces.

—Tommy (@DesignerTom)


The Wireframe:

  • Kick off your XR design journey
  • A goldmine of AR patterns
  • Is XR just another hype cycle?

How to Start Designing for the Future

The next software trend is here. It’s the talk of the tech world, spanning industries from healthcare to manufacturing…and no, I’m not talking about AI.

I’m talking about extended/spatial reality, also known as XR. XR ​​technologies—including virtual reality (VR), augmented reality (AR), and mixed reality (MR)—bring digital elements into our physical environment with tech like Apple’s Vision Pro, Google Glass, and Meta Quest.

With all this momentum around XR, design that meets the tech’s growing needs is coming at a higher and higher premium. But I’ll admit: I’m still new to this myself. So let’s explore the basics of XR design (and how experienced UX designers can tap in) together →

The basics

The first step to understanding XR design? Start thinking in another dimension.

While traditional UX design is confined to the screen, XR design exists in 3D spaces—meaning tons more possibilities for user experiences. To understand these possibilities, it’s helpful to break XR design down into its three key components:

  • Interface design. Instead of thinking in 2D panels and pixels, you’ll be thinking spatially—in meters, centimeters, or millimeters (and yes, you can still do this in Figma).
  • Spatial mapping and 3D visualization. These are the techniques used to anchor virtual objects or info to the user’s physical space (like how Vision Pro presents a detailed digital representation of your surroundings).
  • User input methods. These include gestures, voice commands, or hand controllers that enable users to interact with the digital elements.

The tools

Here are a few XR design tools on my radar—ranging from basic 3D tools to fully-fledged XR dev platforms →

  • For pure 3D modeling and animation, Blender is the go-to for many designers to create detailed 3D assets for XR projects.
  • For 3D modeling with an XR focus, Bezi is a no-code tool that bridges 3D design and XR prototyping, offering both object creation and basic XR presentation capabilities.
  • For XR prototyping and design, ShapesXR is a no-code collaboration platform that emphasizes the presentation layer (but still nails basic 3D manipulation). It’s native to VR/MR headsets and offers Figma and Unity integrations and VR/AR UI libraries.
  • For AR-specific creation, Adobe Aero is a no-code tool that lets you import 2D and 3D assets from Adobe Photoshop and Illustrator with a strong focus on the presentation layer (check out this tutorial to go from Illustrator to Aero in minutes). Similarly, Spark AR is a Meta tool that specializes in creating and presenting AR effects and filters for Meta platforms.
  • For comprehensive XR development, Unity supports both 3D creation and complex XR development (some coding skills required—check out these tutorials to get started on Unity). Unreal Engine is similar, offering high-fidelity graphics and advanced XR capabilities…but with a steeper learning curve.

Putting the tools into action

Most designers have a two-tool workflow. They interface an object in one tool (like Blender or Figma), export it, then put it in a presentation tool to bring a scene into place. There are two ways you can do that with XR design:

1) Develop within an XR framework. Pull the big players’ SDKs and start designing within those frameworks. Use Meta’s Building Blocks, Apple’s ARKit and VisionOS guidelines, or Google’s ARCore to turn your ideas from Unity or Unreal Engine into a working product.

2) Prototype in a friendlier (or fake) environment. Mess around with the tools and make something that merely simulates how your app will behave within whatever form factor you’re designing for, like a headset.

My mantra for XR? Start with something I know, and work my way up to something I don’t know.

For me, that means starting with video—a medium I’m comfortable with already. I’ll record my surrounding area, bring that into a program like After Effects, create some UI there, and then start using new features to engineer an environment.

The bottom line: The best (but definitely not the cheapest) way to really understand XR design? Buy a VR headset and tinker around with it. It might seem intimidating at first, but at the end of the day, spatial interaction design—unlike 2D screens—mimics how the world actually functions. As Axel Rosler, Professor for Interaction Design at University of Washington, explains, “Learning spatial or VR design is really like seeing how the world really is, for the second time around.”


This month, we lost one of the most influential names in Futuristic UIs: Mark Coleran. Mark was the VFX designer for futuristic UIs in movies like Bourne Identity and The Island, inspired a generation of designers to use futuristic UIs as a storytelling device. His influence on “pragmatic futurism” can be seen in many of the XR products coming to market today.

Join me in honoring Mark by checking out his work and thoughtful words.


TOGETHER WITH LEARN UI

Practical, no-nonsense UI design tips

Want to level up your visual design skills?

The Design Hacks newsletter will help you create better design. Features short, original, illustrated UI/UX tips & tactics. No fluff.

Join over 60,000 readers from Apple, Google, Stripe, and more. Check it out.


News, Tools, and Resources: XR Design

  • This library is an absolute goldmine of AR patterns to get you started.
  • If you’re using Figma with ShapesXR, check out this great guide.
  • Looking for some UI inspiration for XR design? I love this collection of resources.
  • I’ve loved following along with XR designers Sterling Crispin and Chris Heinrich.
  • These are the five pillars of AR design that you need to know.
  • Survicate launches Insights Hub, a user feedback analytics and AI assistant designed to help you build better products based on user insights. Effortlessly. Try for free here.*

Got a great tool, podcast episode, idea, or something else? Hit reply and tell me what’s up.

* This is sponsored advertising content.


Ask DesignerTom: The Future of XR

Question: Is it really worth learning XR design, or do you think that XR is just another hype cycle that will die out? – Ali B.

Answer: It's worth it—but with a caveat.

I get the skepticism. Though tech like Apple Vision Pro has introduced incredible innovations in spatial computing, we've seen a parade of overhyped and underwhelming XR wearables (Humane AI Pin and Google Glass, anyone?). The current form factors aren't there yet, and that's a hurdle we need to clear.

But here's the thing: We're on the cusp of something big.

We are about to see two megatrends collide over the next 10 years: generative AI and XR (Nate Jones put it best in this video). This convergence is going to unlock applications that are simply impossible in our current 2D paradigm.

XR isn't just another hype cycle—it's the next computing platform. But it needs time to mature. The clunky headsets and disappointing experiences of today are laying the groundwork for tomorrow's breakthroughs.

So, should you learn XR design? Absolutely. But approach it with patience and a long-term mindset. The designers who start building their XR skills now will be in a great position when that killer hardware finally drops and the XR floodgates open.


The UX Tools Job Board

Hiring for a design role, or know someone who is? Submit it here to hit the inboxes of 75k+ talented designers.


Thanks for reading. Have you already started exploring XR design? If so, what does your toolkit look like? Hit reply and let me know!

See you next week!

Enjoying this newsletter? Let us know here.

UX Tools

Practical lessons, resources, and news for the UX/UI community. Learn the real-world skills, methods, and tools that help you build user-first experiences. We make resources like practical tutorials, the Design Tools Survey, the Design Tools Database, and UX Challenges. Join 60k+ other designers and sign up for the newsletter to get product design mastery in just 5 minutes a week.

Read more from UX Tools

Welcome back. This week's gem: Mobbin's new abtest.design site. It's a goldmine of A/B test results from top apps, and I'm loving it. These resources are crucial for upping our design game. Seeing real-world UX wins and fails is invaluable, and reminds us why the design community rocks—we're always sharing knowledge. By being open about our hits and misses, we all get better. —Tommy (@DesignerTom) The Wireframe: Four (actually useful) design principles How to conduct a heuristic evaluation...

Welcome back. Framer just dropped a game-changer for creators: The new Marketplace Dashboard hands creators the keys to their own digital storefronts. As someone who's been in the trenches of design entrepreneurship, I can't overstate how huge this is. It's not just about organization—it's about empowerment. Whether you're a seasoned template maker or just starting out, this is your moment to turn side hustles into serious businesses. —Tommy (@DesignerTom) The Wireframe: How to build your...

Welcome back. Over the long weekend, I've been thinking about isolation in design—and how it can tank your career in this unpredictable tech landscape. That's why I'm doubling down on community and continuous learning. So here's your post-holiday challenge: As we dive back in, think about how you're pushing your craft forward. Join a Slack group. Hit a virtual meetup. Share what you know. Every bit counts. —Tommy (@DesignerTom) The Wireframe: Five tips for mastering micro interactions Create...