← all entries
B-TIER demo

Mixed-reality infinite canvas — directing Claude Code agents from Quest 3

rndyrbrts · 2026-08-19T07:48:13.000Z · in Creative
Open post ↗

100k ft — What is this?

Mixed-reality infinite canvas — directing Claude Code agents from Quest 3
demoB-tier

50k ft — Why care?

Early demo of an XR infinite canvas on Meta Quest 3 synced with a desktop canvas: drop reference images into a sphere, put your hand in it and speak a request (e.g. components visualizing multi-agent progress), make a fist to send it to Claude Code agents under custom orchestration, and watch results build up in place.
UI/UXagentscreative codingreference

10k ft — Key takeaways

• Gesture-to-send keeps hands-free XR workflows fluid. • Next frontier: point at objects and describe changes in place. • Spatial containers are a neat metaphor for scoping agent context.

5k ft — Tools & techniques

Claude, Claude Code, Meta Quest 3 AI agents, agent orchestration, spatial prompting, workflow automation

1k ft — Deep-dive / try it

Share and follow! I am having a blast inside my Meta quest three working back-and-forth with Claude code agents. This is a very very early demonstration of a workflow that I have in mind for being able to make components for mixed reality inside a mixed reality with a relatively simple system that I can share with my desktop native infinite canvas. Claude code is setting underneath all of this. Tek and I have some of my own custom orchestration and harnessing running. I love the Meta quest three because it’s super light and it’s super easy to prototype with. Stay locked for more!
Transcript excerpt
Okay, I have my Infinite Canvas in XR now. Same place. It's actually in the Infinite Canvas when I was in XR. So I can grab one of these. This is really janky right now, so just bear with me. But I take these images and I put them inside the sphere. The sphere is a container for what I want to talk about, what I want to reference. Because when I put my hand inside the sphere and I record, I can say, make me some components that represent the progress of a multi-agent system in an XR environment where it's easy to read based on these. So it's going to process that. We'll see that it comes back. I'll make my fist in here to get it to sequence, and now it'll send out. Now I've already got one running, and what you can see is it builds up to a network here. And in here it'll start off like one, and then it'll turn into two, and so forth. And we can see the age of progress. And my very first one was a mixture of these two here, and it turned into this guy, which wound up looking awesome just for an early first concept. So again, it's all janky and kind of crazy, but the fact that this is working. And then the next step is to be able to point to something or grab something and talk about it, and have agents work on that so I can start describing exactly what I want changed and why, and seeing it in its native environment at the same time. Super super

Appears in

More from @rndyrbrts

Collection: Creative