The most satisfying result from our recent VRDesktopUltra experiments was a short message after another headset check: “Hooray, the shimmer is gone!”
Behind it were several iterations of image transport, a different way of presenting virtual monitors, and measurements that made us reconsider some early decisions. Our test setup now runs three virtual ultrawide Mac screens alongside the main Retina display, together in a single VR scene on Quest Pro.
VRDesktopUltra grew out of WireDeskVR, our earlier virtual workspace project built with Unreal Engine 5. The goal remains the same — comfortable Mac screens in VR — but the new implementation replaces UE5 with our own Rust and OpenXR stack.
Here is where the system stands as of October 21, 2026: the architecture we chose, why we chose it, and the boundary between a working experiment and an everyday tool.
We need a workspace, not just a large video
Displaying a computer image in a headset is only the beginning. When watching a movie, a small loss of detail might go unnoticed. With a code editor, terminal, or pixel artwork in front of you, the requirements are different: thin lines should not shimmer when you move your head, small text should remain legible, and scrolling should feel responsive.
We also want more than one rectangle attached to our gaze. We want a workspace: arrange screens around us, adjust their distance, size, and curvature, and move between them.
That leads to the first important decision: the Mac sends monitor content; Quest builds and presents the VR scene locally. Head tracking, controllers, and screen geometry are handled on the headset. Turning your head does not require waiting for the computer to render and transmit a new image of the entire virtual world. The last received desktop image stays in place while its next update is still in transit.
The core logic is written in Rust. On the Mac, we use ScreenCaptureKit, Metal, and hardware VideoToolbox; on Quest, OpenXR, OpenGL ES, and hardware decoding. Our protocol runs over a USB NCM network connection. The current path is wired, with no Wi-Fi fallback.

