Sony XYN Turns Real-World Props and Environments Into 3D Assets for Virtual Production

Sony XYN's spatial capture solution, demonstrated at NAB 2026, provides a complete pipeline for turning physical objects and environments into photorealistic 3D assets. The system combines a guided capture app, cloud-based processing, and an Unreal Engine rendering plugin to produce assets ready for virtual production backgrounds, game development, and spatial content creation.

The workflow addresses a practical need in virtual production: creating high-quality LED wall backgrounds from real locations. Rather than relying on procedurally generated environments or purchasing pre-made assets, productions can scan the specific locations, props, or sets they need and generate assets optimized for LED wall display.

The first step uses Spatial Scan Navi, a phone app that attaches to Sony Alpha mirrorless cameras and uses AR to overlay capture guidance directly on the viewfinder. The app highlights where to take photos, which areas have already been covered, and where gaps remain.

The key design choice is fewer, more strategic photos rather than the brute-force approach of shooting hundreds of overlapping frames. The app uses two capture profiles: one for objects requiring 360-degree coverage, and one for virtual production backgrounds where only the relevant wall or environment needs scanning. By guiding the photographer to capture specific angles and skip redundant shots, the app reduces both capture time and processing load.

The app works with Sony Alpha cameras and leverages their image quality for reconstruction. However, the cloud processing step accepts photos from any source, including phone cameras and non-Sony equipment, though Sony recommends its own cameras for best results.

Captured photos upload to XYN Render, a web-based application that processes images in the cloud and generates 3D assets. The system provides a preview before full generation, allowing users to verify coverage and quality before committing to the processing time.

The output supports multiple formats, with PLY (point cloud) as the primary option. Sony emphasizes high-fidelity output, including HDR-ready files designed for virtual production environments where background quality directly affects the believability of LED wall content.

The generated assets work with Sony's Spatial Renderer Plugin for Unreal Engine, which enables lighting adjustments on the 3D assets within the Unreal environment. Productions can change lighting on specific objects or entire scenes, overlay CG elements on top of scanned backgrounds, and adjust the composite for LED wall output.

Sony has also built integrations with Disguise and Pixotope, allowing the scanned assets to feed directly into those platforms' virtual production workflows. For preview and review, Sony offers its Spatial Reality Display, a glasses-free 3D monitor that shows the reconstructed assets in stereo. The display uses face tracking to present the correct perspective to a single viewer standing in the sweet spot.

Sony highlighted the system's ability to handle challenging subjects, including objects with reflective surfaces that typically cause problems for photogrammetry-based scanning. The guided capture approach and cloud processing pipeline are designed to overcome the reflection and transparency issues that make certain materials difficult to reconstruct.

The software will be offered as a paid subscription, with pricing to be announced closer to the summer 2026 launch. The cloud-based processing model means capture and processing can happen from any location with an internet connection, without requiring local compute resources for reconstruction.

Sony positions the XYN spatial capture solution as part of a broader ecosystem that includes hardware, software, and display components. For virtual production teams already working with Unreal Engine, Disguise, or Pixotope, the integration points reduce the friction of adding scanned assets to existing workflows.