MANTIS Is Leica Geosystems' New Camera-Mounted LiDAR System That Records 3D Scene Data as the Camera Rolls

Hexagon launched MANTIS by Leica Geosystems, a camera-mounted LiDAR system that records depth, camera movement and set geometry frame by frame while a cinema camera rolls. Its MANTIS Studio software turns each take into data for existing VFX pipelines, with the aim of cutting the depth, geometry and camera reconstruction VFX teams do after wrap.

• Ships in Q1 2027. Hexagon says in its launch announcement that MANTIS will begin shipping in the first quarter of 2027.
• Works with any cinema camera. According to Leica, the solid-state, time-of-flight LiDAR rigs to the camera, syncs to it by timecode frame by frame, and records silently and passively.
• Markerless camera tracking. Product manager Pascal Strupler told Film and Digital Times that the LiDAR takes about 40,000 measurements per second and needs no tracking points on set. The same report says the unit stores about 7.5 hours of data internally.
• Four data layers in standard formats. MANTIS Studio outputs point clouds, tracked cameras, LiDAR-accurate depth maps and a position pass in EXR, FBX, Alembic, PLY and E57. Leica says the files work with major VFX programs, including Unreal Engine.
• Hands-on demo sessions in Santa Monica. Leica scheduled four sessions at The Clubhouse on December 3 and 10, each pairing a live demo with time operating MANTIS on a cinema camera.

MANTIS captures the 3D set data a cinema camera leaves behind

"A cinema camera captures the image, but much of the 3D information behind that image is left behind on set," Hexagon says in its announcement. The company notes that separate set scans and camera-tracking systems preserve some of that data, but VFX teams may still need to rebuild missing depth, geometry and camera position after filming.

MANTIS records that information during the take instead. Hexagon says it is designed to reduce estimation work in compositing and manual work in matchmove.

"MANTIS records spatial data frame by frame on set, which enriches the images captured. This significantly reduces the steps required to composite live-action footage with CGI," said Chris Richter, President of Hexagon's Media & Entertainment Venture.

Leica Geosystems' scanners already capture sets and locations for productions, part of its broader work bringing LiDAR to film production. Hexagon CTO Burkhard Boeckem said the instruments Hexagon had available "could capture a full scan of the scene," and that the team asked, "why not include perfectly synced spatial data alongside the image?"

MANTIS Studio hands Houdini, Nuke and Unreal artists depth and tracks aligned to the lens

According to Leica, MANTIS Studio aligns every data layer to the camera and lens:

• Point clouds are timecoded. Every point lines up with the footage, and artists can work from one take, a time slice, or several takes combined.
• Camera solves cover every frame. Tracks export as FBX, Alembic or CSV.
• Depth and position are per frame and per pixel. Measured LiDAR depth arrives for every frame, and each pixel's 3D position comes as an EXR pass.

"For Houdini users, that means building effects, environments and simulations around the real geometry of the set from day one, instead of reconstructing it first," said Judith Crow, VP of Strategic Partnerships at Houdini maker SideFX.

Mark Titchener, Principal Product Manager for VFX and Animation Workflows at Foundry, said delivery in standard formats means Nuke compositors "can pick this up straight away and integrate it into their existing workflows."

Fifteen launch workflows run from matchmove to depth-based grading

Leica lists 15 VFX workflows MANTIS data supports at launch. The MANTIS team gave befores & afters a similar list that also names footage stabilization via pose stream, and told the outlet VFX practitioners might use MANTIS in addition to other set scanning and surveying approaches.

• Camera and tracking. Camera tracking and matchmove, plus point-cloud and solve validation.
• Compositing. Depth-based roto and matte extraction, deep compositing and Z-based holdouts, PostVis compositing, and depth-based relighting.
• 3D integration. Set extension and 3D environment integration, CG integration and ground interaction, and 2D-to-3D stereo conversion.
• Depth-driven effects. Atmospheric and volumetric effects, synthetic depth of field and defocus, and selective color grading by depth.
• Data and documentation. Continuous set documentation, depth-model training data, and 3D witness capture.