What Is Mixed Reality Video Production?

What does mixed reality actually mean in production?

It means one frame containing two origins. Part of the image was captured by a camera, part was rendered by a computer, and the production work is making that seam disappear well enough that the audience accepts the whole frame as one place.

The core of it is a camera solve. Software analyses the footage and reconstructs where the real camera was, how it moved, and what lens it had. Once that virtual camera exists, digital elements can be placed in the same space and they will hold their position as the shot moves. Everything after that is matching: lighting, shadows, grain, blur, lens character.

How does it differ from VFX, AR and VR?

Term	What it is	Where the audience sees it
Mixed reality video	Filmed plate plus 3D, composited into a finished film	Any screen; no device needed
VFX	The same techniques, usually aiming to be invisible	Film and series
AR	Digital layer over a live camera feed, rendered on the fly	Phone or headset, in real time
VR	A fully digital environment	Headset
Virtual production / LED volume	Digital environment played on a wall behind the talent, captured in camera	Any screen; created on set

The confusion is worth naming, because clients frequently mean "a film with impressive CGI in real space" and reach for whichever term they heard most recently. If nobody needs a device to watch it, it is mixed reality video, not AR.

How is a mixed reality shot actually built?

1. Decide what stays real. The most important decision, and it happens before the shoot. If everything becomes negotiable in post, the plate is just an expensive background.
2. Plan the shoot for tracking. Lens data, deliberate camera movement, texture in frame, clean plates, lighting reference.
3. Solve the camera. Reconstruct the real camera in 3D.
4. Build and light the digital elements against the plate's actual lighting, not against a neutral studio setup.
5. Animate and simulate inside the solved space so digital motion respects the real perspective.
6. Composite. Add grain, motion blur, depth of field, lens distortion, contact shadows and bounce light.
7. Grade the whole frame together, so the real and digital halves share one colour treatment.

On the Samsung Galaxy Z Flip 7 campaign, steps three to six were the entire job: precise camera tracking, realistic lighting and clean compositing so that the product animation sits inside physical space without breaking immersion.

Why do mixed reality shots fail?

Four reasons, in rough order of frequency:

• The plate cannot be solved. Handheld footage of a plain wall gives the tracker nothing to work with.
• The lighting does not match. Wrong colour temperature or wrong direction is caught instantly, even by viewers who cannot say why.
• Nothing touches anything. A digital object with no contact shadow floats regardless of how well it is rendered.
• The comp is too clean. Real footage has grain, aberration and imperfect focus. A render without them stands out precisely because it is better.

When should a brand use mixed reality instead of full CGI?

When the real environment carries meaning, when a real location or real texture is part of the message, or when building the environment in 3D would cost more than filming it. It also lowers the realism burden: a real room makes a digital object look more plausible than a rendered room would.

Full CGI is the better call when the environment itself is impossible, when no shoot is possible in the timeline, or when many variants are needed. The broader trade-off is laid out in CGI vs live action product commercials.

Does mixed reality work for live shows?

Yes, and it is a different discipline. Show content is built for a specific screen geometry and resolution, has to loop cleanly, and has to read from a distance under stage lighting. My ongoing mixed reality experiments include stylised DJ and music visuals built with exactly those constraints in mind.

What does it cost and how long does it take?

A campaign with a shoot, multiple shots and reformats runs five to eight. The cost drivers are the difficulty of the solve, the amount of roto, the complexity of the digital elements, and how many formats are needed, the same logic as any CGI project, described in what drives the cost of a CGI commercial.

What should be captured on the shoot day?

A short list that costs almost nothing on set and saves days afterwards:

Capture	Why it matters
Lens and focal length per shot	The solve needs to know the camera's optics; guessing them costs hours
Chrome and grey ball, or an HDRI	Gives the real lighting so digital elements can be lit to match
Clean plate of each area with digital elements	Lets the compositor remove markers and rebuild background
Tracking markers on flat surfaces	Featureless walls and floors give a tracker nothing to hold on to
A deliberate camera move	Slow and intentional solves; unstable handheld often does not
A measurement or two of the space	Sets the scale so a digital object is the right size

Vertical framing belongs on this list as well. If the piece runs on social, the shot has to be composed for 9:16 before it is shot, not cropped from a horizontal frame afterwards, the effect usually lives exactly in the part of the frame a crop removes.

What is the single best thing a client can do?

Involve the 3D side before the shoot. Ten minutes of lens notes, one HDRI and two clean plates on the shoot day save more post time than any budget increase afterwards. If you have footage already, send the clip and you will get an honest answer about whether it solves, and the production route is described under mixed reality production.