Mixed Reality Capture Setup Guide

Mixed reality capture places real-world objects in VR. In other words, it mixes images from the real world with the virtual one.

Oculus mixed reality capture is designed for developers, marketers, and influencers to share an intimate VR experience with viewers. By placing yourself and others in a VR scene, you can more effectively demonstrate the interaction between the user and virtual world.

(Courtesy of Medium and artist Dominic Qwek - https://www.oculus.com/medium/)

Note: Mixed reality capture is only available for select Rift applications.

How does mixed reality capture work?

Mixed reality capture uses an external camera to capture real-world images, against a green-screen, and combine them with a scene generated by a VR application to create a single blended scene. A user can then interact with the VR scene, and a video can be created that shows the user’s interactions in the real-world overlaid on the VR experience

Before You Begin

Before any calibrations, configuration, or implementation, you need to acquire a PC and camera that meet the performance requirements of mixed reality capture. Review the System Configurations page for a list of PCs that support the additional performance requirements. These are not required hardware to use mixed reality capture.

Similarly, the Room and Sensor Setup page contains information about the room and sensor setup for use with mixed reality capture. These are not required configurations, but are provided as guidelines when setting up your room for mixed reality capture.

Review the guidelines for creating a safe use environment in the Health and Safety warnings located at https://www.oculus.com/legal/health-and-safety-warnings/.

Getting Started

There are two steps to capturing mixed reality capture scenes. Setting up your external camera and a VR Object, and compositing the final scene.

Setup

First, you need to decide if you’ll be using a static or dynamic camera to capture the scene. Static cameras will only give you a single unmovable view in the scene, while a dynamic camera, a camera that you've associated with a VR Object, allows you to move the camera around in the scene.

If you use a dynamic camera, you’ll start by creating the VR Object. This object, a Touch controller, tracks the position of the camera in relation to the VR scene. Information about creating a VR Object is found in the VR Object Setup. You may skip this step if you’re planning to use a static camera.

The next step is to calibrate your external camera to capture the real world images. The calibration process removes image distortion and determines the depth and position of objects captured in the scene. Information about calibrating the camera is found on the Camera Calibration page.

Compositing

Once you’ve finished the setup and configuration, you can launch the mixed reality capture capable app and composite the final scene together. Please review the Launch the App and Composite the Scene page for information about this process.

Non-Oculus equipment/service references are not recommendations, guarantees, or endorsements of products or performance. Your results may vary. These are instead options to vary your own customized experiences. Furthermore, incorrect installation or use of any products or items can damage your computer, space, or otherwise impact performance. If you’re not experienced or qualified to do so yourself, please seek assistance.

  • System Configurations
    Sucuessful mixed reality capture requires a more performant system than is required by regular VR. This page contains sugguestions for mixed reality capture PC performance and system configurations.
  • Room and Sensor Setup
    Mixed reality captures images from an external camera that you'll compose into the virtual app that you're using. This guide will review some room configurations that Oculus used when developing mixed reality; your use-case and implementation may be different.
  • VR Object Setup
    Mixed reality capture uses a VR Object to track the location of a dynamic camera. This page describes the process of setting up a VR Object. If you’re planning to use a static camera that does not move around the scene, you may skip this step.
  • Camera Calibration
    Calibrate your external camera for mixed reality capture using the provided CameraTool.
  • Launch the App and Composite the Scene
    This page describes the process of launching the app and rendering the final mixed reality capture scene.