Launch the App and Composite the Scene

This page describes the process of launching the app and rendering the final mixed reality capture scene.

There are two ways that apps may choose to render mixed reality capture scenes. One way is that the app may handle all compositing, green screen clipping, and composition internally. The other is to use a 3rd party application, like OBS Studio or XSplit, to compose the scene.

External Compositing with a 3rd Party App

If the app you're using requires you to composite the app externally, this section of the guide describes the process using OBS Studio as an example. Other programs, such as XSplit, can be used for this process as well.

First, if you have not done so already, run the CameraTool to calibrate the camera or load a configuration file. This process needs to be repeated each time you launch Oculus to use a mixed reality capture compatible app. Information about this process can be found on the Camera Calibration page.

Next, launch the app in mixed reality capture mode. How you launch mixed reality capture mode will vary depending on how the mode has been implemented in the app. Please refer to the app's instructions for information about launching in mixed reality capture mode.

Then, in OBS Studio you can start capturing the game and external camera output.

  1. Create your first source. This will be the background of the scene, so we'll use Window Capture to grab the entire mirrored window. Select the window that contains the scene from the VR app.
  2. If the app also captures a foreground of the scene, the output window may contain 2 images, one foreground and one background. Add a filter [Crop/Pad] to only select the half window and associate that with the foreground and background captures. The foreground will also contain the chroma key that you'll clip from the final scene.
  3. Fill both the background and foreground to the whole compositing window by selecting Transform → Stretch to Screen.
  4. Create another source for your external camera and apply the same transform that you did to the foreground and background images.
  5. Add the chroma key effect filter to remove the green-screen from your external camera and foreground captures.
  6. Make any adjustments to the capture windows or chroma keys required. The order of the layers should be foreground, external camera image, then background.

Troubleshooting Your External Compositing

Sometimes things don’t work how you expect. This section will review some of the common issues when compositing externally.

  1. When I try to overlay the OWD frames in OBS, they aren't showing up properly. Make sure that you're using the same resolution in OBS when cropping as the resolution you set in the Camera Calibration tool. Also, uncheck the Relative box in the crop/pad filter.
  2. My camera lagging behind with my Touch controllers in OBS. The VR app renders content faster than your external camera. Add a Render Delay filter to your foreground and background until the images line up.
  3. OBS isn't picking up my webcam after I set everything up in the mixed reality capture app. Make sure that when you finish the CameraTool calibration, first save the calibration to an XML file, then save the camera to OVRserver, and then close the Camera Tool before opening OBS.

Internal Direct Compositing

The app you're using may composite the scene for you by retrieving the input from the external camera and mixing the images with the virtual scene from the app. In this case you'll need to refer to the in-app prompts about how to set up the scene when you launch the app in mixed reality capture mode.

That's it. You've completed the mixed reality capture setup and are ready to share your experiences.