The Mixed Reality Capture Tool allows you to place real-world objects in VR. Using a webcam, green screen and broadcasting software you can capture footage of yourself and superimpose it in the virtual world.
To set up the Mixed Reality Capture Tool:
From your PC, download the Mixed Reality Capture Tool.
Launch the app and click Get Started.
Review the terms and conditions and click Agree.
Click Rift S to begin setup.
If you have an Oculus Quest, follow the setup instructions in this article.
If this is your first time using mixed reality capture, install the Mixed Reality Capture app on your Rift S.
Launch the Mixed Reality Capture app in your Rift S and click Search for Device.
Once the device is found click Calibrate Your Camera.
Select your webcam from the Current Camera menu and then click Begin Calibration.
Follow the onscreen instructions to calibrate your controller and then click Save to Headset.
Locate the game launcher on your computers hard drive. Example location: C:\Program Files\Oculus\Software. Create a shortcut of the launcher.
Right click the shortcut and click Properties.
In the field next to Target add "-mixedreality" to the end of the text in the box.
Launching the game from the shortcut should now open a split screen of your computer.
To use the Mixed Reality Capture Tool with Open Broadcaster Software, or OBS:
Follow the link from the capture tool to install the required OBS version.
Click Generate OBS Scene Collection and launch OBS.
From the top of OBS, click Scene Collection Menu > click Import. Pick the OBS Scene Collection file you saved. Once it's imported, go back to the Scene Collection Menu and select your file from the dropdown.
In the foreground and background layer, right click and click Filters > click Crop/Pad. Make sure you're selecting the background from the left side of the screen and the foreground from the right side of the screen.
If they look incorrectly sized, click on both layers and then click Transform > click Stretch to Screen.
Close out of this screen, right click on the foreground layer, click Filters > click Chroma Key. Make sure the dropdown Key Color Type matches the main background color .
Note: If you're noticing that there is latency between your movements and the ones being captured, you may need to adjust the Render Delay in OBS. To adjust this, right click on your video capture device and click Filters. Click Render Delay and make adjustments until the latency from your movements decreases.
There are two ways to get a 3D asset that meets our Asset Requirements into your Home:
Oculus Home has added a new section to your inventory: My Imports
This is where you will find local 3D assets on your disk that you can place in your Home, including several example assets you can use.
NEKO-ROBO and Slow Living are files on disk that haven't placed in your Home. As soon as you place an object in your Home it will automatically create an inventory object so that visitors to your Home can see it.
Oculus Home will search the following directory for any .glb files:
Documents → Oculus Home→ _Import
Example: C:\Users\myusename\Documents\Oculus Home\_Import
Any files found in this folder will be shown in My Imports
Note: If this folder does not exist, it will be automatically created when you select the My Import section of your Inventory in Oculus Home and we will add a couple example assets.
In Oculus Medium, just like sharing a 3D post to Facebook, you can now Share to Home. This will save your model to Oculus Home's Import Directory and the model will be available to use immediately in your Home.
Note: Medium sculpts are in real world units by default. However, if you scale your model up or down, you will have to take care to make sure this model is the size you expect after Sharing to Home. You can adjust the size and orientation of your shared sculpt by adjusting the Export Transform in the Medium Scene Graph.
Oculus Home supports enabling Customizable Surfaces on your imported 3D Assets. Create/edit your 3D asset by appending Customizable_Surface text in the name of the material to customize.
For example: The following 3D monitor asset's material is named “m_screen_01_Customizable_Surface”
Visual Studio Code: While working with glTF file navigate to materials section and edit the name of the material similar to following:
Blender: While working with 3D assets in blender, name material similar to following:
Maya: While working with 3D assets in Maya, name material similar to following:
Substance Painter: While adjusting materials in Substance painter under texture set list, edit/set the material name similar to as following:
Multiple meshes can share the same customizable surface material. Use this to setup split screen:
16x9 is the default aspect ratio. Optionally, specify aspect ratio of the customizable surfaces in the name of the material, by including aspect ratio after Customizable_Surface
For example: m_screen_01_Customizable_Surface_16X9
A monitor with default 16X9 aspect ratio will look like this:
The same monitor with 4X3 aspect ratio will look like this:
Oculus Home supports glTF 2.0 binary files. To learn more, read the glTF 2.0 specification.
If provided, the following will be read from the extras field in the asset section of the glTF: