Shooting in a VR Environment

It's the future!  In addition to creating Visual Reality content, we can now also "traditionally" film people in virtual reality environments. 

What?? Virtual reality contains many innovative game and narrative experiences. The current problem is that not everyone owns a VR headset. They're still at a high price point and not everyone is into it (until they try it!). The best way to demonstrate a VR experience to someone without access to VR, is to traditionally film a person in VR. Below I will tell you how we (myself + Outpost VFX) accomplished it. 

Why??? Filming a composite of a person in a VR experience allows everyone else to witness the experience from a "normal" viewpoint.  Put it this way, if you only watch someone's point of view in VR, it's like watching footage from a go-pro haphazardly falling out of the sky, the player's point of view spins every direction on a static screen, which is disorienting and chaotic.  If you're watching someone in VR in reality they just look like a crazy person spinning around the room making erratic movements. Compositing the person into the environment is the best way to get a sense of what is going on, since from their POV the environment is static and we now can observe it in a normal way with them. 

How?? By mounting a virtual tracker (or an extra VR controller) onto your camera (any camera) it's possible to film someone while simultaneously showing the VR environment they're inside. This is called "mixed reality."  As a technophobe with an aversion to words like "virtual reality, mixed reality, sautéed reality, and upside-down and backwards reality" I have to say that the process (from a DP stand-point) is simple and a ton of fun.  I couldn't help but get giddily excited once I was operating a camera seeing and filming things that didn't exist in reality.  Once your virtual camera tracker and real camera are aligned, a feed goes to the computer, composites the subject (usually in front of a green screen), and sends the composited image back to your viewfinder allowing for the entire VR environment to be filmed. 

Here are some examples of the evolution of the process. Recently we worked with a game called RacketNX to show their game live at some VR conventions. In the future I am curious how this tool can be used in narrative environments or even as a pre-vis tool.  Hopefully this is just the beginning. 

Filming and live compositing at Unity Summit: 


This is another early test with a green screen where we filmed multiple VR experience being played: 


This is the first test we did with a webcam and no green screen:  

How to Make a Lazy LUT

On some projects with limited time or means for color correction, the best option is to bake a look directly into your camera.  A basic REC709 curve can give you a "normal" looking image that is passable, but once in a while it is nice to a have a look tailored to your tastes or to match the look of another camera you are shooting with. There are many great mathematically accurate ways of doing this (which I will not discuss now, but basically involve measuring the RGB values in decimal points and plotting them onto a graph), but I will describe a simple, non-mathematical way of creating a look to use in-camera.

Download the most recent version of DaVinci Resolve here. The non-studio version is free. 

  1. Shoot your test footage on the camera
    Make sure you are using the codec settings you will be using for your shoot.  Do not add any 709 curves, LUTS, matrix settings, or looks of any type. The 'raw' image should look grey and flat. Make sure your exposure setting, f-stop range of the environment, color temperature, and ISO are all similarly set to how you intend to shoot when using your LUT.  Including a grey card and skin tone is helpful. 
  2. Bring your footage into DaVinci
    - Create a new project file
    - Load clips into Media Pool
    - Go to "Edit" in bottom tab
    - Create a new timeline
    - Drag clips onto Timeline
    - Go to "Color" in bottom tab
  3. Create your look
    To alter the image to your liking in the most simple way, click on the thumbnail of the clip. Use the adjustment curves on the bottom right. There are tiny dark grey dots by the luminance curves charts. Click through these dots and make changes to luminance, hue, and saturation charts. Be sure to check your scopes: Go to View: Video Scopes
  4. Export your look as a LUT
    When you are happy with your image and ready to export your look right click on the original thumbnail and go to generate 3D LUT (.cube) 
  5. Save your look to your camera via an SD Card
    Load the .cube file onto an SD card. Put in your cameras correct file structure on the SD card so it can recognize the card. For the FS7 the card file structure is below. 


Insert the card into your camera and open the .cube file. by going to the File menu then "Monitor 3D Lut" and select "Load SD Card."  Save as User 1, 2, 3, or 4 and make sure to select that User LUT and turn on the LUT in your viewfinder and recording settings to view or bake it in. 

Ba-da-bing, ba-da-boom! You got a LUT.