Make 360 Video for Your Game with the 360 Capture SDK by Facebook

I just had a 360-video-making experience with this “newly”-released 360 Capture SDK by Facebook (by the time I wrote this post)! Here is a video link: Make 360 Video for your Game with 360 Capture SDK.

Basically, this SDK enables you to capture, record, and encode 360 photos and videos with the relevant metadata necessary for detection, where the devs utilized the cube map solution. Videos, screenshots and audio will be saved locally along with the project. Currently the SDK supports Unity, Unreal and native engines. For hardware compatibility issues, please refer to this link.

This tutorial will walk you through the whole process from importing the kit in Unity to publishing your video content on Youtube. Let’s get it started!

There are six main steps to get it to work:

Download the SDK
Open a scene in Unity
Set up encode environment
Set up camera to capture
Record
Publish a 360 video on Youtube

1. Download the SDK

Go to the repo on Github, git clone it, or simply download the ZIP if you don’t have git. When you open the folder, it will look like this:

Then explore a little. In the UnitySampleIntegration Folder, the EncodePackage is crucial to our video making, I will talk about it in more details later.

2. Open a Scene in Unity

Open up 360-Capture-SDK-master -> UnitySampleIntegration -> _Complete-Game or your own scene.

3. Set up Encode Environment

1) Get to know the EncoderObject.prefab

The EncoderObject.prefab has the following components:

The CaptureOption.cs determines whether to do surround capture, i.e. 360-degree capture, the keys to capture and the output video/screenshot sizes. The variable doSurroundCapture will enable SurroundCapture.cs or NonSurroundCapture.cs respectively. The OutputPath is default to [CurrentDirectory]/Gallery folder (replace with your current directory).

For more details, these functions locate in both SurroundCapture.cs and NonSurroundCapture.cs.

The startEncoding function passes the RenderTexture to the encoder for encoding video. You could also set up your desired video encode by modifying videoFPS implemented in SurroundCapture.cs and NonSurroundCapture.cs. The value is default to 30.

The muxingData() function is called after stopEncoding function in order to combine audio and video data together.

The other functions are pretty self-explanatory.

2) Drag the EncoderObject.prefab to the scene

We will use it as spectator camera. The captured information from EncodeObject will be recorded and saved.

You could make the prefab either:

– Attach to your VR headset which will update the view with your head moving, so that the field of view will change with your head.

How: You could make it as child object of MainCamera, or specifically CenterEyeAnchor in your Oculus project.

– Use it as a silent witness sitting at a static position. This we will get a static angle of the view.

How: Drag it in as a root game object.

4. Set up Camera to Capture

1) Render the scene in the Render Texture

Create a Render Texture, I’m naming it as CamRenderTexture. The way it works is that we encode the video with a render texture, everything captured by the camera will write to this render texture.

Then drag it to the Target Texture in the Camera component of the EncodeObject.prefab.

2) Adjust the camera position and angles.

Because we wish to capture the whole 360 scene in the view of a person on the ground, now change the “God view” by dragging the EncodeObject, or the MainCamera it attaches to, closer to the ground and rotating a little bit to gain a better view. Mine’s like this:

3) Let’s add a new script called ReposeCam.cs to do the job:

You might have discovered that what you see from the EncoderObject is not the same as it from your eyes. So you would probably want to always update the capture camera the same position as your eyes.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class ReposeCam : MonoBehaviour {

public Transform eyes;

void Update()
{
   transform.position = eyes.position;
}

}

using System.Collections;

using System.Collections.Generic;

using UnityEngine;

public class ReposeCam : MonoBehaviour {

public Transform eyes;

void Update()

{

transform.position = eyes.position;

}

Attach the script to EncodeObject.

Besides the newly-added script, leave all the other scripts as they are.

5. Record

For Capture Hotkeys, the default keys are listed in the CaptureOption component: F1 to take screenshots, F2 to record video, F3 to stop recording. You could set up other keys as you wish.

Let’s run the scene, press F2 to start, then press F3 to stop when you want to finish. Video is recorded!

You could find it in the Gallery folder in your current directory, which should in parallel with Assets folder.

My result for this step looks like this:

6. Publish a 360 Video on Youtube

Please refer to this link. It will ask you to download a 360 Video Metadata App, which helps get the metadata for making the video in 360.

Open the MP4 file you just captured, check “My video is spherical (360)”, and Inject metadata.

Then you will get a video output with metadata injected.

Publish it on Youtube! Your video will feature with a pan button on the top left.

Enjoy!