What is VR Video? A Layperson's Introduction

Here at 360 Designs we are making a pretty insane VR camera. We’ve learned a thing or two along the way, and we wanted to share what we’ve learned about VR Video (aka Cinematic Video, aka 360 Video, aka Immersive Video), to help those getting started in this fascinating, fast-growing industry.

Introduction to VR Video

VR Video represents one half of the equation for Virtual Reality content. The other half is CG VR (computer generated virtual reality). This represents mainly games, utility, and augmented reality content, but can also crossover into VR video, with CG hybrid films & experiences.

VR Video can be more accurately described as 360 stereoscopic video. A camera captures a scene in all directions, in 3D, and usually 360 audio too.

From a viewer’s perspective, when wearing a VR headset, wherever you look or turn, you see and hear the scene all around you. It is like a spherical screen has been wrapped around your head, and you can’t see the edges. Everywhere you look, turn your head, rotate your head – it’s all there. It’s like you’re inside the movie, or at the World Cup Final.

VR Video is to video, what color was to black and white. With VR Video, the field of view (screen) has expanded from a rectangle covering 60 degrees, to a sphere covering 360 degrees.

It represents a very different medium of expression from a storytelling / director’s perspective, and leads to many questions…

First questions about VR Video

How do you keep or direct a viewer’s attention when they can look in any direction?
Where do you put the viewer inside the scene?
How do you move the viewer around?
How do you hide the tripod?
Where do you hide the crew?!

Fortunately from a movie making (and crewing) perspective, not a great deal changes with VR filmmaking. Some new technology in the camera and audio department, and some new software in post production, but everything else is relatively familiar story-telling stuff.

Distribution is different, but that’s another story. The quick version, and the good news for publishers, innovators and producers, is that distribution is direct to people’s VR headsets via smartphone apps, and it’s easy to charge for VR content via the existing app store payment systems.

The VR apps market is expected to grow exponentially over the next 5 years. VR Video can be 30-50% of that, if producers cater effectively to it.

That means getting on this stuff now! So let’s delve into some fun, but accessible details…

First, let’s answer those questions above…

How do you keep, or direct, a viewer’s attention in VR video, when they can look in any direction?

Techniques that we’ve seen used effectively include having an actor point at something, having actors all look in the same direction suddenly, and lighting effects. 360 sound can be used effectively here too, people will naturally turn towards an unusual or unexpected noise, or someone speaking.

It’s a good idea to place all the action in one or two physical locations in the scene, and think carefully about making your viewer look back and forth too much. It’s literally a pain in the neck.

You can frame many scenes naturally, like a fan at a theater or sports arena for example, and position the viewer as if they are in the crowd. This way they are not going to look around too much. There are lots of examples like this, where the action can be placed in a natural setting, that causes the viewer to naturally look where you want them to look.

Where do you put the viewer in the scene?

VR video opens up a whole new dimension to filmmaking. In a way, you could say it’s what a director and cinematographer have always done… but now, instead of imagining things through a rectangular box, you must imagine them all around.

Where would you put a viewer to give them the most fascinating all round perspective in a scene?

How do you change shots, move viewers around?

This is one of the most interesting challenges with VR video, and there are limitations on what you can do, or risk (literally) making people unwell. We can save you a lot of time and effort by sharing the following tips… no shaky video or fast camera movement, zoom is not really an option, smooth linear tracking shots can work, scenes with faster movement can work where there is a (stabilizing) point of reference in the foreground (e.g inside a car or train), and cuts between shots do work!

How do you hide the tripod for VR video?

Typically a tripod or other support structure is removed in post, but you can also use old fashioned ‘prop’ methods to hide things too.

How do you hide the crew??

In short, the crew need to be out of sight behind a wall, or moved far enough away, or be disguised among the talent, or covered up in post. You’ll need to give careful thought to this when planning scenes for VR video. A wide expansive prairie scene for example, with nothing in any direction for miles, will be a hard place to hide a VR film crew, so you’ll have to get creative. If you have an interview subject or talent, they will often need to work alone. You could put an earphone on them so the Director can talk to them.

Business Case for VR Video

From a business perspective, the only real change for moving images in the last 80 years, since the introduction of color, has been an increase in resolution. 3D never really caught on, and there’s a good and simple reason… 3D was too complicated for people to set up in their homes, and you still felt a bit goofy wearing the glasses in your living room, assuming of course you could find them among all the kid’s video games. 3D wasn’t a compelling enough proposition over existing 2D content to make you go to the trouble of making it work. Walk into a movie theater and have someone hand you the glasses as you enter, with a professional projectionist taking care of the technical details… a different story.

VR video on the other hand is a new and very different proposition from what’s come before.

For one thing, the TV and Film industry is going to get a large helping hand from the games industry. Games are already driving the lions share of investment into VR, and Google has put 1 million VR headsets into people’s hands, with hundreds of millions more expected to be sold or given away in the next 5 years. The price point of a smartphone based VR headset, by far the most popular type, is on average less than $50 to buy and will trend to almost nothing. These will sell like hotcakes once interest in VR hits mainstream, and large scale games content comes on board, starting 2016.

VR headsets are intrinsically 3D and VR video capable. Watching 3D or VR content is as easy as starting an app. In fact, VR headsets may cause a renaissance for 3D content, in our opinion.

The opportunities for VR Video however are staggering. We go into detail about a few on our VR video applications page but have listed 14 below.

Technical Introduction to VR Video

To capture a scene in all directions means having a camera that can look everywhere at once, basically.

You need a camera that replicates the same viewpoint(s) as a human being – with two eyes, and a head that can turn… anywhere.

Until someone invents a spherical or tetracontakaidigon (42 sides) camera sensor, by far the best way to capture a scene in 360 degrees stereoscopic, is with the use of multiple cameras. These can be regular cameras, or light field plenoptic cameras.

Designing a VR Camera

The preferred implementation is a sphere of cameras, all looking outwards. This replicates the turn of a human’s head.

In our VR camera, we use 42 Blackmagic Cinema Cameras, arranged as 3 rings (axis) of 16 cameras each, with cameras at the axis points doing ‘double duty’. This is a top-end professional rig, designed for the highest possible quality of VR video capture, but the same general principles would apply to almost any VR camera rig.

By overlapping each camera’s field of view sufficiently (using wide lenses) with several other cameras nearby, you can capture enough visual information to stitch the video together, and thus make a spherical video, VR video, 3D video, HDR video, cinematic video, panoramic video, or any kind of video you like. With sufficient overlap between cameras, you can create stereoscopic (in-between) viewpoints, using computational photography approaches, such as those being developed by Google, and avoid the need for using ‘left and right eye’ cameras, although this is still a viable approach to achieve stereoscopic views, and is more accessible when starting out. Computational photography incidentally, is similar software to that used for time slicing (aka Bullet Time), and is also used in the concept of free viewpoint television.

Stitching VR Video

Google have announced a free platform for stitching VR video called Jump Assembler, that will integrate with YouTube, and is designed to work with 16 camera, single-axis VR rigs, that use GoPro’s as the camera head. It may work with other cameras, including our own (we’re very hopeful of this).

Commercial VR video stitching software already exists from companies like Kolor, VideoStitch, Nuke, and Dashwood, and these can all output VR video content, for any application.

The main challenge with capturing so much visual data, at high definition, is having a reliable and robust means to do so. There are also huge data capture and storage requirements. Our fully loaded 42 camera rig using 4k cameras captures up to 10GB per second.

Reliability is the No.1 issue!

If you are using 16, 30, or even 42 cameras in your VR camera rig, you will first want to be very sure those cameras will reliably record your video when you need them. With 42 cameras recording simultaneously in parallel, you are multiplying the risk of any single camera failure… by 42.

Although it is possible to cover up a single camera’s failure in post, particularly with wide lenses and sufficient coverage from other cameras, it is a lot of manual post production work, imperfect in terms of output, and in all likelihood you would be better off to reshoot the entire scene until all cameras work properly throughout, and nothing else (talent?) goes wrong either.

So reliability is the No.1 issue for VR cameras, at this scale of simultaneous multiple camera use. If you’ve ever had an action camera fail on you, even once during recording, you know what we’re talking about.

Data Requirements

Capturing, storing, archiving and working with the output of so many cameras presents a challenge. With our standard VR camera rig, using 2k Blackmagic Micro Cinema Cameras and CinemaDNG raw format, the total data capture rate is 2.7 GB/sec, and a half-hour shoot = 5TB of data. That’s a lot of data to handle, and is why we chose the SD-card / SDI-based Blackmagic Micro cameras, rather than Micro-SD card or HDMI based options, to build our rig around. They also offer lossless and ProRes HQ capture, essential for professional production.

Summary

There’s a lot to consider when planning VR video / immersive video production. Talk to us about how we can help you achieve the very best standards of VR production. If you are interested in live 360 video, check out our recent post – Is Live 360 Video the killer application for VR?

Basket

What is VR Video? A Layperson’s Introduction