360° VR panorama videos have been trending on YouTube since the viewing feature was introduced in 2015. Even without a VR headset they offer a uniquely immersive experience. What I’m referring to are mostly live action movies shot with a growing list of 360° video capable cameras. These panoramas are actually two 360×180 equirectangular 2D images stitched together and projected onto a conceptual 3D sphere.

They are often labeled as 3D or VR but it’s not really accurate. Virtual reality (VR) is currently described as a computer-generated world with some level of interactivity. However, with live action panoramas you are a passive viewer rather than an interactive participant. The ambiguity is compounded as VR headsets are the optimal way of viewing these 360° videos.

Some also refer to 360° videos as 3D since you can look all around, either by spinning the sphere or moving a mobile device. They should not be confused with 3D content shown on theatrical screenings or 3D capable televisions, which are viewed with special glasses. Monoscopic panoramas provide no depth information. Stereoscopic panoramas do provide some depth perception but must be viewed with a VR headset.

However, in both cases panoramas add much entertainment value by giving each viewer the potential for a unique experience. The possible applications range from computer-generated animated feature films to architectural fly-by renderings of yet to be built structures.

So that’s a long preamble to this without the words ray tracing. This post is about creating 360° animated panoramas the easy way using the Unreal Engine and Imagination’s Wizard GR6500 ray tracing hardware. But why is creating these panoramas so much easier with ray tracing?

Creating panoramas with a rasterizer

The Unreal Engine is considered to be a rasterizer. The difference between a rasterizer and a ray tracer is that with the former the geometry dictates which screen pixel is affected, but with the latter, the pixel decides which geometry affects its colour. Due to the limitations of a rasterizer, there are currently two ways to create 360° panorama stills or video sequences with the Unreal Engine:

• Render six sides of a cube into a cubemap by rotating a 90° field of view camera in a central location of the scene. The cubemap image is then converted to an equirectangular spherical 360° panorama with some well-defined projection methods essentially, projecting or shrink wrapping the cube onto a sphere. This method is fast but can result in stretching artefacts at the cube corners. An interesting discussion on this method can be found on the Unreal Engine answerhub.
• Render many sub panels of the full 360° by rotating a camera with a fixed field of view in a central location. You then stitch each panel together to create the 360° panorama. This method is more accurate but can be very time consuming depending on the required final resolution of the panorama. A detailed description of the process using render “slices” can be found on the Unreal Engine blog.

Creating panoramas with a ray tracer

With Imagination’s implementation of ray tracing in the Unreal Engine the panoramas are created in one single render. Using a lens shader, which applies a spherical projection establishing the camera direction for each pixel, we can create a complete 360° flat panorama image. Of course, not only do you get a 360° image or video in one easy step, you also get physically correct reflections, refractions and shadows. This could also be applied in creating stereoscopic panoramas by appropriately shifting the camera direction for each eye.

With Imagination’s implementation of ray tracing in the Unreal Engine the panoramas are created in one single render. Using a lens shader, which applies a spherical projection establishing the camera direction for each pixel, we can create a complete 360° flat panorama image. Of course, not only do you get a 360° image or video in one easy step, but you also get physically correct reflections, refractions and shadows. This could also be applied in creating stereoscopic panoramas by appropriately shifting the camera direction for each eye.

Creating panoramas in Unreal can be difficult at first, but we’ve embedded two in-depth videos to show you how to make a flat panorama in the engine via The Chairs and Table project, a well-recognised tutorial for learning Unreal.

Find out how to create 360° panoramas with our ray tracing technology below..

This movie is the flat panorama before it is wrapped on a sphere:

This movie is the panorama with the player in 360° mode:​

Helping You Make The Most of Unreal Engine Ray Tracing

In early March at GDC 2017 in San Francisco, Imagination demonstrated the implementation of ray tracing in the Unreal Engine running in real-time on our Wizard GR6500 development hardware platform. The demo, based on the Vulkan API, enables you to toggle between several modes, including raster, ray traced and panorama. We have another blog post where you can read a full, in-depth discussion of the making of the demo and the challenges involved.