In my previous post, I gave an overview of the new IMG CXT GPU and explained why it’s able to offer such incredible performance for both regular rasterisation and ray tracing effects in a mobile power budget. However, even a few years in from ray tracing becoming standard on new desktop graphics cards some still question why ray tracing is important. Therefore, in this post, we’ll take a quick look at some of the benefits it brings for both gamers and developers.

Die Environment Maps, Die.

Developers have been working with rasterisation since the beginning of 3D graphics, so it’s no surprise that they have got very good at using it to “fake” light a scene.

To create shadows in games requires the creation of a “shadow map”. This tests whether a pixel is in view of a light source, and each one of these must be manually placed into the scene by the developer. However, the shadow map must be rendered from the point of view of each light, stored as a texture and then reprojected onto itself. As we can see in the image below from our ray tracing demo, the problem is that shadow maps create hard-edged shadows and lack precision so are prone to aliasing, where a pixel seen by the light does not correspond to a pixel seen by the camera. There are ways around this, such as cascaded shadow maps, but these require a huge amount of geometry and pixel throughput to render the scene at multiple different resolutions.

With ray traced shadows this all goes away – you emit a ray from every point on your visible surface, toward the source light. If the ray reaches the light, then that surface is lit, but if the ray hits anything before it reaches the light, then that ray is discarded, which means that surface is shadowed.

With ray traced shadows you are not limited by resolution so your shadows can be made incredibly subtle, smooth and diffuse. These are known as soft shadows. An object can generate a true penumbra with the result that it feels as if it’s truly located in the world.

You also get true contact shadows – as in shadows that are accurately generated by objects in the scene and that look as if they are connected to them – rather than the approximations you get with shadow maps. Ray tracing also offers dynamically generated photorealistic shadows, whereas before you’d have to precompute them to get any sense of realism.

What’s more, as ray traced shadows work with objects in motion, our figure casts shadows on the floor as he walks. This type of effect makes objects and characters truly look grounded in the scene, rather than looking disconnected, or appearing to float.

Ray tracing also means developers can do away with cube maps, removing common problems such as light bleeding from an object not blocking a light source, and you’ll see things reflected in surfaces, which all helps to tie the scene together.

Space-saving gamers

A side benefit of being able to do away with environment maps will be smaller games. The maps are essentially very large textures so a small but appreciable advantage of a fully ray traced game is that it could be smaller to download and install – which is something that could be of value on a mobile device, where storage and data bandwidth comes at a premium.

Reflection map upgrades

The current state-of-the-art in games for reflections are screen-space reflections (SSR), which provide dynamic reflections in real-time, but, as the name suggests, only in screen space – as in they reflect only what the viewer is looking at that moment. For example, this means that in a first-person shooter game if you have a scene where say, the sky and trees are reflected in water, but you then look down, the sky and trees will no longer be reflected as you can’t see them on screen. It will instead have to fall back to an imprecise reflection map, which reduces realism and immersion.

Ray tracing changes this radically. It enables objects that are not in screen space, which not only increases immersion but can materially impact the way games can be played. You can now see enemy reflection coming at you from behind or to the sides, or you can use reflective surfaces to see around corners. This can be used to provide a competitive advantage or, if the game is well designed, to add to the drama of playing.

Global illumination

A popular technique in games to increase realism is global illumination which simulates the bouncing of light across multiple surfaces to either light or shadow other objects. The benefits of this are a much more nuanced, more accurate look to the world.

This is very demanding on a GPU to do in real-time but naturally, an efficient ray tracer is a prime candidate to do this more efficiently and more accurately. Ray traced global illumination (RT GI) provides the cherry on the cake for realism, as, in addition to ray traced shadows and reflections, we now get objects in the scene having an impact on each other. As you can see in the screenshot from the demo, RT GI provides additional shadowing in areas such as between the large silver cylinders and the pipes causing shadows on the wall. Not only that, but surfaces take on the colour of light bouncing onto those surfaces, giving them a warmer, and more realistic hue, which all adds up to giving the images real depth and immersion.

Timesaving

As well as all the visual enhancements that ray tracing brings to end-users, perhaps the biggest fans should be developers themselves. Assuming they can create a game that only targets a ray tracing GPU, they can save a significant amount of time not having to create the environment maps we described earlier.

In a game lit by traditional rasterisation techniques, artists must manually place fill lights around the scene. To mimic “bounce lighting”, as in lighting areas that would be impacted indirectly by the lights, game artists have to place all the fill lights in the scene – essentially by eye, and then manually adjust the colour and intensity of the lights to make it look real – which is all very time intensive. Furthermore, if for any reason the lights need to change, the whole process has to be done again.

With ray traced global illumination, all they need to do is toggle it on, and the entire room is flooded with light. This not only saves many hours of work, but is more physically accurate, and convincing. And if anything needs changing, it updates dynamically with no additional effort. That’s the power of ray tracing.

User-generated content

In the same vein, imagine modders creating their own new level of a game (while modding on mobile may seem unlikely, it is a thing). The problem is that these homemade levels will have poor lighting – as there will be no pre-baked lighting as created by the original game developers. However, with real-time ray tracing hardware, the lighting of the scene could be done on the fly on the device itself as it loads, meaning that user-created content could effectively look as good a professional game content – at least in terms of lighting. There might a small delay as the lighting was computed, but the increased level of quality will make this worthwhile.

Conclusion

With ray tracing then we see both visual quality and workflow benefits, which, in many ways are tied up together. Developers get major productivity gains and can design games in new ways to take advantage of its possibilities, while end-users get better-looking, and more immersive worlds to spend time in.

This is why we’re excited to bring the IMG CXT to market. Our CXT 48-1536 RT3 core offers 48GTexels/s, equivalent to 1.5 TFLOPS FP32 to superb rasterisation performance, closely coupled with 1.3Gray/s or performance. It can also deliver 6 TOPS of AI performance for good measure – and all within a mobile power budget of around 1-2 watts.

This is why we can be sure that when devices that use this core will be able to deliver a level of visual quality that is at least the equal of what we currently enjoy on desktop PCs and consoles – at between 30 and 60fps depending on the target resolution.

Just as Imagination enabled mobile gaming at the start of the smartphone revolution, with CXT we are once again redefining the possibilities of mobile graphics – and of course, this is only just the beginning.