Forty-five years ago, my father was studying radio engineering in the Soviet Union. The transistors he studied came in a plastic package he could hold between finger and thumb. A phone processor now contains billions of transistors.
When people outside the chip industry ask me what all those transistors do, I often start with the GPU. It was built to draw pictures. Scientists then borrowed it to do their sums, and it went on to train the systems that recognise images and, later, the ones that generate text. Now the favour is returned: those networks run on the GPU to help it draw better pictures, and to produce video and sound.
Across those changes, three engineering concerns kept returning: doing many calculations at once, keeping data close, and giving programmers room to try something new. Those same concerns shaped the E-Series architecture.
Turn the camera in a game and a different part of the world comes into view. Reflections shift and objects emerge from behind walls. At sixty frames per second, a new image is due every 16.7 milliseconds. Some of the best stories of the last thirty years have been told inside that deadline.
A 1080p image contains just over two million pixels. At sixty frames a second, that means producing more than 124 million output pixels every second. Each may require many calculations to determine what is visible, how it is lit and what colour it should be.
Much of that world is described as triangles. The GPU works out where they appear on the screen and calculates how their surfaces look. Many of these calculations can run in parallel across different triangles and pixels, making graphics a good fit for a processor built to handle many calculations at once.
One recurring operation is simple: multiply two numbers, add the result to a running total, then repeat. This is multiply-accumulate: it helps position objects and calculate lighting. Applied across large arrays of numbers, the same operation is also a building block of neural networks.
In the mid-1990s, these chips were usually called 3D accelerators. Many processed triangles as they arrived, repeatedly reading and updating colour and depth information in memory outside the chip. Moving that data could hold up the drawing.
At VideoLogic, the British company that became Imagination, Martin Ashton and Simon Fenney began the project that became PowerVR. Their approach divided the screen into small tiles, keeping each tile’s working colour and depth data on the chip to reduce traffic to external memory. It also determined which surfaces were visible before shading them, avoiding work on surfaces hidden behind others. If a wall completely hides a chair, calculating the colour of its upholstery adds nothing to the image.
This combination is called tile-based deferred rendering: reduce the data that moves off the chip and avoid calculations that cannot affect the picture. An idea born of thrift, as so many great ideas are.
PowerVR reached a wider audience in Sega's Dreamcast, released in Japan in 1998, and less than a decade later, it was inside the iPhone. That attention to memory traffic had become especially valuable as graphics moved to phones, which have little room to shed heat. A dashboard computer may sit in a hot cabin. A headset rests on your face. In edge devices like these, the power budget is something a person will notice.
Early accelerators offered a largely fixed sequence of operations with settings developers could adjust.
Then programmers were given more freedom. In the early 2000s, programmable shaders let them write more of the calculations themselves. They could invent better ways to show water, skin and light.
Scientists saw another possibility. A machine that could apply similar calculations across large sets of data could also simulate fluids or the forces between particles.
So they disguised their data as pictures. A grid of numbers went in as a texture, and a calculation became a shader applied across a rectangle. Later, CUDA and OpenCL made general-purpose GPU computing more direct. The numbers could stay numbers.
In 2012, a team using the neural-network architecture known as AlexNet won the ImageNet image-recognition competition by a substantial margin. GPUs helped make that result possible.
Training a network involves processing examples, measuring errors and adjusting numerical parameters. Much of the work requires multiplying and combining large arrays of numbers. GPUs could perform many of those calculations in parallel, shortening the time and effort, and making larger experiments practical. The hardware that was developed to draw objects was helping a machine learn to recognise them.
Through all of this, GPUs kept drawing pictures. Then neural rendering brought trained networks into the process.
The greatest example is neural temporal upscaling. A trained network reconstructs a higher-resolution image from a lower-resolution render, using previous frames and information about how the scene is moving. Render at half the width and half the height, and the starting image has a quarter as many pixels. Make the network smart and efficient enough, and you save more rendering time than it costs you to run the model, all while you preserve the image quality in motion.
The network needs data the GPU has just produced, so where it runs matters. A separate AI accelerator may execute it efficiently, but passing work between processors can add memory traffic, data conversion and synchronisation. Teams may also need to integrate separate tools and drivers, coordinate execution and debug across processors. Choosing where to run the network means accounting for those costs as well as the computation itself.
Imagination’s E-Series places specialised matrix hardware inside the GPU’s shading clusters, where it shares register storage with the arithmetic units. Registers hold the values currently being processed. This lets shader programs use matrix acceleration within the graphics pipeline, keeping the calculations close to the data they need.
E-Series also reduces data movement within the arithmetic hardware. Its Burst Processors keep intermediate results close to the operations that use them, reducing repeated reads and writes to register storage.
The architecture remains programmable through interfaces such as Vulkan and OpenCL. Specialised hardware accelerates common operations and developers decide how to combine them. New algorithms must still fit the chip’s compute and memory limits, but the software can keep changing after the silicon is fixed.
Imagination’s Neural Super Resolution brings all these ideas back to the screen, applying E-Series’ matrix acceleration to temporal upscaling within the graphics pipeline. Sharing the GPU still means competing for execution time and memory bandwidth. Turn the camera again, and geometry, lighting and neural reconstruction must deliver one coherent image before the next frame is due. But make it efficient enough, and you enable your hardware to do much more than what it was previously capable of doing with the same power constraints.
This month, we have been showcasing E-Series to customers and media in China. The story behind it has taken an extraordinary turn: hardware built to draw pictures helped make modern AI possible. Now AI is helping it draw, think, and create.
I think back to my father studying radio engineering forty-five years ago. That transistor between finger and thumb was something to understand: how it worked, what it could do, what could be built with it. Those questions still feel familiar. We ask them of processors containing billions of transistors, with software that gives us new ways to put them to work.
Turn the camera in a game. A doorway opens into view. Light catches a moving surface. Geometry, shading and a trained network work together to make the next image. The deadline is still 16.7 milliseconds.
What we can do inside it keeps changing. What we will do next has yet to be imagined.
Talk to us about what you want to run and the performance, power and memory constraints it must meet. Those requirements are the starting point for evaluating E-Series for your design.