On embedded graphics

I often read about ARM systems, Linux and embedded graphics as if x86 and ARM devices were basically the same. Well they are in some ways of course, but if you are not aware of the differences it is easy to make faulty assumptions. As a "graphics guy" working for ST-Ericsson I thought I would point out some of the distinctions that might not be obvious.

ARM vs. x86

First off: There is no such thing as an ARM CPU that you can go out and buy. While Intel and others make x86 chips that you can put in a socket on you motherboard, ARM sells its CPU designs to companies that embed those designs in their own SoCs. A SoC, or System-on-Chip, embeds not only the CPU cores but also other hardware IPs like graphics IPs, video IPs, ISPs and so on.

Since these different hardware blocks don't sit on a PCI bus like in your PC there is no standard mechanism for hardware discovery that the OS kernel can rely on to do initialization. As a consequence an ARM SoC would normally run a customized kernel configuration that wouldn't run on a different hardware configuration (although things are progressing in this area).

Apple A*, Texas Instruments OMAP, Nvidia Tegra, Samsung Exynos, Qualcomm Snapdragon and ST-Ericsson NovaThor are all examples of ARM based SoC families that either use ARM's CPU designs or their own tweaked designs based on ARM's CPU instruction set. The last two even integrate cellular modems on the same die.

Embedded vs desktop graphics

Discrete GPUs, the kind that you would plug into your desktop PC, are actually complex subsystems with many logical parts, often including 3D graphics cores, 2D blitters, video decoders, display controllers and dedicated memory systems.

In ARM land you won't find GPUs like that. First of all it wouldn't be a separate drop-in board or even chip. Most of the IP blocks that make up a graphics system would sit right next to the CPU cores on the same chip. Furthermore the various graphics blocks would often be designed by different IP vendors.

As an example the ST-Ericsson NovaThor platform that powers the Samsung Galaxy S III mini has a graphics subsystem consisting of a 3D graphics block from ARM (Mali-400) and video, blitter and display controller IPs from ST Microelectronics. The memory is shared with the CPU and the rest of the platform.

The shared memory allows buffers to be passed between all these IP blocks and the CPU without the need to make copies. Not all hardware blocks can access memory in a scatter/gather fashion so a contiguous memory allocator ensures that the graphics buffers are made up from sequential physical memory pages when needed.

A clean and safe solution to passing around these memory handles between the different device drivers and across processes is DMA-BUF which was created by the ARM Linux organization Linaro. It has since found use in x86 drivers also. Before DMA-BUF there were a number of vendor specific memory drivers that did similar things, often combining buffer sharing with the actual memory allocator itself.

What is commonly referred to as the GPU is strictly a memory-to-memory device that renders 3D graphics into memory buffers. It has nothing to do with presenting those buffers on the display! That would be the job of the display controller. For this reason I am bit reluctant to call it a GPU as this part tends to confuse people.

Drivers

Even if most embedded 3D hardware has open source kernel drivers these are usually not accepted into the mainline Linux kernel as they are quite useless without the user space libraries that implement OpenGL ES, EGL etc. Those libraries are under proprietary license and since there is no way of testing the kernel parts without relying on that closed code the kernel drivers are left out of the mainline kernel.

For display controllers the situation is different. They neither require closed source libraries to be useful nor are they as closely guarded as the 3D blocks. When graphics drivers for ARM show up in the mainline kernel they tend to be drivers for display controllers. KMS is an example of a DRM framework for display controllers that has nothing to do with 3D graphics. These drivers are usually written by the SoC vendors themselves whereas 3D drivers are usually licensed from 3rd parties like ARM or Imagination Technologies.

What is collectively known as DRM is actually a set of separate things. A rendered 3D frame could e.g. use ION for allocating a render target, DMA-BUF for sharing that buffer with the 3D HW driver that renders to it and with KMS that posts the result to the display. Looking back at our example platform those drivers would be written by Google, Linaro, ARM and ST-Ericsson respectively.

So when you read about a DRM driver for an ARM SoC you have to look closer to know what that really means. It rarely has anything to do with 3D graphics...