Data and instructions are sent to the GPU for processing. The rendered results are stored in a framebuffer, whose content is scanned by the display controller and sent to the screen.
Drivers without freely (and legally) -available source code are commonly known as binary drivers. Binary drivers used in the context of operating systems that are prone to ongoing development and change (such as Linux) create problems for end users and package maintainers. These problems, which affect system stability, security and performance, are the main reason for the independent development of free and open-source drivers. When no technical documentation is available, an understanding of the underlying hardware is often gained by clean-room reverse engineering. Based on this understanding, device drivers may be written and legally published under any software license.
In rare cases, a manufacturer's driver source code is available on the Internet without a free license. This means that the code can be studied and altered for personal use, but the altered (and usually the original) source code cannot be freely distributed. Solutions to bugs in the driver cannot be easily shared in the form of modified versions of the driver. Therefore, the utility of such drivers is significantly reduced in comparison to free and open-source drivers.
Problems with proprietary drivers
Software developer's view
Illustration of the Linux graphics stack
There are objections to binary-only drivers based on copyright, security, reliability and development concerns. As part of a wider campaign against binary blobs, OpenBSD lead developer Theo de Raadt said that with a binary driver there is "no way to fix it when it breaks (and it will break)"; when a product which relies on binary drivers is declared to be end-of-life by the manufacturer, it is effectively "broken forever."[1] The project has also stated that binary drivers[2] "hide bugs and workarounds for bugs",[3] an observation which has been somewhat vindicated by flaws found in binary drivers (including an exploitablebug in Nvidia's 3D drivers discovered in October 2006 by Rapid7). It is speculated that the bug has existed since 2004; Nvidia has denied this, asserting that the issue was only communicated to them in July 2006 and the 2004 bug was a bug in X.Org (not in Nvidia's driver).[4]
Binary drivers often do not work with current versions of open-source software, and rarely support development snapshots of open-source software; it is usually not directly possible for a developer to use Nvidia's or ATI's proprietary drivers with a development snapshot of an X server or a development snapshot of the Linux kernel. Features like kernel mode-setting cannot be added to binary drivers by anyone but the vendors, which prevents their inclusion if the vendor lacks capacity or interest.
In the Linux kernel development community, Linus Torvalds has made strong statements on the issue of binary-only modules: "I refuse to even consider tying my hands over some binary-only module ... I want people to know that when they use binary-only modules, it's their problem".[5] Another kernel developer, Greg Kroah-Hartman, has said that a binary-only kernel module does not comply with the kernel's license (the GNU General Public License); it "just violates the GPL due to fun things like derivative works and linking and other stuff."[6] Writer and computer scientist Peter Gutmann has expressed concern that the digital rights management scheme in Microsoft's Windows Vista operating system may limit the availability of the documentation required to write open drivers, since it "requires that the operational details of the device be kept confidential."[7]
In the case of binary drivers, there are objections due to free software philosophy, software quality and security concerns.[8] In 2006, Greg Kroah-Hartman concluded that:
"Closed source Linux kernel modules are illegal. That's it, it is very simple. I've had the misfortune of talking to a lot of different IP lawyers over the years about this topic, and every one that I've talked to all agree that there is no way that anyone can create a Linux kernel module, today, that can be closed source. It just violates the GPL due to fun things like derivative works and linking."
[9]
In the future, libGL-fglrx-glx could use the libDRM of the Radeon open-source driver instead of the proprietary binary blob; most of the investment is in the userspace driver.
When applications such as a 3D game engine or a 3D computer graphics software shunt calculations from the CPU to the GPU, they usually use a special-purpose API like OpenGL or Direct3D and do not address the hardware directly. Because all translation (from API calls to GPU opcodes) is done by the device driver, it contains specialized knowledge and is an object of optimization. Due to the history of the rigidity of proprietary driver development there has been a recent surge in the number of community-backed device drivers for desktop and mobile GPUs. Free and Open Hardware organizations like FOSSi, LowRISC, and others, would also benefit from the development of an open graphical hardware standard. This would then provide computer manufacturers, hobbyists, and the like with a complete, royalty-free platform with which to develop computing hardware and related devices.
The desktop computer market was long dominated by PC hardware using the x86/x86-64 instruction set and GPUs available for the PC. With three major competitors (Nvidia, AMD and Intel). The main competing factor was the price of hardware and raw performance in 3D computer games, which is greatly affected by the efficient translation of API calls into GPU opcodes. The display driver and the video decoder are inherent parts of the graphics card: hardware designed to assist in the calculations necessary for the decoding of video streams. As the market for PC hardware has dwindled, it seems unlikely that new competitors will enter this market and it is unclear how much more knowledge one company could gain by seeing the source code of other companies' drivers.
The mobile sector presents a different situation. The functional blocks (the application-specific integrated circuit display driver, 2- and 3D acceleration and video decoding and encoding) are separate semiconductor intellectual property (SIP) blocks on the chip, since hardware devices vary substantially; some portable media players require a display driver that accelerates video decoding, but does not require 3D acceleration. The development goal is not only raw 3D performance, but system integration, power consumption and 2D capabilities. There is also an approach which abandons the traditional method (Vsync) of updating the display and makes better use of sample and hold technology to lower power consumption.
During the second quarter of 2013 79.3 percent of smartphones sold worldwide were running a version of Android,[12] and the Linux kernel dominates smartphones. Hardware developers have an incentive to deliver Linux drivers for their hardware but, due to competition, no incentive to make these drivers free and open-source. Additional problems are the Android-specific augmentations to the Linux kernel which have not been accepted in mainline, such as the Atomic Display Framework (ADF).[13] ADF is a feature of 3.10 AOSP kernels which provides a dma-buf-centric framework between Android's hwcomposer HAL and the kernel driver. ADF significantly overlaps with the DRM-KMS framework. ADF has not been accepted into mainline, but a different set of solutions addressing the same problems (known as atomic mode setting) is under development. Projects such as libhybris harness Android device drivers to run on Linux platforms other than Android.
Software architecture
Mesa (DRI) and Gallium3D have different driver models, but they share free and open-source code.An example matrix of the Gallium3D driver model. With the introduction of the Gallium3D tracker and WinSys interfaces, 18 modules are required instead of 36. Each WinSys module can work with each Gallium3D device driver module and each State Tracker module.
Free and open-source drivers are primarily developed on and for Linux by Linux kernel developers, third-party programming enthusiasts and employees of companies such as Advanced Micro Devices. Each driver has five parts:
A libDRM user-space component (a wrapper library for DRM system calls, which should only be used by Mesa 3D)
A Mesa 3D user-space component. This component is hardware-specific; it is executed on the CPU and translates OpenGL commands, for example, into machine code for the GPU. Because the device driver is split, marshalling is possible. Mesa 3D is the only free and open-source implementation of OpenGL, OpenGL ES, OpenVG, GLX, EGL and OpenCL. In July 2014, most of the components conformed to Gallium3D specifications. A fully functional State Tracker for Direct3D version 9 is written in C, and an unmaintained tracker for Direct3D versions 10 and 11 is written in C++.[14]Wine has Direct3D version 9. Another Wine component translates Direct3D calls into OpenGL calls, working with OpenGL.
The DRM is kernel-specific. A VESA driver is generally available for any operating system. The VESA driver supports most graphics cards without acceleration and at display resolutions limited to a set programmed in the Video BIOS by the manufacturer.[15]
Linux device drivers for AMD hardware in August 2016
AMD's proprietary driver, AMD Catalyst for their Radeon, is available for Microsoft Windows and Linux (formerly fglrx). A current version can be downloaded from AMD's site, and some Linux distributions contain it in their repositories. It is in the process of being replaced with an AMDGPU-PRO hybrid driver combining the open-source kernel, X and Mesa multimedia drivers with closed-source OpenGL, OpenCL and Vulkan drivers derived from Catalyst.
The FOSS drivers for ATI-AMD GPUs are being developed under the name Radeon (xf86-video-ati or xserver-xorg-video-radeon). They still must load proprietary microcode into the GPU to enable hardware acceleration.[16][failed verification]
Radeon 3D code is split into six drivers, according to GPU technology: the radeon, r200 and r300 classic drivers and r300g, r600g and radeonsi Gallium3D drivers:
In addition to providing the necessary documentation, AMD employees contribute code to support their hardware and features.[18]
All components of the Radeon graphics device driver are developed by core contributors and interested parties worldwide. In 2011, the r300g outperformed Catalyst in some cases.
At the 2014 Game Developers Conference, AMD announced that they were exploring a strategy change to re-base the user-space part of Catalyst on a free and open-source DRM kernel module instead of their proprietary kernel blob.[27]
The release of the new AMDGPU kernel module and stack was announced on the dri-devel mailing list in April 2015.[28] Although AMDGPU only officially supports GCN 1.2 and later graphics cards,[29] experimental support for GCN 1.0 and 1.1 graphics cards (which are only officially supported by the Radeon driver) may be enabled via a kernel parameter.[30][31] A separate libdrm, libdrm-amdgpu, has been included since libdrm 2.4.63.[32]
The radeonsi 3D code mentioned in the previous Radeon paragraph is also used with amdgpu; the 3D driver has back ends for both radeon and amdgpu.
Screenshot of REnouveau, a program which collects data for most of Nouveau's reverse-engineering work
Nvidia's proprietary driver, Nvidia GeForce driver for GeForce, is available for Windowsx86/x86-64, Linux x86/x86-64/ARM, OS X 10.5 and later, Solaris x86/x86-64 and FreeBSD x86/x86-64. A current version can be downloaded from the Internet, and some Linux distributions contain it in their repositories. The 4 October 2013 beta Nvidia GeForce driver 331.13 supports the EGL interface, enabling support for Wayland in conjunction with this driver.[33][34]
Nvidia's free and open-source driver is named nv.[35] It is limited (supporting only 2D acceleration), and Matthew Garrett, Dirk Hohndel and others have called its source code confusing.[36][37][38] Nvidia decided to deprecate nv, not adding support for Fermi or later GPUs and DisplayPort, in March 2010.[39]
In December 2009, Nvidia announced they would not support free graphics initiatives.[40] On 23 September 2013, the company announced that they would release some documentation of their GPUs.[41]
Nouveau is based almost entirely on information gained through reverse engineering. This project aims to produce 3D acceleration for X.Org/Wayland using Gallium3D.[42] On March 26, 2012, Nouveau's DRM component was marked stable and promoted from the staging area of the Linux kernel.[43] Nouveau supports Tesla- (and earlier), Fermi-, Kepler- and Maxwell-based GPUs.[44] On 31 January 2014, Nvidia employee Alexandre Courbot committed an extensive patch set which adds initial support for the GK20A (Tegra K1) to Nouveau.[45] In June 2014, Codethink reportedly ran a Wayland-based Westoncompositor with Linux kernel 3.15, using EGL and a "100% open-source graphics driver stack" on a Tegra K1.[46] A feature matrix is available.[47] In July 2014, Nouveau was unable to outperform the Nvidia GeForce driver due to missing re-clocking support. Tegra-re is a project which is working to reverse-engineer nVidia's VLIW-based Tegra series of GPUs that predate Tegra K1.[48]
Nvidia distributes proprietary device drivers for Tegra through OEMs and as part of its Linux for Tegra (formerly L4T) development kit.[49] Nvidia and a partner, Avionic Design, were working on submitting Grate (free and open-source drivers for Tegra) upstream of the mainline Linux kernel in April 2012.[50][51]
The company's co-founder and CEO laid out the Tegra processor roadmap with Ubuntu Unity at the 2013 GPU Technology Conference.[52]
Nvidia's Unified Memory driver (nvidia-uvm.ko), which implements memory management for Pascal and Volta GPUs on Linux, is MIT licensed. The source code is available in the Nvidia Linux driver downloads on systems that support nvidia-uvm.ko.
In May 2022 Nvidia Announced a new initiative and policy to open source its GPULoadable Kernel Modules with dual GPL/MIT license, but only new models at alpha quality. But said "These changes are for the kernel modules, while the user-mode components are untouched. The user-mode remains closed source and is published with prebuilt binaries in the driver and the CUDA toolkit."[53]
Intel has a history of producing (or commissioning) open-source drivers for its graphics chips, with the exception of their PowerVR-based chips.[54] Their 2D X.Org driver is called xf86-video-intel. The kernel mode-setting driver in the Linux kernel does not use the video BIOS for switching video modes; since some BIOSes have a limited range of modes, this provides more reliable access to those supported by Intel video adapters.
The company worked on optimizing their free Linux drivers for performance approaching their Windows counterparts, especially on Sandy Bridge and newer hardware where performance optimizations have allowed the Intel driver to outperform their proprietary Windows drivers in certain tasks, in 2011.[55][56][57] Some of the performance enhancements may also benefit users of older hardware.[58]
Support for Intel's LLC (Last Level Cache, L4-Cache, Crystalwell and Iris Pro) was added in Linux kernel 3.12.[59][60] In 2013, the company had 20 to 30 full-time Linux graphics developers.[61]
Matrox
Matrox develops and manufactures the Matrox Mystique, Parhelia, G200, G400 and G550. Although the company provides free and open-source drivers for their chipsets which are older than the G550; chipsets newer than the G550 are supported by a closed-source driver.
Arm Ltd is a fabless semiconductor company which licenses semiconductor intellectual property cores. Although they are known for the licensing the ARM instruction set and CPUs based on it, they also develop and license the Mali series of GPUs, and more recently Imortalis GPUs that support ray-tracing. On January 21, 2012, Phoronix reported that Luc Verhaegen was driving a reverse-engineering attempt aimed at the Arm Mali series of GPUs (specifically, the Mali-200 and Mali-400 versions). The reverse-engineering project, known as Lima, was presented at FOSDEM on February 4, 2012.[63][64] On February 2, 2013, Verhaegen demonstrated Quake III Arena in timedemo mode, running on top of the Lima driver.[65] In May 2018, a Lima developer posted the driver for inclusion in the Linux kernel.[66] As of May 2019, the Lima driver is part of the mainline Linux kernel.[67]
Panfrost is a reverse-engineered driver effort for Mali Txxx (Midgard) and Gxx (Bifrost) GPUs. Introducing Panfrost talk was presented at X.Org Developer's Conference 2018. As of May 2019, the Panfrost driver is part of the mainline Linux kernel.[68]
ARM has indicated no intention of providing support for their graphics acceleration hardware licensed under a free and open-source license. However, ARM employees sent patches for the Linux kernel to support their ARM HDLCD display controller and Mali DP500, DP550 and DP650 SIP blocks in December 2015 and April 2016.[69][70]
Vivante Corporation is a fabless semiconductor company which licenses semiconductor intellectual property cores and develops the GCxxxx series of GPUs. A Vivante proprietary, closed-source Linux driver consists of kernel- and user-space parts. Although the kernel component is open-source (GPL), the user-space components—consisting of the GLES(2) implementations and a HAL library—are not; these contain the bulk of the driver logic.
Wladimir J. van der Laan found and documented the state bits, command stream and shader ISA by studying how the blobs work, examining and manipulating command-stream dumps. The Etnaviv Gallium3D driver is being written based on this documentation. Van der Laan's work was inspired by the Lima driver, and the project has produced a functional-but-unoptimized Gallium3D LLVM driver. The Etnaviv driver has performed better than Vivante's proprietary code in some benchmarks, and it supports Vivante's GC400, GC800, GC1000, GC2000, GC3000 and GC7000 series.[73] In January 2017, Etnaviv was added to Mesa with both OpenGL ES 2.0 and Desktop OpenGL 2.1 support.[74]
Qualcomm develops the Adreno (formerly ATI Imageon) mobile GPU series, and includes it as part of their Snapdragon mobile SoC series. Phoronix and Slashdot reported in 2012 that Rob Clark, inspired by the Lima driver, was working on reverse-engineering drivers for the Adreno GPU series.[75][76] In a referenced blog post, Clark wrote that he was doing the project in his spare time and that the Qualcomm platform was his only viable target for working on open 3D graphics. His employers (Texas Instruments and Linaro) were affiliated with the ImaginationPowerVR and ARM Mali cores, which would have been his primary targets; he had working command streams for 2D support, and 3D commands seemed to have the same characteristics.[77] The driver code was published on Gitorious "freedreno",[78]
and has been moved to Mesa.[79][80] In 2012, a working shader assembler was completed;[81]
demonstration versions were developed for texture mapping[82] and phong shading,[83] using the reverse-engineered shader compiler.
Clark demonstrated Freedreno running desktop compositing, the XBMC media player and Quake III Arena at FOSDEM on February 2, 2013.[84]
In August 2013, the kernel component of freedreno (MSM driver) was accepted into mainline and is available in Linux kernel 3.12 and later.[85] The DDX driver gained support for server-managed file descriptors requiring X.Org Server version 1.16 and above in July 2014.[86] In January 2016, the Mesa Gallium3D-style driver gained support for Adreno 430;[87] in November of that year, the driver added support for the Adreno 500 series.[88] Freedreno can be used on devices such as 96Boards Dragonboard 410c and Nexus 7 (2013) in traditional Linux distributions (like Debian and Fedora) and on Android.
The Mesa driver for VideoCore4, VC4, was written from scratch by Broadcom's Eric Anholt.[89]
Broadcom develops and designs the VideoCore GPU series as part of their SoCs. Since it is used by the Raspberry Pi, there has been considerable interest in a FOSS driver for VideoCore.[90] The Raspberry Pi Foundation, in co-operation with Broadcom, announced on October 24, 2012, that they open-sourced "all the ARM (CPU) code that drives the GPU".[citation needed] However, the announcement was misleading; according to the author of the reverse-engineered Lima driver, the newly open-sourced components only allowed message-passing between the ARM CPU and VideoCore but offered little insight into Videocore and little additional programability.[91] The Videocore GPU runs an RTOS which handles the processing; video acceleration is done with RTOS firmware coded for its proprietary GPU, and the firmware was not open-sourced on that date.[92] Since there was neither a toolchain targeting the proprietary GPU nor a documented instruction set, no advantage could be taken if the firmware source code became available. The Videocoreiv project[93] attempted to document the VideoCore GPUs.
On February 28, 2014 (the Raspberry Pi's second anniversary), Broadcom and the Raspberry Pi Foundation announced the release of full documentation for the VideoCore IV graphics core and a complete source release of the graphics stack under a 3-clause BSD license.[94][95] The free-license 3D graphics code was committed to Mesa on 29 August 2014,[96] and first appeared on Mesa's 10.3 release.
Other vendors
Although Silicon Integrated Systems and VIA Technologies have expressed limited interest in open-source drivers, both have released source code which has been integrated into X.Org by FOSS developers.[38] In July 2008, VIA opened documentation of their products to improve its image in the Linux and open-source communities.[97] The company has failed to work with the open-source community to provide documentation and a working DRM driver, leaving expectations of Linux support unfulfilled.[98] On January 6, 2011, it was announced that VIA was no longer interested in supporting free graphics initiatives.[99]
DisplayLink announced an open-source project, Libdlo,[100] with the goal of bringing support for their USB graphics technology to Linux and other platforms. Its code is available under the LGPL license,[101] but it has not been integrated into an X.Org driver. DisplayLink graphics support is available through the kernel udlfb driver (with fbdev) in mainline and udl/drm driver, which in March 2012 was only available in the drm-next tree.
Non-hardware-related vendors may also assist free graphics initiatives. Red Hat has two full-time employees (David Airlie and Jérôme Glisse) working on Radeon software,[102] and the Fedora Project sponsors a Fedora Graphics Test Week event before the launch of their new Linux distribution versions to test free graphics drivers.[103] Other companies which have provided development or support include Novell and VMware.
Project VGA aims to create a low-budget, open-source VGA-compatible video card.[104]
The Open Graphics Project aims to create an open-hardware GPU. The Open Graphics Device v1 has dual DVI-I outputs and a 100-pin IDC connector. In September 2010, the first 25 OGD1 boards were made available for grant application and purchase.[105] The Milkymistsystem on a chip, targeted at embedded graphics instead of desktop computers, supports a VGA output, a limited vertex shader and a 2D texturing unit.[106]
The Nyuzi,[107] an experimental GPGPU processor, includes a synthesizable hardware design written in System Verilog, an instruction set emulator, an LLVM-based C-C++ compiler, software libraries and tests and explores parallel software and hardware. It can run on a Terasic DE2-115 field-programmable gate array board.[108][109]
If a project uses FPGAs, it generally has a partially (or completely) closed-source toolchain. There are currently a couple of open-source toolchains available, however, for Lattice-based FPGAs (notably for iCE40 and ECP5 boards) which utilize Project IceStorm,[110] and Trellis,[111] respectively. There is also a larger, ongoing effort to create the "GCC of FPGAs" called SymbiFlow[112] which includes the aforementioned FPGA toolchains as well as an early-stage open-source toolchain for Xilinx-based FPGAs.
^"Patch by Dirk Hohndel". 1998-11-18. Archived from the original on 2014-02-01. Retrieved 2014-07-15. ... opposed to such obfuscated code. We do not regard this as free software according to our standards
