.. | .. |
---|
| 1 | +.. Copyright 2020 DisplayLink (UK) Ltd. |
---|
| 2 | + |
---|
1 | 3 | =================== |
---|
2 | 4 | Userland interfaces |
---|
3 | 5 | =================== |
---|
.. | .. |
---|
85 | 87 | - The userspace side must be fully reviewed and tested to the standards of that |
---|
86 | 88 | userspace project. For e.g. mesa this means piglit testcases and review on the |
---|
87 | 89 | mailing list. This is again to ensure that the new interface actually gets the |
---|
88 | | - job done. |
---|
| 90 | + job done. The userspace-side reviewer should also provide an Acked-by on the |
---|
| 91 | + kernel uAPI patch indicating that they believe the proposed uAPI is sound and |
---|
| 92 | + sufficiently documented and validated for userspace's consumption. |
---|
89 | 93 | |
---|
90 | 94 | - The userspace patches must be against the canonical upstream, not some vendor |
---|
91 | 95 | fork. This is to make sure that no one cheats on the review and testing |
---|
92 | 96 | requirements by doing a quick fork. |
---|
93 | 97 | |
---|
94 | 98 | - The kernel patch can only be merged after all the above requirements are met, |
---|
95 | | - but it **must** be merged **before** the userspace patches land. uAPI always flows |
---|
96 | | - from the kernel, doing things the other way round risks divergence of the uAPI |
---|
97 | | - definitions and header files. |
---|
| 99 | + but it **must** be merged to either drm-next or drm-misc-next **before** the |
---|
| 100 | + userspace patches land. uAPI always flows from the kernel, doing things the |
---|
| 101 | + other way round risks divergence of the uAPI definitions and header files. |
---|
98 | 102 | |
---|
99 | 103 | These are fairly steep requirements, but have grown out from years of shared |
---|
100 | 104 | pain and experience with uAPI added hastily, and almost always regretted about |
---|
.. | .. |
---|
160 | 164 | visible to user-space and accessible beyond open-file boundaries, they |
---|
161 | 165 | cannot support render nodes. |
---|
162 | 166 | |
---|
| 167 | +Device Hot-Unplug |
---|
| 168 | +================= |
---|
| 169 | + |
---|
| 170 | +.. note:: |
---|
| 171 | + The following is the plan. Implementation is not there yet |
---|
| 172 | + (2020 May). |
---|
| 173 | + |
---|
| 174 | +Graphics devices (display and/or render) may be connected via USB (e.g. |
---|
| 175 | +display adapters or docking stations) or Thunderbolt (e.g. eGPU). An end |
---|
| 176 | +user is able to hot-unplug this kind of devices while they are being |
---|
| 177 | +used, and expects that the very least the machine does not crash. Any |
---|
| 178 | +damage from hot-unplugging a DRM device needs to be limited as much as |
---|
| 179 | +possible and userspace must be given the chance to handle it if it wants |
---|
| 180 | +to. Ideally, unplugging a DRM device still lets a desktop continue to |
---|
| 181 | +run, but that is going to need explicit support throughout the whole |
---|
| 182 | +graphics stack: from kernel and userspace drivers, through display |
---|
| 183 | +servers, via window system protocols, and in applications and libraries. |
---|
| 184 | + |
---|
| 185 | +Other scenarios that should lead to the same are: unrecoverable GPU |
---|
| 186 | +crash, PCI device disappearing off the bus, or forced unbind of a driver |
---|
| 187 | +from the physical device. |
---|
| 188 | + |
---|
| 189 | +In other words, from userspace perspective everything needs to keep on |
---|
| 190 | +working more or less, until userspace stops using the disappeared DRM |
---|
| 191 | +device and closes it completely. Userspace will learn of the device |
---|
| 192 | +disappearance from the device removed uevent, ioctls returning ENODEV |
---|
| 193 | +(or driver-specific ioctls returning driver-specific things), or open() |
---|
| 194 | +returning ENXIO. |
---|
| 195 | + |
---|
| 196 | +Only after userspace has closed all relevant DRM device and dmabuf file |
---|
| 197 | +descriptors and removed all mmaps, the DRM driver can tear down its |
---|
| 198 | +instance for the device that no longer exists. If the same physical |
---|
| 199 | +device somehow comes back in the mean time, it shall be a new DRM |
---|
| 200 | +device. |
---|
| 201 | + |
---|
| 202 | +Similar to PIDs, chardev minor numbers are not recycled immediately. A |
---|
| 203 | +new DRM device always picks the next free minor number compared to the |
---|
| 204 | +previous one allocated, and wraps around when minor numbers are |
---|
| 205 | +exhausted. |
---|
| 206 | + |
---|
| 207 | +The goal raises at least the following requirements for the kernel and |
---|
| 208 | +drivers. |
---|
| 209 | + |
---|
| 210 | +Requirements for KMS UAPI |
---|
| 211 | +------------------------- |
---|
| 212 | + |
---|
| 213 | +- KMS connectors must change their status to disconnected. |
---|
| 214 | + |
---|
| 215 | +- Legacy modesets and pageflips, and atomic commits, both real and |
---|
| 216 | + TEST_ONLY, and any other ioctls either fail with ENODEV or fake |
---|
| 217 | + success. |
---|
| 218 | + |
---|
| 219 | +- Pending non-blocking KMS operations deliver the DRM events userspace |
---|
| 220 | + is expecting. This applies also to ioctls that faked success. |
---|
| 221 | + |
---|
| 222 | +- open() on a device node whose underlying device has disappeared will |
---|
| 223 | + fail with ENXIO. |
---|
| 224 | + |
---|
| 225 | +- Attempting to create a DRM lease on a disappeared DRM device will |
---|
| 226 | + fail with ENODEV. Existing DRM leases remain and work as listed |
---|
| 227 | + above. |
---|
| 228 | + |
---|
| 229 | +Requirements for Render and Cross-Device UAPI |
---|
| 230 | +--------------------------------------------- |
---|
| 231 | + |
---|
| 232 | +- All GPU jobs that can no longer run must have their fences |
---|
| 233 | + force-signalled to avoid inflicting hangs on userspace. |
---|
| 234 | + The associated error code is ENODEV. |
---|
| 235 | + |
---|
| 236 | +- Some userspace APIs already define what should happen when the device |
---|
| 237 | + disappears (OpenGL, GL ES: `GL_KHR_robustness`_; `Vulkan`_: |
---|
| 238 | + VK_ERROR_DEVICE_LOST; etc.). DRM drivers are free to implement this |
---|
| 239 | + behaviour the way they see best, e.g. returning failures in |
---|
| 240 | + driver-specific ioctls and handling those in userspace drivers, or |
---|
| 241 | + rely on uevents, and so on. |
---|
| 242 | + |
---|
| 243 | +- dmabuf which point to memory that has disappeared will either fail to |
---|
| 244 | + import with ENODEV or continue to be successfully imported if it would |
---|
| 245 | + have succeeded before the disappearance. See also about memory maps |
---|
| 246 | + below for already imported dmabufs. |
---|
| 247 | + |
---|
| 248 | +- Attempting to import a dmabuf to a disappeared device will either fail |
---|
| 249 | + with ENODEV or succeed if it would have succeeded without the |
---|
| 250 | + disappearance. |
---|
| 251 | + |
---|
| 252 | +- open() on a device node whose underlying device has disappeared will |
---|
| 253 | + fail with ENXIO. |
---|
| 254 | + |
---|
| 255 | +.. _GL_KHR_robustness: https://www.khronos.org/registry/OpenGL/extensions/KHR/KHR_robustness.txt |
---|
| 256 | +.. _Vulkan: https://www.khronos.org/vulkan/ |
---|
| 257 | + |
---|
| 258 | +Requirements for Memory Maps |
---|
| 259 | +---------------------------- |
---|
| 260 | + |
---|
| 261 | +Memory maps have further requirements that apply to both existing maps |
---|
| 262 | +and maps created after the device has disappeared. If the underlying |
---|
| 263 | +memory disappears, the map is created or modified such that reads and |
---|
| 264 | +writes will still complete successfully but the result is undefined. |
---|
| 265 | +This applies to both userspace mmap()'d memory and memory pointed to by |
---|
| 266 | +dmabuf which might be mapped to other devices (cross-device dmabuf |
---|
| 267 | +imports). |
---|
| 268 | + |
---|
| 269 | +Raising SIGBUS is not an option, because userspace cannot realistically |
---|
| 270 | +handle it. Signal handlers are global, which makes them extremely |
---|
| 271 | +difficult to use correctly from libraries like those that Mesa produces. |
---|
| 272 | +Signal handlers are not composable, you can't have different handlers |
---|
| 273 | +for GPU1 and GPU2 from different vendors, and a third handler for |
---|
| 274 | +mmapped regular files. Threads cause additional pain with signal |
---|
| 275 | +handling as well. |
---|
| 276 | + |
---|
163 | 277 | .. _drm_driver_ioctl: |
---|
164 | 278 | |
---|
165 | 279 | IOCTL Support on Device Nodes |
---|
.. | .. |
---|
190 | 304 | |
---|
191 | 305 | Simply running out of kernel/system memory is signalled through ENOMEM. |
---|
192 | 306 | |
---|
193 | | -EPERM/EACCESS: |
---|
| 307 | +EPERM/EACCES: |
---|
194 | 308 | Returned for an operation that is valid, but needs more privileges. |
---|
195 | 309 | E.g. root-only or much more common, DRM master-only operations return |
---|
196 | | - this when when called by unpriviledged clients. There's no clear |
---|
197 | | - difference between EACCESS and EPERM. |
---|
| 310 | + this when called by unpriviledged clients. There's no clear |
---|
| 311 | + difference between EACCES and EPERM. |
---|
198 | 312 | |
---|
199 | 313 | ENODEV: |
---|
| 314 | + The device is not present anymore or is not yet fully initialized. |
---|
| 315 | + |
---|
| 316 | +EOPNOTSUPP: |
---|
200 | 317 | Feature (like PRIME, modesetting, GEM) is not supported by the driver. |
---|
201 | 318 | |
---|
202 | 319 | ENXIO: |
---|
.. | .. |
---|
235 | 352 | Testing and validation |
---|
236 | 353 | ====================== |
---|
237 | 354 | |
---|
| 355 | +Testing Requirements for userspace API |
---|
| 356 | +-------------------------------------- |
---|
| 357 | + |
---|
| 358 | +New cross-driver userspace interface extensions, like new IOCTL, new KMS |
---|
| 359 | +properties, new files in sysfs or anything else that constitutes an API change |
---|
| 360 | +should have driver-agnostic testcases in IGT for that feature, if such a test |
---|
| 361 | +can be reasonably made using IGT for the target hardware. |
---|
| 362 | + |
---|
238 | 363 | Validating changes with IGT |
---|
239 | 364 | --------------------------- |
---|
240 | 365 | |
---|
241 | 366 | There's a collection of tests that aims to cover the whole functionality of |
---|
242 | 367 | DRM drivers and that can be used to check that changes to DRM drivers or the |
---|
243 | 368 | core don't regress existing functionality. This test suite is called IGT and |
---|
244 | | -its code can be found in https://cgit.freedesktop.org/drm/igt-gpu-tools/. |
---|
| 369 | +its code and instructions to build and run can be found in |
---|
| 370 | +https://gitlab.freedesktop.org/drm/igt-gpu-tools/. |
---|
245 | 371 | |
---|
246 | | -To build IGT, start by installing its build dependencies. In Debian-based |
---|
247 | | -systems:: |
---|
| 372 | +Using VKMS to test DRM API |
---|
| 373 | +-------------------------- |
---|
248 | 374 | |
---|
249 | | - # apt-get build-dep intel-gpu-tools |
---|
| 375 | +VKMS is a software-only model of a KMS driver that is useful for testing |
---|
| 376 | +and for running compositors. VKMS aims to enable a virtual display without |
---|
| 377 | +the need for a hardware display capability. These characteristics made VKMS |
---|
| 378 | +a perfect tool for validating the DRM core behavior and also support the |
---|
| 379 | +compositor developer. VKMS makes it possible to test DRM functions in a |
---|
| 380 | +virtual machine without display, simplifying the validation of some of the |
---|
| 381 | +core changes. |
---|
250 | 382 | |
---|
251 | | -And in Fedora-based systems:: |
---|
| 383 | +To Validate changes in DRM API with VKMS, start setting the kernel: make |
---|
| 384 | +sure to enable VKMS module; compile the kernel with the VKMS enabled and |
---|
| 385 | +install it in the target machine. VKMS can be run in a Virtual Machine |
---|
| 386 | +(QEMU, virtme or similar). It's recommended the use of KVM with the minimum |
---|
| 387 | +of 1GB of RAM and four cores. |
---|
252 | 388 | |
---|
253 | | - # dnf builddep intel-gpu-tools |
---|
| 389 | +It's possible to run the IGT-tests in a VM in two ways: |
---|
254 | 390 | |
---|
255 | | -Then clone the repository:: |
---|
| 391 | + 1. Use IGT inside a VM |
---|
| 392 | + 2. Use IGT from the host machine and write the results in a shared directory. |
---|
256 | 393 | |
---|
257 | | - $ git clone git://anongit.freedesktop.org/drm/igt-gpu-tools |
---|
| 394 | +As follow, there is an example of using a VM with a shared directory with |
---|
| 395 | +the host machine to run igt-tests. As an example it's used virtme:: |
---|
258 | 396 | |
---|
259 | | -Configure the build system and start the build:: |
---|
| 397 | + $ virtme-run --rwdir /path/for/shared_dir --kdir=path/for/kernel/directory --mods=auto |
---|
260 | 398 | |
---|
261 | | - $ cd igt-gpu-tools && ./autogen.sh && make -j6 |
---|
| 399 | +Run the igt-tests in the guest machine, as example it's ran the 'kms_flip' |
---|
| 400 | +tests:: |
---|
262 | 401 | |
---|
263 | | -Download the piglit dependency:: |
---|
| 402 | + $ /path/for/igt-gpu-tools/scripts/run-tests.sh -p -s -t "kms_flip.*" -v |
---|
264 | 403 | |
---|
265 | | - $ ./scripts/run-tests.sh -d |
---|
266 | | - |
---|
267 | | -And run the tests:: |
---|
268 | | - |
---|
269 | | - $ ./scripts/run-tests.sh -t kms -t core -s |
---|
270 | | - |
---|
271 | | -run-tests.sh is a wrapper around piglit that will execute the tests matching |
---|
272 | | -the -t options. A report in HTML format will be available in |
---|
273 | | -./results/html/index.html. Results can be compared with piglit. |
---|
| 404 | +In this example, instead of build the igt_runner, Piglit is used |
---|
| 405 | +(-p option); it's created html summary of the tests results and it's saved |
---|
| 406 | +in the folder "igt-gpu-tools/results"; it's executed only the igt-tests |
---|
| 407 | +matching the -t option. |
---|
274 | 408 | |
---|
275 | 409 | Display CRC Support |
---|
276 | 410 | ------------------- |
---|
.. | .. |
---|
316 | 450 | mode setting, since on many devices the vertical blank counter is |
---|
317 | 451 | reset to 0 at some point during modeset. Modern drivers should not |
---|
318 | 452 | call this any more since with kernel mode setting it is a no-op. |
---|
| 453 | + |
---|
| 454 | +Userspace API Structures |
---|
| 455 | +======================== |
---|
| 456 | + |
---|
| 457 | +.. kernel-doc:: include/uapi/drm/drm_mode.h |
---|
| 458 | + :doc: overview |
---|
| 459 | + |
---|
| 460 | +.. kernel-doc:: include/uapi/drm/drm_mode.h |
---|
| 461 | + :internal: |
---|