| .. | .. |
|---|
| 1 | +.. Copyright 2020 DisplayLink (UK) Ltd. |
|---|
| 2 | + |
|---|
| 1 | 3 | =================== |
|---|
| 2 | 4 | Userland interfaces |
|---|
| 3 | 5 | =================== |
|---|
| .. | .. |
|---|
| 85 | 87 | - The userspace side must be fully reviewed and tested to the standards of that |
|---|
| 86 | 88 | userspace project. For e.g. mesa this means piglit testcases and review on the |
|---|
| 87 | 89 | mailing list. This is again to ensure that the new interface actually gets the |
|---|
| 88 | | - job done. |
|---|
| 90 | + job done. The userspace-side reviewer should also provide an Acked-by on the |
|---|
| 91 | + kernel uAPI patch indicating that they believe the proposed uAPI is sound and |
|---|
| 92 | + sufficiently documented and validated for userspace's consumption. |
|---|
| 89 | 93 | |
|---|
| 90 | 94 | - The userspace patches must be against the canonical upstream, not some vendor |
|---|
| 91 | 95 | fork. This is to make sure that no one cheats on the review and testing |
|---|
| 92 | 96 | requirements by doing a quick fork. |
|---|
| 93 | 97 | |
|---|
| 94 | 98 | - The kernel patch can only be merged after all the above requirements are met, |
|---|
| 95 | | - but it **must** be merged **before** the userspace patches land. uAPI always flows |
|---|
| 96 | | - from the kernel, doing things the other way round risks divergence of the uAPI |
|---|
| 97 | | - definitions and header files. |
|---|
| 99 | + but it **must** be merged to either drm-next or drm-misc-next **before** the |
|---|
| 100 | + userspace patches land. uAPI always flows from the kernel, doing things the |
|---|
| 101 | + other way round risks divergence of the uAPI definitions and header files. |
|---|
| 98 | 102 | |
|---|
| 99 | 103 | These are fairly steep requirements, but have grown out from years of shared |
|---|
| 100 | 104 | pain and experience with uAPI added hastily, and almost always regretted about |
|---|
| .. | .. |
|---|
| 160 | 164 | visible to user-space and accessible beyond open-file boundaries, they |
|---|
| 161 | 165 | cannot support render nodes. |
|---|
| 162 | 166 | |
|---|
| 167 | +Device Hot-Unplug |
|---|
| 168 | +================= |
|---|
| 169 | + |
|---|
| 170 | +.. note:: |
|---|
| 171 | + The following is the plan. Implementation is not there yet |
|---|
| 172 | + (2020 May). |
|---|
| 173 | + |
|---|
| 174 | +Graphics devices (display and/or render) may be connected via USB (e.g. |
|---|
| 175 | +display adapters or docking stations) or Thunderbolt (e.g. eGPU). An end |
|---|
| 176 | +user is able to hot-unplug this kind of devices while they are being |
|---|
| 177 | +used, and expects that the very least the machine does not crash. Any |
|---|
| 178 | +damage from hot-unplugging a DRM device needs to be limited as much as |
|---|
| 179 | +possible and userspace must be given the chance to handle it if it wants |
|---|
| 180 | +to. Ideally, unplugging a DRM device still lets a desktop continue to |
|---|
| 181 | +run, but that is going to need explicit support throughout the whole |
|---|
| 182 | +graphics stack: from kernel and userspace drivers, through display |
|---|
| 183 | +servers, via window system protocols, and in applications and libraries. |
|---|
| 184 | + |
|---|
| 185 | +Other scenarios that should lead to the same are: unrecoverable GPU |
|---|
| 186 | +crash, PCI device disappearing off the bus, or forced unbind of a driver |
|---|
| 187 | +from the physical device. |
|---|
| 188 | + |
|---|
| 189 | +In other words, from userspace perspective everything needs to keep on |
|---|
| 190 | +working more or less, until userspace stops using the disappeared DRM |
|---|
| 191 | +device and closes it completely. Userspace will learn of the device |
|---|
| 192 | +disappearance from the device removed uevent, ioctls returning ENODEV |
|---|
| 193 | +(or driver-specific ioctls returning driver-specific things), or open() |
|---|
| 194 | +returning ENXIO. |
|---|
| 195 | + |
|---|
| 196 | +Only after userspace has closed all relevant DRM device and dmabuf file |
|---|
| 197 | +descriptors and removed all mmaps, the DRM driver can tear down its |
|---|
| 198 | +instance for the device that no longer exists. If the same physical |
|---|
| 199 | +device somehow comes back in the mean time, it shall be a new DRM |
|---|
| 200 | +device. |
|---|
| 201 | + |
|---|
| 202 | +Similar to PIDs, chardev minor numbers are not recycled immediately. A |
|---|
| 203 | +new DRM device always picks the next free minor number compared to the |
|---|
| 204 | +previous one allocated, and wraps around when minor numbers are |
|---|
| 205 | +exhausted. |
|---|
| 206 | + |
|---|
| 207 | +The goal raises at least the following requirements for the kernel and |
|---|
| 208 | +drivers. |
|---|
| 209 | + |
|---|
| 210 | +Requirements for KMS UAPI |
|---|
| 211 | +------------------------- |
|---|
| 212 | + |
|---|
| 213 | +- KMS connectors must change their status to disconnected. |
|---|
| 214 | + |
|---|
| 215 | +- Legacy modesets and pageflips, and atomic commits, both real and |
|---|
| 216 | + TEST_ONLY, and any other ioctls either fail with ENODEV or fake |
|---|
| 217 | + success. |
|---|
| 218 | + |
|---|
| 219 | +- Pending non-blocking KMS operations deliver the DRM events userspace |
|---|
| 220 | + is expecting. This applies also to ioctls that faked success. |
|---|
| 221 | + |
|---|
| 222 | +- open() on a device node whose underlying device has disappeared will |
|---|
| 223 | + fail with ENXIO. |
|---|
| 224 | + |
|---|
| 225 | +- Attempting to create a DRM lease on a disappeared DRM device will |
|---|
| 226 | + fail with ENODEV. Existing DRM leases remain and work as listed |
|---|
| 227 | + above. |
|---|
| 228 | + |
|---|
| 229 | +Requirements for Render and Cross-Device UAPI |
|---|
| 230 | +--------------------------------------------- |
|---|
| 231 | + |
|---|
| 232 | +- All GPU jobs that can no longer run must have their fences |
|---|
| 233 | + force-signalled to avoid inflicting hangs on userspace. |
|---|
| 234 | + The associated error code is ENODEV. |
|---|
| 235 | + |
|---|
| 236 | +- Some userspace APIs already define what should happen when the device |
|---|
| 237 | + disappears (OpenGL, GL ES: `GL_KHR_robustness`_; `Vulkan`_: |
|---|
| 238 | + VK_ERROR_DEVICE_LOST; etc.). DRM drivers are free to implement this |
|---|
| 239 | + behaviour the way they see best, e.g. returning failures in |
|---|
| 240 | + driver-specific ioctls and handling those in userspace drivers, or |
|---|
| 241 | + rely on uevents, and so on. |
|---|
| 242 | + |
|---|
| 243 | +- dmabuf which point to memory that has disappeared will either fail to |
|---|
| 244 | + import with ENODEV or continue to be successfully imported if it would |
|---|
| 245 | + have succeeded before the disappearance. See also about memory maps |
|---|
| 246 | + below for already imported dmabufs. |
|---|
| 247 | + |
|---|
| 248 | +- Attempting to import a dmabuf to a disappeared device will either fail |
|---|
| 249 | + with ENODEV or succeed if it would have succeeded without the |
|---|
| 250 | + disappearance. |
|---|
| 251 | + |
|---|
| 252 | +- open() on a device node whose underlying device has disappeared will |
|---|
| 253 | + fail with ENXIO. |
|---|
| 254 | + |
|---|
| 255 | +.. _GL_KHR_robustness: https://www.khronos.org/registry/OpenGL/extensions/KHR/KHR_robustness.txt |
|---|
| 256 | +.. _Vulkan: https://www.khronos.org/vulkan/ |
|---|
| 257 | + |
|---|
| 258 | +Requirements for Memory Maps |
|---|
| 259 | +---------------------------- |
|---|
| 260 | + |
|---|
| 261 | +Memory maps have further requirements that apply to both existing maps |
|---|
| 262 | +and maps created after the device has disappeared. If the underlying |
|---|
| 263 | +memory disappears, the map is created or modified such that reads and |
|---|
| 264 | +writes will still complete successfully but the result is undefined. |
|---|
| 265 | +This applies to both userspace mmap()'d memory and memory pointed to by |
|---|
| 266 | +dmabuf which might be mapped to other devices (cross-device dmabuf |
|---|
| 267 | +imports). |
|---|
| 268 | + |
|---|
| 269 | +Raising SIGBUS is not an option, because userspace cannot realistically |
|---|
| 270 | +handle it. Signal handlers are global, which makes them extremely |
|---|
| 271 | +difficult to use correctly from libraries like those that Mesa produces. |
|---|
| 272 | +Signal handlers are not composable, you can't have different handlers |
|---|
| 273 | +for GPU1 and GPU2 from different vendors, and a third handler for |
|---|
| 274 | +mmapped regular files. Threads cause additional pain with signal |
|---|
| 275 | +handling as well. |
|---|
| 276 | + |
|---|
| 163 | 277 | .. _drm_driver_ioctl: |
|---|
| 164 | 278 | |
|---|
| 165 | 279 | IOCTL Support on Device Nodes |
|---|
| .. | .. |
|---|
| 190 | 304 | |
|---|
| 191 | 305 | Simply running out of kernel/system memory is signalled through ENOMEM. |
|---|
| 192 | 306 | |
|---|
| 193 | | -EPERM/EACCESS: |
|---|
| 307 | +EPERM/EACCES: |
|---|
| 194 | 308 | Returned for an operation that is valid, but needs more privileges. |
|---|
| 195 | 309 | E.g. root-only or much more common, DRM master-only operations return |
|---|
| 196 | | - this when when called by unpriviledged clients. There's no clear |
|---|
| 197 | | - difference between EACCESS and EPERM. |
|---|
| 310 | + this when called by unpriviledged clients. There's no clear |
|---|
| 311 | + difference between EACCES and EPERM. |
|---|
| 198 | 312 | |
|---|
| 199 | 313 | ENODEV: |
|---|
| 314 | + The device is not present anymore or is not yet fully initialized. |
|---|
| 315 | + |
|---|
| 316 | +EOPNOTSUPP: |
|---|
| 200 | 317 | Feature (like PRIME, modesetting, GEM) is not supported by the driver. |
|---|
| 201 | 318 | |
|---|
| 202 | 319 | ENXIO: |
|---|
| .. | .. |
|---|
| 235 | 352 | Testing and validation |
|---|
| 236 | 353 | ====================== |
|---|
| 237 | 354 | |
|---|
| 355 | +Testing Requirements for userspace API |
|---|
| 356 | +-------------------------------------- |
|---|
| 357 | + |
|---|
| 358 | +New cross-driver userspace interface extensions, like new IOCTL, new KMS |
|---|
| 359 | +properties, new files in sysfs or anything else that constitutes an API change |
|---|
| 360 | +should have driver-agnostic testcases in IGT for that feature, if such a test |
|---|
| 361 | +can be reasonably made using IGT for the target hardware. |
|---|
| 362 | + |
|---|
| 238 | 363 | Validating changes with IGT |
|---|
| 239 | 364 | --------------------------- |
|---|
| 240 | 365 | |
|---|
| 241 | 366 | There's a collection of tests that aims to cover the whole functionality of |
|---|
| 242 | 367 | DRM drivers and that can be used to check that changes to DRM drivers or the |
|---|
| 243 | 368 | core don't regress existing functionality. This test suite is called IGT and |
|---|
| 244 | | -its code can be found in https://cgit.freedesktop.org/drm/igt-gpu-tools/. |
|---|
| 369 | +its code and instructions to build and run can be found in |
|---|
| 370 | +https://gitlab.freedesktop.org/drm/igt-gpu-tools/. |
|---|
| 245 | 371 | |
|---|
| 246 | | -To build IGT, start by installing its build dependencies. In Debian-based |
|---|
| 247 | | -systems:: |
|---|
| 372 | +Using VKMS to test DRM API |
|---|
| 373 | +-------------------------- |
|---|
| 248 | 374 | |
|---|
| 249 | | - # apt-get build-dep intel-gpu-tools |
|---|
| 375 | +VKMS is a software-only model of a KMS driver that is useful for testing |
|---|
| 376 | +and for running compositors. VKMS aims to enable a virtual display without |
|---|
| 377 | +the need for a hardware display capability. These characteristics made VKMS |
|---|
| 378 | +a perfect tool for validating the DRM core behavior and also support the |
|---|
| 379 | +compositor developer. VKMS makes it possible to test DRM functions in a |
|---|
| 380 | +virtual machine without display, simplifying the validation of some of the |
|---|
| 381 | +core changes. |
|---|
| 250 | 382 | |
|---|
| 251 | | -And in Fedora-based systems:: |
|---|
| 383 | +To Validate changes in DRM API with VKMS, start setting the kernel: make |
|---|
| 384 | +sure to enable VKMS module; compile the kernel with the VKMS enabled and |
|---|
| 385 | +install it in the target machine. VKMS can be run in a Virtual Machine |
|---|
| 386 | +(QEMU, virtme or similar). It's recommended the use of KVM with the minimum |
|---|
| 387 | +of 1GB of RAM and four cores. |
|---|
| 252 | 388 | |
|---|
| 253 | | - # dnf builddep intel-gpu-tools |
|---|
| 389 | +It's possible to run the IGT-tests in a VM in two ways: |
|---|
| 254 | 390 | |
|---|
| 255 | | -Then clone the repository:: |
|---|
| 391 | + 1. Use IGT inside a VM |
|---|
| 392 | + 2. Use IGT from the host machine and write the results in a shared directory. |
|---|
| 256 | 393 | |
|---|
| 257 | | - $ git clone git://anongit.freedesktop.org/drm/igt-gpu-tools |
|---|
| 394 | +As follow, there is an example of using a VM with a shared directory with |
|---|
| 395 | +the host machine to run igt-tests. As an example it's used virtme:: |
|---|
| 258 | 396 | |
|---|
| 259 | | -Configure the build system and start the build:: |
|---|
| 397 | + $ virtme-run --rwdir /path/for/shared_dir --kdir=path/for/kernel/directory --mods=auto |
|---|
| 260 | 398 | |
|---|
| 261 | | - $ cd igt-gpu-tools && ./autogen.sh && make -j6 |
|---|
| 399 | +Run the igt-tests in the guest machine, as example it's ran the 'kms_flip' |
|---|
| 400 | +tests:: |
|---|
| 262 | 401 | |
|---|
| 263 | | -Download the piglit dependency:: |
|---|
| 402 | + $ /path/for/igt-gpu-tools/scripts/run-tests.sh -p -s -t "kms_flip.*" -v |
|---|
| 264 | 403 | |
|---|
| 265 | | - $ ./scripts/run-tests.sh -d |
|---|
| 266 | | - |
|---|
| 267 | | -And run the tests:: |
|---|
| 268 | | - |
|---|
| 269 | | - $ ./scripts/run-tests.sh -t kms -t core -s |
|---|
| 270 | | - |
|---|
| 271 | | -run-tests.sh is a wrapper around piglit that will execute the tests matching |
|---|
| 272 | | -the -t options. A report in HTML format will be available in |
|---|
| 273 | | -./results/html/index.html. Results can be compared with piglit. |
|---|
| 404 | +In this example, instead of build the igt_runner, Piglit is used |
|---|
| 405 | +(-p option); it's created html summary of the tests results and it's saved |
|---|
| 406 | +in the folder "igt-gpu-tools/results"; it's executed only the igt-tests |
|---|
| 407 | +matching the -t option. |
|---|
| 274 | 408 | |
|---|
| 275 | 409 | Display CRC Support |
|---|
| 276 | 410 | ------------------- |
|---|
| .. | .. |
|---|
| 316 | 450 | mode setting, since on many devices the vertical blank counter is |
|---|
| 317 | 451 | reset to 0 at some point during modeset. Modern drivers should not |
|---|
| 318 | 452 | call this any more since with kernel mode setting it is a no-op. |
|---|
| 453 | + |
|---|
| 454 | +Userspace API Structures |
|---|
| 455 | +======================== |
|---|
| 456 | + |
|---|
| 457 | +.. kernel-doc:: include/uapi/drm/drm_mode.h |
|---|
| 458 | + :doc: overview |
|---|
| 459 | + |
|---|
| 460 | +.. kernel-doc:: include/uapi/drm/drm_mode.h |
|---|
| 461 | + :internal: |
|---|