.. | .. |
---|
1 | 1 | Tainted kernels |
---|
2 | 2 | --------------- |
---|
3 | 3 | |
---|
4 | | -Some oops reports contain the string **'Tainted: '** after the program |
---|
5 | | -counter. This indicates that the kernel has been tainted by some |
---|
6 | | -mechanism. The string is followed by a series of position-sensitive |
---|
7 | | -characters, each representing a particular tainted value. |
---|
| 4 | +The kernel will mark itself as 'tainted' when something occurs that might be |
---|
| 5 | +relevant later when investigating problems. Don't worry too much about this, |
---|
| 6 | +most of the time it's not a problem to run a tainted kernel; the information is |
---|
| 7 | +mainly of interest once someone wants to investigate some problem, as its real |
---|
| 8 | +cause might be the event that got the kernel tainted. That's why bug reports |
---|
| 9 | +from tainted kernels will often be ignored by developers, hence try to reproduce |
---|
| 10 | +problems with an untainted kernel. |
---|
8 | 11 | |
---|
9 | | - 1) ``G`` if all modules loaded have a GPL or compatible license, ``P`` if |
---|
| 12 | +Note the kernel will remain tainted even after you undo what caused the taint |
---|
| 13 | +(i.e. unload a proprietary kernel module), to indicate the kernel remains not |
---|
| 14 | +trustworthy. That's also why the kernel will print the tainted state when it |
---|
| 15 | +notices an internal problem (a 'kernel bug'), a recoverable error |
---|
| 16 | +('kernel oops') or a non-recoverable error ('kernel panic') and writes debug |
---|
| 17 | +information about this to the logs ``dmesg`` outputs. It's also possible to |
---|
| 18 | +check the tainted state at runtime through a file in ``/proc/``. |
---|
| 19 | + |
---|
| 20 | + |
---|
| 21 | +Tainted flag in bugs, oops or panics messages |
---|
| 22 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
| 23 | + |
---|
| 24 | +You find the tainted state near the top in a line starting with 'CPU:'; if or |
---|
| 25 | +why the kernel was tainted is shown after the Process ID ('PID:') and a shortened |
---|
| 26 | +name of the command ('Comm:') that triggered the event:: |
---|
| 27 | + |
---|
| 28 | + BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 |
---|
| 29 | + Oops: 0002 [#1] SMP PTI |
---|
| 30 | + CPU: 0 PID: 4424 Comm: insmod Tainted: P W O 4.20.0-0.rc6.fc30 #1 |
---|
| 31 | + Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011 |
---|
| 32 | + RIP: 0010:my_oops_init+0x13/0x1000 [kpanic] |
---|
| 33 | + [...] |
---|
| 34 | + |
---|
| 35 | +You'll find a 'Not tainted: ' there if the kernel was not tainted at the |
---|
| 36 | +time of the event; if it was, then it will print 'Tainted: ' and characters |
---|
| 37 | +either letters or blanks. In above example it looks like this:: |
---|
| 38 | + |
---|
| 39 | + Tainted: P W O |
---|
| 40 | + |
---|
| 41 | +The meaning of those characters is explained in the table below. In this case |
---|
| 42 | +the kernel got tainted earlier because a proprietary Module (``P``) was loaded, |
---|
| 43 | +a warning occurred (``W``), and an externally-built module was loaded (``O``). |
---|
| 44 | +To decode other letters use the table below. |
---|
| 45 | + |
---|
| 46 | + |
---|
| 47 | +Decoding tainted state at runtime |
---|
| 48 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
| 49 | + |
---|
| 50 | +At runtime, you can query the tainted state by reading |
---|
| 51 | +``cat /proc/sys/kernel/tainted``. If that returns ``0``, the kernel is not |
---|
| 52 | +tainted; any other number indicates the reasons why it is. The easiest way to |
---|
| 53 | +decode that number is the script ``tools/debugging/kernel-chktaint``, which your |
---|
| 54 | +distribution might ship as part of a package called ``linux-tools`` or |
---|
| 55 | +``kernel-tools``; if it doesn't you can download the script from |
---|
| 56 | +`git.kernel.org <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/tools/debugging/kernel-chktaint>`_ |
---|
| 57 | +and execute it with ``sh kernel-chktaint``, which would print something like |
---|
| 58 | +this on the machine that had the statements in the logs that were quoted earlier:: |
---|
| 59 | + |
---|
| 60 | + Kernel is Tainted for following reasons: |
---|
| 61 | + * Proprietary module was loaded (#0) |
---|
| 62 | + * Kernel issued warning (#9) |
---|
| 63 | + * Externally-built ('out-of-tree') module was loaded (#12) |
---|
| 64 | + See Documentation/admin-guide/tainted-kernels.rst in the Linux kernel or |
---|
| 65 | + https://www.kernel.org/doc/html/latest/admin-guide/tainted-kernels.html for |
---|
| 66 | + a more details explanation of the various taint flags. |
---|
| 67 | + Raw taint value as int/string: 4609/'P W O ' |
---|
| 68 | + |
---|
| 69 | +You can try to decode the number yourself. That's easy if there was only one |
---|
| 70 | +reason that got your kernel tainted, as in this case you can find the number |
---|
| 71 | +with the table below. If there were multiple reasons you need to decode the |
---|
| 72 | +number, as it is a bitfield, where each bit indicates the absence or presence of |
---|
| 73 | +a particular type of taint. It's best to leave that to the aforementioned |
---|
| 74 | +script, but if you need something quick you can use this shell command to check |
---|
| 75 | +which bits are set:: |
---|
| 76 | + |
---|
| 77 | + $ for i in $(seq 18); do echo $(($i-1)) $(($(cat /proc/sys/kernel/tainted)>>($i-1)&1));done |
---|
| 78 | + |
---|
| 79 | +Table for decoding tainted state |
---|
| 80 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
| 81 | + |
---|
| 82 | +=== === ====== ======================================================== |
---|
| 83 | +Bit Log Number Reason that got the kernel tainted |
---|
| 84 | +=== === ====== ======================================================== |
---|
| 85 | + 0 G/P 1 proprietary module was loaded |
---|
| 86 | + 1 _/F 2 module was force loaded |
---|
| 87 | + 2 _/S 4 SMP kernel oops on an officially SMP incapable processor |
---|
| 88 | + 3 _/R 8 module was force unloaded |
---|
| 89 | + 4 _/M 16 processor reported a Machine Check Exception (MCE) |
---|
| 90 | + 5 _/B 32 bad page referenced or some unexpected page flags |
---|
| 91 | + 6 _/U 64 taint requested by userspace application |
---|
| 92 | + 7 _/D 128 kernel died recently, i.e. there was an OOPS or BUG |
---|
| 93 | + 8 _/A 256 ACPI table overridden by user |
---|
| 94 | + 9 _/W 512 kernel issued warning |
---|
| 95 | + 10 _/C 1024 staging driver was loaded |
---|
| 96 | + 11 _/I 2048 workaround for bug in platform firmware applied |
---|
| 97 | + 12 _/O 4096 externally-built ("out-of-tree") module was loaded |
---|
| 98 | + 13 _/E 8192 unsigned module was loaded |
---|
| 99 | + 14 _/L 16384 soft lockup occurred |
---|
| 100 | + 15 _/K 32768 kernel has been live patched |
---|
| 101 | + 16 _/X 65536 auxiliary taint, defined for and used by distros |
---|
| 102 | + 17 _/T 131072 kernel was built with the struct randomization plugin |
---|
| 103 | +=== === ====== ======================================================== |
---|
| 104 | + |
---|
| 105 | +Note: The character ``_`` is representing a blank in this table to make reading |
---|
| 106 | +easier. |
---|
| 107 | + |
---|
| 108 | +More detailed explanation for tainting |
---|
| 109 | +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
---|
| 110 | + |
---|
| 111 | + 0) ``G`` if all modules loaded have a GPL or compatible license, ``P`` if |
---|
10 | 112 | any proprietary module has been loaded. Modules without a |
---|
11 | 113 | MODULE_LICENSE or with a MODULE_LICENSE that is not recognised by |
---|
12 | 114 | insmod as GPL compatible are assumed to be proprietary. |
---|
13 | 115 | |
---|
14 | | - 2) ``F`` if any module was force loaded by ``insmod -f``, ``' '`` if all |
---|
| 116 | + 1) ``F`` if any module was force loaded by ``insmod -f``, ``' '`` if all |
---|
15 | 117 | modules were loaded normally. |
---|
16 | 118 | |
---|
17 | | - 3) ``S`` if the oops occurred on an SMP kernel running on hardware that |
---|
| 119 | + 2) ``S`` if the oops occurred on an SMP kernel running on hardware that |
---|
18 | 120 | hasn't been certified as safe to run multiprocessor. |
---|
19 | 121 | Currently this occurs only on various Athlons that are not |
---|
20 | 122 | SMP capable. |
---|
21 | 123 | |
---|
22 | | - 4) ``R`` if a module was force unloaded by ``rmmod -f``, ``' '`` if all |
---|
| 124 | + 3) ``R`` if a module was force unloaded by ``rmmod -f``, ``' '`` if all |
---|
23 | 125 | modules were unloaded normally. |
---|
24 | 126 | |
---|
25 | | - 5) ``M`` if any processor has reported a Machine Check Exception, |
---|
| 127 | + 4) ``M`` if any processor has reported a Machine Check Exception, |
---|
26 | 128 | ``' '`` if no Machine Check Exceptions have occurred. |
---|
27 | 129 | |
---|
28 | | - 6) ``B`` if a page-release function has found a bad page reference or |
---|
29 | | - some unexpected page flags. |
---|
| 130 | + 5) ``B`` If a page-release function has found a bad page reference or some |
---|
| 131 | + unexpected page flags. This indicates a hardware problem or a kernel bug; |
---|
| 132 | + there should be other information in the log indicating why this tainting |
---|
| 133 | + occurred. |
---|
30 | 134 | |
---|
31 | | - 7) ``U`` if a user or user application specifically requested that the |
---|
| 135 | + 6) ``U`` if a user or user application specifically requested that the |
---|
32 | 136 | Tainted flag be set, ``' '`` otherwise. |
---|
33 | 137 | |
---|
34 | | - 8) ``D`` if the kernel has died recently, i.e. there was an OOPS or BUG. |
---|
| 138 | + 7) ``D`` if the kernel has died recently, i.e. there was an OOPS or BUG. |
---|
35 | 139 | |
---|
36 | | - 9) ``A`` if the ACPI table has been overridden. |
---|
| 140 | + 8) ``A`` if an ACPI table has been overridden. |
---|
37 | 141 | |
---|
38 | | - 10) ``W`` if a warning has previously been issued by the kernel. |
---|
| 142 | + 9) ``W`` if a warning has previously been issued by the kernel. |
---|
39 | 143 | (Though some warnings may set more specific taint flags.) |
---|
40 | 144 | |
---|
41 | | - 11) ``C`` if a staging driver has been loaded. |
---|
| 145 | + 10) ``C`` if a staging driver has been loaded. |
---|
42 | 146 | |
---|
43 | | - 12) ``I`` if the kernel is working around a severe bug in the platform |
---|
| 147 | + 11) ``I`` if the kernel is working around a severe bug in the platform |
---|
44 | 148 | firmware (BIOS or similar). |
---|
45 | 149 | |
---|
46 | | - 13) ``O`` if an externally-built ("out-of-tree") module has been loaded. |
---|
| 150 | + 12) ``O`` if an externally-built ("out-of-tree") module has been loaded. |
---|
47 | 151 | |
---|
48 | | - 14) ``E`` if an unsigned module has been loaded in a kernel supporting |
---|
| 152 | + 13) ``E`` if an unsigned module has been loaded in a kernel supporting |
---|
49 | 153 | module signature. |
---|
50 | 154 | |
---|
51 | | - 15) ``L`` if a soft lockup has previously occurred on the system. |
---|
| 155 | + 14) ``L`` if a soft lockup has previously occurred on the system. |
---|
52 | 156 | |
---|
53 | | - 16) ``K`` if the kernel has been live patched. |
---|
| 157 | + 15) ``K`` if the kernel has been live patched. |
---|
54 | 158 | |
---|
55 | | -The primary reason for the **'Tainted: '** string is to tell kernel |
---|
56 | | -debuggers if this is a clean kernel or if anything unusual has |
---|
57 | | -occurred. Tainting is permanent: even if an offending module is |
---|
58 | | -unloaded, the tainted value remains to indicate that the kernel is not |
---|
59 | | -trustworthy. |
---|
| 159 | + 16) ``X`` Auxiliary taint, defined for and used by Linux distributors. |
---|
| 160 | + |
---|
| 161 | + 17) ``T`` Kernel was build with the randstruct plugin, which can intentionally |
---|
| 162 | + produce extremely unusual kernel structure layouts (even performance |
---|
| 163 | + pathological ones), which is important to know when debugging. Set at |
---|
| 164 | + build time. |
---|