Loading...
1.. SPDX-License-Identifier: GPL-2.0
2.. Copyright (C) 2019, Google LLC.
3
4Kernel Concurrency Sanitizer (KCSAN)
5====================================
6
7The Kernel Concurrency Sanitizer (KCSAN) is a dynamic race detector, which
8relies on compile-time instrumentation, and uses a watchpoint-based sampling
9approach to detect races. KCSAN's primary purpose is to detect `data races`_.
10
11Usage
12-----
13
14KCSAN is supported by both GCC and Clang. With GCC we require version 11 or
15later, and with Clang also require version 11 or later.
16
17To enable KCSAN configure the kernel with::
18
19 CONFIG_KCSAN = y
20
21KCSAN provides several other configuration options to customize behaviour (see
22the respective help text in ``lib/Kconfig.kcsan`` for more info).
23
24Error reports
25~~~~~~~~~~~~~
26
27A typical data race report looks like this::
28
29 ==================================================================
30 BUG: KCSAN: data-race in test_kernel_read / test_kernel_write
31
32 write to 0xffffffffc009a628 of 8 bytes by task 487 on cpu 0:
33 test_kernel_write+0x1d/0x30
34 access_thread+0x89/0xd0
35 kthread+0x23e/0x260
36 ret_from_fork+0x22/0x30
37
38 read to 0xffffffffc009a628 of 8 bytes by task 488 on cpu 6:
39 test_kernel_read+0x10/0x20
40 access_thread+0x89/0xd0
41 kthread+0x23e/0x260
42 ret_from_fork+0x22/0x30
43
44 value changed: 0x00000000000009a6 -> 0x00000000000009b2
45
46 Reported by Kernel Concurrency Sanitizer on:
47 CPU: 6 PID: 488 Comm: access_thread Not tainted 5.12.0-rc2+ #1
48 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
49 ==================================================================
50
51The header of the report provides a short summary of the functions involved in
52the race. It is followed by the access types and stack traces of the 2 threads
53involved in the data race. If KCSAN also observed a value change, the observed
54old value and new value are shown on the "value changed" line respectively.
55
56The other less common type of data race report looks like this::
57
58 ==================================================================
59 BUG: KCSAN: data-race in test_kernel_rmw_array+0x71/0xd0
60
61 race at unknown origin, with read to 0xffffffffc009bdb0 of 8 bytes by task 515 on cpu 2:
62 test_kernel_rmw_array+0x71/0xd0
63 access_thread+0x89/0xd0
64 kthread+0x23e/0x260
65 ret_from_fork+0x22/0x30
66
67 value changed: 0x0000000000002328 -> 0x0000000000002329
68
69 Reported by Kernel Concurrency Sanitizer on:
70 CPU: 2 PID: 515 Comm: access_thread Not tainted 5.12.0-rc2+ #1
71 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
72 ==================================================================
73
74This report is generated where it was not possible to determine the other
75racing thread, but a race was inferred due to the data value of the watched
76memory location having changed. These reports always show a "value changed"
77line. A common reason for reports of this type are missing instrumentation in
78the racing thread, but could also occur due to e.g. DMA accesses. Such reports
79are shown only if ``CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN=y``, which is
80enabled by default.
81
82Selective analysis
83~~~~~~~~~~~~~~~~~~
84
85It may be desirable to disable data race detection for specific accesses,
86functions, compilation units, or entire subsystems. For static blacklisting,
87the below options are available:
88
89* KCSAN understands the ``data_race(expr)`` annotation, which tells KCSAN that
90 any data races due to accesses in ``expr`` should be ignored and resulting
91 behaviour when encountering a data race is deemed safe. Please see
92 `"Marking Shared-Memory Accesses" in the LKMM`_ for more information.
93
94* Similar to ``data_race(...)``, the type qualifier ``__data_racy`` can be used
95 to document that all data races due to accesses to a variable are intended
96 and should be ignored by KCSAN::
97
98 struct foo {
99 ...
100 int __data_racy stats_counter;
101 ...
102 };
103
104* Disabling data race detection for entire functions can be accomplished by
105 using the function attribute ``__no_kcsan``::
106
107 __no_kcsan
108 void foo(void) {
109 ...
110
111 To dynamically limit for which functions to generate reports, see the
112 `DebugFS interface`_ blacklist/whitelist feature.
113
114* To disable data race detection for a particular compilation unit, add to the
115 ``Makefile``::
116
117 KCSAN_SANITIZE_file.o := n
118
119* To disable data race detection for all compilation units listed in a
120 ``Makefile``, add to the respective ``Makefile``::
121
122 KCSAN_SANITIZE := n
123
124.. _"Marking Shared-Memory Accesses" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/access-marking.txt
125
126Furthermore, it is possible to tell KCSAN to show or hide entire classes of
127data races, depending on preferences. These can be changed via the following
128Kconfig options:
129
130* ``CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY``: If enabled and a conflicting write
131 is observed via a watchpoint, but the data value of the memory location was
132 observed to remain unchanged, do not report the data race.
133
134* ``CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC``: Assume that plain aligned writes
135 up to word size are atomic by default. Assumes that such writes are not
136 subject to unsafe compiler optimizations resulting in data races. The option
137 causes KCSAN to not report data races due to conflicts where the only plain
138 accesses are aligned writes up to word size.
139
140* ``CONFIG_KCSAN_PERMISSIVE``: Enable additional permissive rules to ignore
141 certain classes of common data races. Unlike the above, the rules are more
142 complex involving value-change patterns, access type, and address. This
143 option depends on ``CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY=y``. For details
144 please see the ``kernel/kcsan/permissive.h``. Testers and maintainers that
145 only focus on reports from specific subsystems and not the whole kernel are
146 recommended to disable this option.
147
148To use the strictest possible rules, select ``CONFIG_KCSAN_STRICT=y``, which
149configures KCSAN to follow the Linux-kernel memory consistency model (LKMM) as
150closely as possible.
151
152DebugFS interface
153~~~~~~~~~~~~~~~~~
154
155The file ``/sys/kernel/debug/kcsan`` provides the following interface:
156
157* Reading ``/sys/kernel/debug/kcsan`` returns various runtime statistics.
158
159* Writing ``on`` or ``off`` to ``/sys/kernel/debug/kcsan`` allows turning KCSAN
160 on or off, respectively.
161
162* Writing ``!some_func_name`` to ``/sys/kernel/debug/kcsan`` adds
163 ``some_func_name`` to the report filter list, which (by default) blacklists
164 reporting data races where either one of the top stackframes are a function
165 in the list.
166
167* Writing either ``blacklist`` or ``whitelist`` to ``/sys/kernel/debug/kcsan``
168 changes the report filtering behaviour. For example, the blacklist feature
169 can be used to silence frequently occurring data races; the whitelist feature
170 can help with reproduction and testing of fixes.
171
172Tuning performance
173~~~~~~~~~~~~~~~~~~
174
175Core parameters that affect KCSAN's overall performance and bug detection
176ability are exposed as kernel command-line arguments whose defaults can also be
177changed via the corresponding Kconfig options.
178
179* ``kcsan.skip_watch`` (``CONFIG_KCSAN_SKIP_WATCH``): Number of per-CPU memory
180 operations to skip, before another watchpoint is set up. Setting up
181 watchpoints more frequently will result in the likelihood of races to be
182 observed to increase. This parameter has the most significant impact on
183 overall system performance and race detection ability.
184
185* ``kcsan.udelay_task`` (``CONFIG_KCSAN_UDELAY_TASK``): For tasks, the
186 microsecond delay to stall execution after a watchpoint has been set up.
187 Larger values result in the window in which we may observe a race to
188 increase.
189
190* ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For
191 interrupts, the microsecond delay to stall execution after a watchpoint has
192 been set up. Interrupts have tighter latency requirements, and their delay
193 should generally be smaller than the one chosen for tasks.
194
195They may be tweaked at runtime via ``/sys/module/kcsan/parameters/``.
196
197Data Races
198----------
199
200In an execution, two memory accesses form a *data race* if they *conflict*,
201they happen concurrently in different threads, and at least one of them is a
202*plain access*; they *conflict* if both access the same memory location, and at
203least one is a write. For a more thorough discussion and definition, see `"Plain
204Accesses and Data Races" in the LKMM`_.
205
206.. _"Plain Accesses and Data Races" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/explanation.txt#n1922
207
208Relationship with the Linux-Kernel Memory Consistency Model (LKMM)
209~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
210
211The LKMM defines the propagation and ordering rules of various memory
212operations, which gives developers the ability to reason about concurrent code.
213Ultimately this allows to determine the possible executions of concurrent code,
214and if that code is free from data races.
215
216KCSAN is aware of *marked atomic operations* (``READ_ONCE``, ``WRITE_ONCE``,
217``atomic_*``, etc.), and a subset of ordering guarantees implied by memory
218barriers. With ``CONFIG_KCSAN_WEAK_MEMORY=y``, KCSAN models load or store
219buffering, and can detect missing ``smp_mb()``, ``smp_wmb()``, ``smp_rmb()``,
220``smp_store_release()``, and all ``atomic_*`` operations with equivalent
221implied barriers.
222
223Note, KCSAN will not report all data races due to missing memory ordering,
224specifically where a memory barrier would be required to prohibit subsequent
225memory operation from reordering before the barrier. Developers should
226therefore carefully consider the required memory ordering requirements that
227remain unchecked.
228
229Race Detection Beyond Data Races
230--------------------------------
231
232For code with complex concurrency design, race-condition bugs may not always
233manifest as data races. Race conditions occur if concurrently executing
234operations result in unexpected system behaviour. On the other hand, data races
235are defined at the C-language level. The following macros can be used to check
236properties of concurrent code where bugs would not manifest as data races.
237
238.. kernel-doc:: include/linux/kcsan-checks.h
239 :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_WRITER_SCOPED
240 ASSERT_EXCLUSIVE_ACCESS ASSERT_EXCLUSIVE_ACCESS_SCOPED
241 ASSERT_EXCLUSIVE_BITS
242
243Implementation Details
244----------------------
245
246KCSAN relies on observing that two accesses happen concurrently. Crucially, we
247want to (a) increase the chances of observing races (especially for races that
248manifest rarely), and (b) be able to actually observe them. We can accomplish
249(a) by injecting various delays, and (b) by using address watchpoints (or
250breakpoints).
251
252If we deliberately stall a memory access, while we have a watchpoint for its
253address set up, and then observe the watchpoint to fire, two accesses to the
254same address just raced. Using hardware watchpoints, this is the approach taken
255in `DataCollider
256<http://usenix.org/legacy/events/osdi10/tech/full_papers/Erickson.pdf>`_.
257Unlike DataCollider, KCSAN does not use hardware watchpoints, but instead
258relies on compiler instrumentation and "soft watchpoints".
259
260In KCSAN, watchpoints are implemented using an efficient encoding that stores
261access type, size, and address in a long; the benefits of using "soft
262watchpoints" are portability and greater flexibility. KCSAN then relies on the
263compiler instrumenting plain accesses. For each instrumented plain access:
264
2651. Check if a matching watchpoint exists; if yes, and at least one access is a
266 write, then we encountered a racing access.
267
2682. Periodically, if no matching watchpoint exists, set up a watchpoint and
269 stall for a small randomized delay.
270
2713. Also check the data value before the delay, and re-check the data value
272 after delay; if the values mismatch, we infer a race of unknown origin.
273
274To detect data races between plain and marked accesses, KCSAN also annotates
275marked accesses, but only to check if a watchpoint exists; i.e. KCSAN never
276sets up a watchpoint on marked accesses. By never setting up watchpoints for
277marked operations, if all accesses to a variable that is accessed concurrently
278are properly marked, KCSAN will never trigger a watchpoint and therefore never
279report the accesses.
280
281Modeling Weak Memory
282~~~~~~~~~~~~~~~~~~~~
283
284KCSAN's approach to detecting data races due to missing memory barriers is
285based on modeling access reordering (with ``CONFIG_KCSAN_WEAK_MEMORY=y``).
286Each plain memory access for which a watchpoint is set up, is also selected for
287simulated reordering within the scope of its function (at most 1 in-flight
288access).
289
290Once an access has been selected for reordering, it is checked along every
291other access until the end of the function scope. If an appropriate memory
292barrier is encountered, the access will no longer be considered for simulated
293reordering.
294
295When the result of a memory operation should be ordered by a barrier, KCSAN can
296then detect data races where the conflict only occurs as a result of a missing
297barrier. Consider the example::
298
299 int x, flag;
300 void T1(void)
301 {
302 x = 1; // data race!
303 WRITE_ONCE(flag, 1); // correct: smp_store_release(&flag, 1)
304 }
305 void T2(void)
306 {
307 while (!READ_ONCE(flag)); // correct: smp_load_acquire(&flag)
308 ... = x; // data race!
309 }
310
311When weak memory modeling is enabled, KCSAN can consider ``x`` in ``T1`` for
312simulated reordering. After the write of ``flag``, ``x`` is again checked for
313concurrent accesses: because ``T2`` is able to proceed after the write of
314``flag``, a data race is detected. With the correct barriers in place, ``x``
315would not be considered for reordering after the proper release of ``flag``,
316and no data race would be detected.
317
318Deliberate trade-offs in complexity but also practical limitations mean only a
319subset of data races due to missing memory barriers can be detected. With
320currently available compiler support, the implementation is limited to modeling
321the effects of "buffering" (delaying accesses), since the runtime cannot
322"prefetch" accesses. Also recall that watchpoints are only set up for plain
323accesses, and the only access type for which KCSAN simulates reordering. This
324means reordering of marked accesses is not modeled.
325
326A consequence of the above is that acquire operations do not require barrier
327instrumentation (no prefetching). Furthermore, marked accesses introducing
328address or control dependencies do not require special handling (the marked
329access cannot be reordered, later dependent accesses cannot be prefetched).
330
331Key Properties
332~~~~~~~~~~~~~~
333
3341. **Memory Overhead:** The overall memory overhead is only a few MiB
335 depending on configuration. The current implementation uses a small array of
336 longs to encode watchpoint information, which is negligible.
337
3382. **Performance Overhead:** KCSAN's runtime aims to be minimal, using an
339 efficient watchpoint encoding that does not require acquiring any shared
340 locks in the fast-path. For kernel boot on a system with 8 CPUs:
341
342 - 5.0x slow-down with the default KCSAN config;
343 - 2.8x slow-down from runtime fast-path overhead only (set very large
344 ``KCSAN_SKIP_WATCH`` and unset ``KCSAN_SKIP_WATCH_RANDOMIZE``).
345
3463. **Annotation Overheads:** Minimal annotations are required outside the KCSAN
347 runtime. As a result, maintenance overheads are minimal as the kernel
348 evolves.
349
3504. **Detects Racy Writes from Devices:** Due to checking data values upon
351 setting up watchpoints, racy writes from devices can also be detected.
352
3535. **Memory Ordering:** KCSAN is aware of only a subset of LKMM ordering rules;
354 this may result in missed data races (false negatives).
355
3566. **Analysis Accuracy:** For observed executions, due to using a sampling
357 strategy, the analysis is *unsound* (false negatives possible), but aims to
358 be complete (no false positives).
359
360Alternatives Considered
361-----------------------
362
363An alternative data race detection approach for the kernel can be found in the
364`Kernel Thread Sanitizer (KTSAN)
365<https://github.com/google/kernel-sanitizers/blob/master/KTSAN.md>`_.
366KTSAN is a happens-before data race detector, which explicitly establishes the
367happens-before order between memory operations, which can then be used to
368determine data races as defined in `Data Races`_.
369
370To build a correct happens-before relation, KTSAN must be aware of all ordering
371rules of the LKMM and synchronization primitives. Unfortunately, any omission
372leads to large numbers of false positives, which is especially detrimental in
373the context of the kernel which includes numerous custom synchronization
374mechanisms. To track the happens-before relation, KTSAN's implementation
375requires metadata for each memory location (shadow memory), which for each page
376corresponds to 4 pages of shadow memory, and can translate into overhead of
377tens of GiB on a large system.
1.. SPDX-License-Identifier: GPL-2.0
2.. Copyright (C) 2019, Google LLC.
3
4The Kernel Concurrency Sanitizer (KCSAN)
5========================================
6
7The Kernel Concurrency Sanitizer (KCSAN) is a dynamic race detector, which
8relies on compile-time instrumentation, and uses a watchpoint-based sampling
9approach to detect races. KCSAN's primary purpose is to detect `data races`_.
10
11Usage
12-----
13
14KCSAN is supported by both GCC and Clang. With GCC we require version 11 or
15later, and with Clang also require version 11 or later.
16
17To enable KCSAN configure the kernel with::
18
19 CONFIG_KCSAN = y
20
21KCSAN provides several other configuration options to customize behaviour (see
22the respective help text in ``lib/Kconfig.kcsan`` for more info).
23
24Error reports
25~~~~~~~~~~~~~
26
27A typical data race report looks like this::
28
29 ==================================================================
30 BUG: KCSAN: data-race in test_kernel_read / test_kernel_write
31
32 write to 0xffffffffc009a628 of 8 bytes by task 487 on cpu 0:
33 test_kernel_write+0x1d/0x30
34 access_thread+0x89/0xd0
35 kthread+0x23e/0x260
36 ret_from_fork+0x22/0x30
37
38 read to 0xffffffffc009a628 of 8 bytes by task 488 on cpu 6:
39 test_kernel_read+0x10/0x20
40 access_thread+0x89/0xd0
41 kthread+0x23e/0x260
42 ret_from_fork+0x22/0x30
43
44 value changed: 0x00000000000009a6 -> 0x00000000000009b2
45
46 Reported by Kernel Concurrency Sanitizer on:
47 CPU: 6 PID: 488 Comm: access_thread Not tainted 5.12.0-rc2+ #1
48 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
49 ==================================================================
50
51The header of the report provides a short summary of the functions involved in
52the race. It is followed by the access types and stack traces of the 2 threads
53involved in the data race. If KCSAN also observed a value change, the observed
54old value and new value are shown on the "value changed" line respectively.
55
56The other less common type of data race report looks like this::
57
58 ==================================================================
59 BUG: KCSAN: data-race in test_kernel_rmw_array+0x71/0xd0
60
61 race at unknown origin, with read to 0xffffffffc009bdb0 of 8 bytes by task 515 on cpu 2:
62 test_kernel_rmw_array+0x71/0xd0
63 access_thread+0x89/0xd0
64 kthread+0x23e/0x260
65 ret_from_fork+0x22/0x30
66
67 value changed: 0x0000000000002328 -> 0x0000000000002329
68
69 Reported by Kernel Concurrency Sanitizer on:
70 CPU: 2 PID: 515 Comm: access_thread Not tainted 5.12.0-rc2+ #1
71 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014
72 ==================================================================
73
74This report is generated where it was not possible to determine the other
75racing thread, but a race was inferred due to the data value of the watched
76memory location having changed. These reports always show a "value changed"
77line. A common reason for reports of this type are missing instrumentation in
78the racing thread, but could also occur due to e.g. DMA accesses. Such reports
79are shown only if ``CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN=y``, which is
80enabled by default.
81
82Selective analysis
83~~~~~~~~~~~~~~~~~~
84
85It may be desirable to disable data race detection for specific accesses,
86functions, compilation units, or entire subsystems. For static blacklisting,
87the below options are available:
88
89* KCSAN understands the ``data_race(expr)`` annotation, which tells KCSAN that
90 any data races due to accesses in ``expr`` should be ignored and resulting
91 behaviour when encountering a data race is deemed safe. Please see
92 `"Marking Shared-Memory Accesses" in the LKMM`_ for more information.
93
94* Disabling data race detection for entire functions can be accomplished by
95 using the function attribute ``__no_kcsan``::
96
97 __no_kcsan
98 void foo(void) {
99 ...
100
101 To dynamically limit for which functions to generate reports, see the
102 `DebugFS interface`_ blacklist/whitelist feature.
103
104* To disable data race detection for a particular compilation unit, add to the
105 ``Makefile``::
106
107 KCSAN_SANITIZE_file.o := n
108
109* To disable data race detection for all compilation units listed in a
110 ``Makefile``, add to the respective ``Makefile``::
111
112 KCSAN_SANITIZE := n
113
114.. _"Marking Shared-Memory Accesses" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/access-marking.txt
115
116Furthermore, it is possible to tell KCSAN to show or hide entire classes of
117data races, depending on preferences. These can be changed via the following
118Kconfig options:
119
120* ``CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY``: If enabled and a conflicting write
121 is observed via a watchpoint, but the data value of the memory location was
122 observed to remain unchanged, do not report the data race.
123
124* ``CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC``: Assume that plain aligned writes
125 up to word size are atomic by default. Assumes that such writes are not
126 subject to unsafe compiler optimizations resulting in data races. The option
127 causes KCSAN to not report data races due to conflicts where the only plain
128 accesses are aligned writes up to word size.
129
130DebugFS interface
131~~~~~~~~~~~~~~~~~
132
133The file ``/sys/kernel/debug/kcsan`` provides the following interface:
134
135* Reading ``/sys/kernel/debug/kcsan`` returns various runtime statistics.
136
137* Writing ``on`` or ``off`` to ``/sys/kernel/debug/kcsan`` allows turning KCSAN
138 on or off, respectively.
139
140* Writing ``!some_func_name`` to ``/sys/kernel/debug/kcsan`` adds
141 ``some_func_name`` to the report filter list, which (by default) blacklists
142 reporting data races where either one of the top stackframes are a function
143 in the list.
144
145* Writing either ``blacklist`` or ``whitelist`` to ``/sys/kernel/debug/kcsan``
146 changes the report filtering behaviour. For example, the blacklist feature
147 can be used to silence frequently occurring data races; the whitelist feature
148 can help with reproduction and testing of fixes.
149
150Tuning performance
151~~~~~~~~~~~~~~~~~~
152
153Core parameters that affect KCSAN's overall performance and bug detection
154ability are exposed as kernel command-line arguments whose defaults can also be
155changed via the corresponding Kconfig options.
156
157* ``kcsan.skip_watch`` (``CONFIG_KCSAN_SKIP_WATCH``): Number of per-CPU memory
158 operations to skip, before another watchpoint is set up. Setting up
159 watchpoints more frequently will result in the likelihood of races to be
160 observed to increase. This parameter has the most significant impact on
161 overall system performance and race detection ability.
162
163* ``kcsan.udelay_task`` (``CONFIG_KCSAN_UDELAY_TASK``): For tasks, the
164 microsecond delay to stall execution after a watchpoint has been set up.
165 Larger values result in the window in which we may observe a race to
166 increase.
167
168* ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For
169 interrupts, the microsecond delay to stall execution after a watchpoint has
170 been set up. Interrupts have tighter latency requirements, and their delay
171 should generally be smaller than the one chosen for tasks.
172
173They may be tweaked at runtime via ``/sys/module/kcsan/parameters/``.
174
175Data Races
176----------
177
178In an execution, two memory accesses form a *data race* if they *conflict*,
179they happen concurrently in different threads, and at least one of them is a
180*plain access*; they *conflict* if both access the same memory location, and at
181least one is a write. For a more thorough discussion and definition, see `"Plain
182Accesses and Data Races" in the LKMM`_.
183
184.. _"Plain Accesses and Data Races" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/explanation.txt#n1922
185
186Relationship with the Linux-Kernel Memory Consistency Model (LKMM)
187~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
188
189The LKMM defines the propagation and ordering rules of various memory
190operations, which gives developers the ability to reason about concurrent code.
191Ultimately this allows to determine the possible executions of concurrent code,
192and if that code is free from data races.
193
194KCSAN is aware of *marked atomic operations* (``READ_ONCE``, ``WRITE_ONCE``,
195``atomic_*``, etc.), but is oblivious of any ordering guarantees and simply
196assumes that memory barriers are placed correctly. In other words, KCSAN
197assumes that as long as a plain access is not observed to race with another
198conflicting access, memory operations are correctly ordered.
199
200This means that KCSAN will not report *potential* data races due to missing
201memory ordering. Developers should therefore carefully consider the required
202memory ordering requirements that remain unchecked. If, however, missing
203memory ordering (that is observable with a particular compiler and
204architecture) leads to an observable data race (e.g. entering a critical
205section erroneously), KCSAN would report the resulting data race.
206
207Race Detection Beyond Data Races
208--------------------------------
209
210For code with complex concurrency design, race-condition bugs may not always
211manifest as data races. Race conditions occur if concurrently executing
212operations result in unexpected system behaviour. On the other hand, data races
213are defined at the C-language level. The following macros can be used to check
214properties of concurrent code where bugs would not manifest as data races.
215
216.. kernel-doc:: include/linux/kcsan-checks.h
217 :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_WRITER_SCOPED
218 ASSERT_EXCLUSIVE_ACCESS ASSERT_EXCLUSIVE_ACCESS_SCOPED
219 ASSERT_EXCLUSIVE_BITS
220
221Implementation Details
222----------------------
223
224KCSAN relies on observing that two accesses happen concurrently. Crucially, we
225want to (a) increase the chances of observing races (especially for races that
226manifest rarely), and (b) be able to actually observe them. We can accomplish
227(a) by injecting various delays, and (b) by using address watchpoints (or
228breakpoints).
229
230If we deliberately stall a memory access, while we have a watchpoint for its
231address set up, and then observe the watchpoint to fire, two accesses to the
232same address just raced. Using hardware watchpoints, this is the approach taken
233in `DataCollider
234<http://usenix.org/legacy/events/osdi10/tech/full_papers/Erickson.pdf>`_.
235Unlike DataCollider, KCSAN does not use hardware watchpoints, but instead
236relies on compiler instrumentation and "soft watchpoints".
237
238In KCSAN, watchpoints are implemented using an efficient encoding that stores
239access type, size, and address in a long; the benefits of using "soft
240watchpoints" are portability and greater flexibility. KCSAN then relies on the
241compiler instrumenting plain accesses. For each instrumented plain access:
242
2431. Check if a matching watchpoint exists; if yes, and at least one access is a
244 write, then we encountered a racing access.
245
2462. Periodically, if no matching watchpoint exists, set up a watchpoint and
247 stall for a small randomized delay.
248
2493. Also check the data value before the delay, and re-check the data value
250 after delay; if the values mismatch, we infer a race of unknown origin.
251
252To detect data races between plain and marked accesses, KCSAN also annotates
253marked accesses, but only to check if a watchpoint exists; i.e. KCSAN never
254sets up a watchpoint on marked accesses. By never setting up watchpoints for
255marked operations, if all accesses to a variable that is accessed concurrently
256are properly marked, KCSAN will never trigger a watchpoint and therefore never
257report the accesses.
258
259Key Properties
260~~~~~~~~~~~~~~
261
2621. **Memory Overhead:** The overall memory overhead is only a few MiB
263 depending on configuration. The current implementation uses a small array of
264 longs to encode watchpoint information, which is negligible.
265
2662. **Performance Overhead:** KCSAN's runtime aims to be minimal, using an
267 efficient watchpoint encoding that does not require acquiring any shared
268 locks in the fast-path. For kernel boot on a system with 8 CPUs:
269
270 - 5.0x slow-down with the default KCSAN config;
271 - 2.8x slow-down from runtime fast-path overhead only (set very large
272 ``KCSAN_SKIP_WATCH`` and unset ``KCSAN_SKIP_WATCH_RANDOMIZE``).
273
2743. **Annotation Overheads:** Minimal annotations are required outside the KCSAN
275 runtime. As a result, maintenance overheads are minimal as the kernel
276 evolves.
277
2784. **Detects Racy Writes from Devices:** Due to checking data values upon
279 setting up watchpoints, racy writes from devices can also be detected.
280
2815. **Memory Ordering:** KCSAN is *not* explicitly aware of the LKMM's ordering
282 rules; this may result in missed data races (false negatives).
283
2846. **Analysis Accuracy:** For observed executions, due to using a sampling
285 strategy, the analysis is *unsound* (false negatives possible), but aims to
286 be complete (no false positives).
287
288Alternatives Considered
289-----------------------
290
291An alternative data race detection approach for the kernel can be found in the
292`Kernel Thread Sanitizer (KTSAN) <https://github.com/google/ktsan/wiki>`_.
293KTSAN is a happens-before data race detector, which explicitly establishes the
294happens-before order between memory operations, which can then be used to
295determine data races as defined in `Data Races`_.
296
297To build a correct happens-before relation, KTSAN must be aware of all ordering
298rules of the LKMM and synchronization primitives. Unfortunately, any omission
299leads to large numbers of false positives, which is especially detrimental in
300the context of the kernel which includes numerous custom synchronization
301mechanisms. To track the happens-before relation, KTSAN's implementation
302requires metadata for each memory location (shadow memory), which for each page
303corresponds to 4 pages of shadow memory, and can translate into overhead of
304tens of GiB on a large system.