Loading...
1=========================
2CPU hotplug in the Kernel
3=========================
4
5:Date: September, 2021
6:Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
7 Rusty Russell <rusty@rustcorp.com.au>,
8 Srivatsa Vaddagiri <vatsa@in.ibm.com>,
9 Ashok Raj <ashok.raj@intel.com>,
10 Joel Schopp <jschopp@austin.ibm.com>,
11 Thomas Gleixner <tglx@linutronix.de>
12
13Introduction
14============
15
16Modern advances in system architectures have introduced advanced error
17reporting and correction capabilities in processors. There are couple OEMS that
18support NUMA hardware which are hot pluggable as well, where physical node
19insertion and removal require support for CPU hotplug.
20
21Such advances require CPUs available to a kernel to be removed either for
22provisioning reasons, or for RAS purposes to keep an offending CPU off
23system execution path. Hence the need for CPU hotplug support in the
24Linux kernel.
25
26A more novel use of CPU-hotplug support is its use today in suspend resume
27support for SMP. Dual-core and HT support makes even a laptop run SMP kernels
28which didn't support these methods.
29
30
31Command Line Switches
32=====================
33``maxcpus=n``
34 Restrict boot time CPUs to *n*. Say if you have four CPUs, using
35 ``maxcpus=2`` will only boot two. You can choose to bring the
36 other CPUs later online.
37
38``nr_cpus=n``
39 Restrict the total amount of CPUs the kernel will support. If the number
40 supplied here is lower than the number of physically available CPUs, then
41 those CPUs can not be brought online later.
42
43``possible_cpus=n``
44 This option sets ``possible_cpus`` bits in ``cpu_possible_mask``.
45
46 This option is limited to the X86 and S390 architecture.
47
48``cpu0_hotplug``
49 Allow to shutdown CPU0.
50
51 This option is limited to the X86 architecture.
52
53CPU maps
54========
55
56``cpu_possible_mask``
57 Bitmap of possible CPUs that can ever be available in the
58 system. This is used to allocate some boot time memory for per_cpu variables
59 that aren't designed to grow/shrink as CPUs are made available or removed.
60 Once set during boot time discovery phase, the map is static, i.e no bits
61 are added or removed anytime. Trimming it accurately for your system needs
62 upfront can save some boot time memory.
63
64``cpu_online_mask``
65 Bitmap of all CPUs currently online. Its set in ``__cpu_up()``
66 after a CPU is available for kernel scheduling and ready to receive
67 interrupts from devices. Its cleared when a CPU is brought down using
68 ``__cpu_disable()``, before which all OS services including interrupts are
69 migrated to another target CPU.
70
71``cpu_present_mask``
72 Bitmap of CPUs currently present in the system. Not all
73 of them may be online. When physical hotplug is processed by the relevant
74 subsystem (e.g ACPI) can change and new bit either be added or removed
75 from the map depending on the event is hot-add/hot-remove. There are currently
76 no locking rules as of now. Typical usage is to init topology during boot,
77 at which time hotplug is disabled.
78
79You really don't need to manipulate any of the system CPU maps. They should
80be read-only for most use. When setting up per-cpu resources almost always use
81``cpu_possible_mask`` or ``for_each_possible_cpu()`` to iterate. To macro
82``for_each_cpu()`` can be used to iterate over a custom CPU mask.
83
84Never use anything other than ``cpumask_t`` to represent bitmap of CPUs.
85
86
87Using CPU hotplug
88=================
89
90The kernel option *CONFIG_HOTPLUG_CPU* needs to be enabled. It is currently
91available on multiple architectures including ARM, MIPS, PowerPC and X86. The
92configuration is done via the sysfs interface::
93
94 $ ls -lh /sys/devices/system/cpu
95 total 0
96 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu0
97 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu1
98 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu2
99 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu3
100 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu4
101 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu5
102 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu6
103 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu7
104 drwxr-xr-x 2 root root 0 Dec 21 16:33 hotplug
105 -r--r--r-- 1 root root 4.0K Dec 21 16:33 offline
106 -r--r--r-- 1 root root 4.0K Dec 21 16:33 online
107 -r--r--r-- 1 root root 4.0K Dec 21 16:33 possible
108 -r--r--r-- 1 root root 4.0K Dec 21 16:33 present
109
110The files *offline*, *online*, *possible*, *present* represent the CPU masks.
111Each CPU folder contains an *online* file which controls the logical on (1) and
112off (0) state. To logically shutdown CPU4::
113
114 $ echo 0 > /sys/devices/system/cpu/cpu4/online
115 smpboot: CPU 4 is now offline
116
117Once the CPU is shutdown, it will be removed from */proc/interrupts*,
118*/proc/cpuinfo* and should also not be shown visible by the *top* command. To
119bring CPU4 back online::
120
121 $ echo 1 > /sys/devices/system/cpu/cpu4/online
122 smpboot: Booting Node 0 Processor 4 APIC 0x1
123
124The CPU is usable again. This should work on all CPUs, but CPU0 is often special
125and excluded from CPU hotplug.
126
127The CPU hotplug coordination
128============================
129
130The offline case
131----------------
132
133Once a CPU has been logically shutdown the teardown callbacks of registered
134hotplug states will be invoked, starting with ``CPUHP_ONLINE`` and terminating
135at state ``CPUHP_OFFLINE``. This includes:
136
137* If tasks are frozen due to a suspend operation then *cpuhp_tasks_frozen*
138 will be set to true.
139* All processes are migrated away from this outgoing CPU to new CPUs.
140 The new CPU is chosen from each process' current cpuset, which may be
141 a subset of all online CPUs.
142* All interrupts targeted to this CPU are migrated to a new CPU
143* timers are also migrated to a new CPU
144* Once all services are migrated, kernel calls an arch specific routine
145 ``__cpu_disable()`` to perform arch specific cleanup.
146
147
148The CPU hotplug API
149===================
150
151CPU hotplug state machine
152-------------------------
153
154CPU hotplug uses a trivial state machine with a linear state space from
155CPUHP_OFFLINE to CPUHP_ONLINE. Each state has a startup and a teardown
156callback.
157
158When a CPU is onlined, the startup callbacks are invoked sequentially until
159the state CPUHP_ONLINE is reached. They can also be invoked when the
160callbacks of a state are set up or an instance is added to a multi-instance
161state.
162
163When a CPU is offlined the teardown callbacks are invoked in the reverse
164order sequentially until the state CPUHP_OFFLINE is reached. They can also
165be invoked when the callbacks of a state are removed or an instance is
166removed from a multi-instance state.
167
168If a usage site requires only a callback in one direction of the hotplug
169operations (CPU online or CPU offline) then the other not-required callback
170can be set to NULL when the state is set up.
171
172The state space is divided into three sections:
173
174* The PREPARE section
175
176 The PREPARE section covers the state space from CPUHP_OFFLINE to
177 CPUHP_BRINGUP_CPU.
178
179 The startup callbacks in this section are invoked before the CPU is
180 started during a CPU online operation. The teardown callbacks are invoked
181 after the CPU has become dysfunctional during a CPU offline operation.
182
183 The callbacks are invoked on a control CPU as they can't obviously run on
184 the hotplugged CPU which is either not yet started or has become
185 dysfunctional already.
186
187 The startup callbacks are used to setup resources which are required to
188 bring a CPU successfully online. The teardown callbacks are used to free
189 resources or to move pending work to an online CPU after the hotplugged
190 CPU became dysfunctional.
191
192 The startup callbacks are allowed to fail. If a callback fails, the CPU
193 online operation is aborted and the CPU is brought down to the previous
194 state (usually CPUHP_OFFLINE) again.
195
196 The teardown callbacks in this section are not allowed to fail.
197
198* The STARTING section
199
200 The STARTING section covers the state space between CPUHP_BRINGUP_CPU + 1
201 and CPUHP_AP_ONLINE.
202
203 The startup callbacks in this section are invoked on the hotplugged CPU
204 with interrupts disabled during a CPU online operation in the early CPU
205 setup code. The teardown callbacks are invoked with interrupts disabled
206 on the hotplugged CPU during a CPU offline operation shortly before the
207 CPU is completely shut down.
208
209 The callbacks in this section are not allowed to fail.
210
211 The callbacks are used for low level hardware initialization/shutdown and
212 for core subsystems.
213
214* The ONLINE section
215
216 The ONLINE section covers the state space between CPUHP_AP_ONLINE + 1 and
217 CPUHP_ONLINE.
218
219 The startup callbacks in this section are invoked on the hotplugged CPU
220 during a CPU online operation. The teardown callbacks are invoked on the
221 hotplugged CPU during a CPU offline operation.
222
223 The callbacks are invoked in the context of the per CPU hotplug thread,
224 which is pinned on the hotplugged CPU. The callbacks are invoked with
225 interrupts and preemption enabled.
226
227 The callbacks are allowed to fail. When a callback fails the hotplug
228 operation is aborted and the CPU is brought back to the previous state.
229
230CPU online/offline operations
231-----------------------------
232
233A successful online operation looks like this::
234
235 [CPUHP_OFFLINE]
236 [CPUHP_OFFLINE + 1]->startup() -> success
237 [CPUHP_OFFLINE + 2]->startup() -> success
238 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
239 ...
240 [CPUHP_BRINGUP_CPU]->startup() -> success
241 === End of PREPARE section
242 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
243 ...
244 [CPUHP_AP_ONLINE]->startup() -> success
245 === End of STARTUP section
246 [CPUHP_AP_ONLINE + 1]->startup() -> success
247 ...
248 [CPUHP_ONLINE - 1]->startup() -> success
249 [CPUHP_ONLINE]
250
251A successful offline operation looks like this::
252
253 [CPUHP_ONLINE]
254 [CPUHP_ONLINE - 1]->teardown() -> success
255 ...
256 [CPUHP_AP_ONLINE + 1]->teardown() -> success
257 === Start of STARTUP section
258 [CPUHP_AP_ONLINE]->teardown() -> success
259 ...
260 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
261 ...
262 === Start of PREPARE section
263 [CPUHP_BRINGUP_CPU]->teardown()
264 [CPUHP_OFFLINE + 3]->teardown()
265 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
266 [CPUHP_OFFLINE + 1]->teardown()
267 [CPUHP_OFFLINE]
268
269A failed online operation looks like this::
270
271 [CPUHP_OFFLINE]
272 [CPUHP_OFFLINE + 1]->startup() -> success
273 [CPUHP_OFFLINE + 2]->startup() -> success
274 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
275 ...
276 [CPUHP_BRINGUP_CPU]->startup() -> success
277 === End of PREPARE section
278 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
279 ...
280 [CPUHP_AP_ONLINE]->startup() -> success
281 === End of STARTUP section
282 [CPUHP_AP_ONLINE + 1]->startup() -> success
283 ---
284 [CPUHP_AP_ONLINE + N]->startup() -> fail
285 [CPUHP_AP_ONLINE + (N - 1)]->teardown()
286 ...
287 [CPUHP_AP_ONLINE + 1]->teardown()
288 === Start of STARTUP section
289 [CPUHP_AP_ONLINE]->teardown()
290 ...
291 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
292 ...
293 === Start of PREPARE section
294 [CPUHP_BRINGUP_CPU]->teardown()
295 [CPUHP_OFFLINE + 3]->teardown()
296 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
297 [CPUHP_OFFLINE + 1]->teardown()
298 [CPUHP_OFFLINE]
299
300A failed offline operation looks like this::
301
302 [CPUHP_ONLINE]
303 [CPUHP_ONLINE - 1]->teardown() -> success
304 ...
305 [CPUHP_ONLINE - N]->teardown() -> fail
306 [CPUHP_ONLINE - (N - 1)]->startup()
307 ...
308 [CPUHP_ONLINE - 1]->startup()
309 [CPUHP_ONLINE]
310
311Recursive failures cannot be handled sensibly. Look at the following
312example of a recursive fail due to a failed offline operation: ::
313
314 [CPUHP_ONLINE]
315 [CPUHP_ONLINE - 1]->teardown() -> success
316 ...
317 [CPUHP_ONLINE - N]->teardown() -> fail
318 [CPUHP_ONLINE - (N - 1)]->startup() -> success
319 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
320
321The CPU hotplug state machine stops right here and does not try to go back
322down again because that would likely result in an endless loop::
323
324 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
325 [CPUHP_ONLINE - N]->teardown() -> fail
326 [CPUHP_ONLINE - (N - 1)]->startup() -> success
327 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
328 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
329 [CPUHP_ONLINE - N]->teardown() -> fail
330
331Lather, rinse and repeat. In this case the CPU left in state::
332
333 [CPUHP_ONLINE - (N - 1)]
334
335which at least lets the system make progress and gives the user a chance to
336debug or even resolve the situation.
337
338Allocating a state
339------------------
340
341There are two ways to allocate a CPU hotplug state:
342
343* Static allocation
344
345 Static allocation has to be used when the subsystem or driver has
346 ordering requirements versus other CPU hotplug states. E.g. the PERF core
347 startup callback has to be invoked before the PERF driver startup
348 callbacks during a CPU online operation. During a CPU offline operation
349 the driver teardown callbacks have to be invoked before the core teardown
350 callback. The statically allocated states are described by constants in
351 the cpuhp_state enum which can be found in include/linux/cpuhotplug.h.
352
353 Insert the state into the enum at the proper place so the ordering
354 requirements are fulfilled. The state constant has to be used for state
355 setup and removal.
356
357 Static allocation is also required when the state callbacks are not set
358 up at runtime and are part of the initializer of the CPU hotplug state
359 array in kernel/cpu.c.
360
361* Dynamic allocation
362
363 When there are no ordering requirements for the state callbacks then
364 dynamic allocation is the preferred method. The state number is allocated
365 by the setup function and returned to the caller on success.
366
367 Only the PREPARE and ONLINE sections provide a dynamic allocation
368 range. The STARTING section does not as most of the callbacks in that
369 section have explicit ordering requirements.
370
371Setup of a CPU hotplug state
372----------------------------
373
374The core code provides the following functions to setup a state:
375
376* cpuhp_setup_state(state, name, startup, teardown)
377* cpuhp_setup_state_nocalls(state, name, startup, teardown)
378* cpuhp_setup_state_cpuslocked(state, name, startup, teardown)
379* cpuhp_setup_state_nocalls_cpuslocked(state, name, startup, teardown)
380
381For cases where a driver or a subsystem has multiple instances and the same
382CPU hotplug state callbacks need to be invoked for each instance, the CPU
383hotplug core provides multi-instance support. The advantage over driver
384specific instance lists is that the instance related functions are fully
385serialized against CPU hotplug operations and provide the automatic
386invocations of the state callbacks on add and removal. To set up such a
387multi-instance state the following function is available:
388
389* cpuhp_setup_state_multi(state, name, startup, teardown)
390
391The @state argument is either a statically allocated state or one of the
392constants for dynamically allocated states - CPUHP_BP_PREPARE_DYN,
393CPUHP_AP_ONLINE_DYN - depending on the state section (PREPARE, ONLINE) for
394which a dynamic state should be allocated.
395
396The @name argument is used for sysfs output and for instrumentation. The
397naming convention is "subsys:mode" or "subsys/driver:mode",
398e.g. "perf:mode" or "perf/x86:mode". The common mode names are:
399
400======== =======================================================
401prepare For states in the PREPARE section
402
403dead For states in the PREPARE section which do not provide
404 a startup callback
405
406starting For states in the STARTING section
407
408dying For states in the STARTING section which do not provide
409 a startup callback
410
411online For states in the ONLINE section
412
413offline For states in the ONLINE section which do not provide
414 a startup callback
415======== =======================================================
416
417As the @name argument is only used for sysfs and instrumentation other mode
418descriptors can be used as well if they describe the nature of the state
419better than the common ones.
420
421Examples for @name arguments: "perf/online", "perf/x86:prepare",
422"RCU/tree:dying", "sched/waitempty"
423
424The @startup argument is a function pointer to the callback which should be
425invoked during a CPU online operation. If the usage site does not require a
426startup callback set the pointer to NULL.
427
428The @teardown argument is a function pointer to the callback which should
429be invoked during a CPU offline operation. If the usage site does not
430require a teardown callback set the pointer to NULL.
431
432The functions differ in the way how the installed callbacks are treated:
433
434 * cpuhp_setup_state_nocalls(), cpuhp_setup_state_nocalls_cpuslocked()
435 and cpuhp_setup_state_multi() only install the callbacks
436
437 * cpuhp_setup_state() and cpuhp_setup_state_cpuslocked() install the
438 callbacks and invoke the @startup callback (if not NULL) for all online
439 CPUs which have currently a state greater than the newly installed
440 state. Depending on the state section the callback is either invoked on
441 the current CPU (PREPARE section) or on each online CPU (ONLINE
442 section) in the context of the CPU's hotplug thread.
443
444 If a callback fails for CPU N then the teardown callback for CPU
445 0 .. N-1 is invoked to rollback the operation. The state setup fails,
446 the callbacks for the state are not installed and in case of dynamic
447 allocation the allocated state is freed.
448
449The state setup and the callback invocations are serialized against CPU
450hotplug operations. If the setup function has to be called from a CPU
451hotplug read locked region, then the _cpuslocked() variants have to be
452used. These functions cannot be used from within CPU hotplug callbacks.
453
454The function return values:
455 ======== ===================================================================
456 0 Statically allocated state was successfully set up
457
458 >0 Dynamically allocated state was successfully set up.
459
460 The returned number is the state number which was allocated. If
461 the state callbacks have to be removed later, e.g. module
462 removal, then this number has to be saved by the caller and used
463 as @state argument for the state remove function. For
464 multi-instance states the dynamically allocated state number is
465 also required as @state argument for the instance add/remove
466 operations.
467
468 <0 Operation failed
469 ======== ===================================================================
470
471Removal of a CPU hotplug state
472------------------------------
473
474To remove a previously set up state, the following functions are provided:
475
476* cpuhp_remove_state(state)
477* cpuhp_remove_state_nocalls(state)
478* cpuhp_remove_state_nocalls_cpuslocked(state)
479* cpuhp_remove_multi_state(state)
480
481The @state argument is either a statically allocated state or the state
482number which was allocated in the dynamic range by cpuhp_setup_state*(). If
483the state is in the dynamic range, then the state number is freed and
484available for dynamic allocation again.
485
486The functions differ in the way how the installed callbacks are treated:
487
488 * cpuhp_remove_state_nocalls(), cpuhp_remove_state_nocalls_cpuslocked()
489 and cpuhp_remove_multi_state() only remove the callbacks.
490
491 * cpuhp_remove_state() removes the callbacks and invokes the teardown
492 callback (if not NULL) for all online CPUs which have currently a state
493 greater than the removed state. Depending on the state section the
494 callback is either invoked on the current CPU (PREPARE section) or on
495 each online CPU (ONLINE section) in the context of the CPU's hotplug
496 thread.
497
498 In order to complete the removal, the teardown callback should not fail.
499
500The state removal and the callback invocations are serialized against CPU
501hotplug operations. If the remove function has to be called from a CPU
502hotplug read locked region, then the _cpuslocked() variants have to be
503used. These functions cannot be used from within CPU hotplug callbacks.
504
505If a multi-instance state is removed then the caller has to remove all
506instances first.
507
508Multi-Instance state instance management
509----------------------------------------
510
511Once the multi-instance state is set up, instances can be added to the
512state:
513
514 * cpuhp_state_add_instance(state, node)
515 * cpuhp_state_add_instance_nocalls(state, node)
516
517The @state argument is either a statically allocated state or the state
518number which was allocated in the dynamic range by cpuhp_setup_state_multi().
519
520The @node argument is a pointer to an hlist_node which is embedded in the
521instance's data structure. The pointer is handed to the multi-instance
522state callbacks and can be used by the callback to retrieve the instance
523via container_of().
524
525The functions differ in the way how the installed callbacks are treated:
526
527 * cpuhp_state_add_instance_nocalls() and only adds the instance to the
528 multi-instance state's node list.
529
530 * cpuhp_state_add_instance() adds the instance and invokes the startup
531 callback (if not NULL) associated with @state for all online CPUs which
532 have currently a state greater than @state. The callback is only
533 invoked for the to be added instance. Depending on the state section
534 the callback is either invoked on the current CPU (PREPARE section) or
535 on each online CPU (ONLINE section) in the context of the CPU's hotplug
536 thread.
537
538 If a callback fails for CPU N then the teardown callback for CPU
539 0 .. N-1 is invoked to rollback the operation, the function fails and
540 the instance is not added to the node list of the multi-instance state.
541
542To remove an instance from the state's node list these functions are
543available:
544
545 * cpuhp_state_remove_instance(state, node)
546 * cpuhp_state_remove_instance_nocalls(state, node)
547
548The arguments are the same as for the cpuhp_state_add_instance*()
549variants above.
550
551The functions differ in the way how the installed callbacks are treated:
552
553 * cpuhp_state_remove_instance_nocalls() only removes the instance from the
554 state's node list.
555
556 * cpuhp_state_remove_instance() removes the instance and invokes the
557 teardown callback (if not NULL) associated with @state for all online
558 CPUs which have currently a state greater than @state. The callback is
559 only invoked for the to be removed instance. Depending on the state
560 section the callback is either invoked on the current CPU (PREPARE
561 section) or on each online CPU (ONLINE section) in the context of the
562 CPU's hotplug thread.
563
564 In order to complete the removal, the teardown callback should not fail.
565
566The node list add/remove operations and the callback invocations are
567serialized against CPU hotplug operations. These functions cannot be used
568from within CPU hotplug callbacks and CPU hotplug read locked regions.
569
570Examples
571--------
572
573Setup and teardown a statically allocated state in the STARTING section for
574notifications on online and offline operations::
575
576 ret = cpuhp_setup_state(CPUHP_SUBSYS_STARTING, "subsys:starting", subsys_cpu_starting, subsys_cpu_dying);
577 if (ret < 0)
578 return ret;
579 ....
580 cpuhp_remove_state(CPUHP_SUBSYS_STARTING);
581
582Setup and teardown a dynamically allocated state in the ONLINE section
583for notifications on offline operations::
584
585 state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "subsys:offline", NULL, subsys_cpu_offline);
586 if (state < 0)
587 return state;
588 ....
589 cpuhp_remove_state(state);
590
591Setup and teardown a dynamically allocated state in the ONLINE section
592for notifications on online operations without invoking the callbacks::
593
594 state = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "subsys:online", subsys_cpu_online, NULL);
595 if (state < 0)
596 return state;
597 ....
598 cpuhp_remove_state_nocalls(state);
599
600Setup, use and teardown a dynamically allocated multi-instance state in the
601ONLINE section for notifications on online and offline operation::
602
603 state = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "subsys:online", subsys_cpu_online, subsys_cpu_offline);
604 if (state < 0)
605 return state;
606 ....
607 ret = cpuhp_state_add_instance(state, &inst1->node);
608 if (ret)
609 return ret;
610 ....
611 ret = cpuhp_state_add_instance(state, &inst2->node);
612 if (ret)
613 return ret;
614 ....
615 cpuhp_remove_instance(state, &inst1->node);
616 ....
617 cpuhp_remove_instance(state, &inst2->node);
618 ....
619 cpuhp_remove_multi_state(state);
620
621
622Testing of hotplug states
623=========================
624
625One way to verify whether a custom state is working as expected or not is to
626shutdown a CPU and then put it online again. It is also possible to put the CPU
627to certain state (for instance *CPUHP_AP_ONLINE*) and then go back to
628*CPUHP_ONLINE*. This would simulate an error one state after *CPUHP_AP_ONLINE*
629which would lead to rollback to the online state.
630
631All registered states are enumerated in ``/sys/devices/system/cpu/hotplug/states`` ::
632
633 $ tail /sys/devices/system/cpu/hotplug/states
634 138: mm/vmscan:online
635 139: mm/vmstat:online
636 140: lib/percpu_cnt:online
637 141: acpi/cpu-drv:online
638 142: base/cacheinfo:online
639 143: virtio/net:online
640 144: x86/mce:online
641 145: printk:online
642 168: sched:active
643 169: online
644
645To rollback CPU4 to ``lib/percpu_cnt:online`` and back online just issue::
646
647 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
648 169
649 $ echo 140 > /sys/devices/system/cpu/cpu4/hotplug/target
650 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
651 140
652
653It is important to note that the teardown callback of state 140 have been
654invoked. And now get back online::
655
656 $ echo 169 > /sys/devices/system/cpu/cpu4/hotplug/target
657 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
658 169
659
660With trace events enabled, the individual steps are visible, too::
661
662 # TASK-PID CPU# TIMESTAMP FUNCTION
663 # | | | | |
664 bash-394 [001] 22.976: cpuhp_enter: cpu: 0004 target: 140 step: 169 (cpuhp_kick_ap_work)
665 cpuhp/4-31 [004] 22.977: cpuhp_enter: cpu: 0004 target: 140 step: 168 (sched_cpu_deactivate)
666 cpuhp/4-31 [004] 22.990: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
667 cpuhp/4-31 [004] 22.991: cpuhp_enter: cpu: 0004 target: 140 step: 144 (mce_cpu_pre_down)
668 cpuhp/4-31 [004] 22.992: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
669 cpuhp/4-31 [004] 22.993: cpuhp_multi_enter: cpu: 0004 target: 140 step: 143 (virtnet_cpu_down_prep)
670 cpuhp/4-31 [004] 22.994: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
671 cpuhp/4-31 [004] 22.995: cpuhp_enter: cpu: 0004 target: 140 step: 142 (cacheinfo_cpu_pre_down)
672 cpuhp/4-31 [004] 22.996: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
673 bash-394 [001] 22.997: cpuhp_exit: cpu: 0004 state: 140 step: 169 ret: 0
674 bash-394 [005] 95.540: cpuhp_enter: cpu: 0004 target: 169 step: 140 (cpuhp_kick_ap_work)
675 cpuhp/4-31 [004] 95.541: cpuhp_enter: cpu: 0004 target: 169 step: 141 (acpi_soft_cpu_online)
676 cpuhp/4-31 [004] 95.542: cpuhp_exit: cpu: 0004 state: 141 step: 141 ret: 0
677 cpuhp/4-31 [004] 95.543: cpuhp_enter: cpu: 0004 target: 169 step: 142 (cacheinfo_cpu_online)
678 cpuhp/4-31 [004] 95.544: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
679 cpuhp/4-31 [004] 95.545: cpuhp_multi_enter: cpu: 0004 target: 169 step: 143 (virtnet_cpu_online)
680 cpuhp/4-31 [004] 95.546: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
681 cpuhp/4-31 [004] 95.547: cpuhp_enter: cpu: 0004 target: 169 step: 144 (mce_cpu_online)
682 cpuhp/4-31 [004] 95.548: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
683 cpuhp/4-31 [004] 95.549: cpuhp_enter: cpu: 0004 target: 169 step: 145 (console_cpu_notify)
684 cpuhp/4-31 [004] 95.550: cpuhp_exit: cpu: 0004 state: 145 step: 145 ret: 0
685 cpuhp/4-31 [004] 95.551: cpuhp_enter: cpu: 0004 target: 169 step: 168 (sched_cpu_activate)
686 cpuhp/4-31 [004] 95.552: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
687 bash-394 [005] 95.553: cpuhp_exit: cpu: 0004 state: 169 step: 140 ret: 0
688
689As it an be seen, CPU4 went down until timestamp 22.996 and then back up until
69095.552. All invoked callbacks including their return codes are visible in the
691trace.
692
693Architecture's requirements
694===========================
695
696The following functions and configurations are required:
697
698``CONFIG_HOTPLUG_CPU``
699 This entry needs to be enabled in Kconfig
700
701``__cpu_up()``
702 Arch interface to bring up a CPU
703
704``__cpu_disable()``
705 Arch interface to shutdown a CPU, no more interrupts can be handled by the
706 kernel after the routine returns. This includes the shutdown of the timer.
707
708``__cpu_die()``
709 This actually supposed to ensure death of the CPU. Actually look at some
710 example code in other arch that implement CPU hotplug. The processor is taken
711 down from the ``idle()`` loop for that specific architecture. ``__cpu_die()``
712 typically waits for some per_cpu state to be set, to ensure the processor dead
713 routine is called to be sure positively.
714
715User Space Notification
716=======================
717
718After CPU successfully onlined or offline udev events are sent. A udev rule like::
719
720 SUBSYSTEM=="cpu", DRIVERS=="processor", DEVPATH=="/devices/system/cpu/*", RUN+="the_hotplug_receiver.sh"
721
722will receive all events. A script like::
723
724 #!/bin/sh
725
726 if [ "${ACTION}" = "offline" ]
727 then
728 echo "CPU ${DEVPATH##*/} offline"
729
730 elif [ "${ACTION}" = "online" ]
731 then
732 echo "CPU ${DEVPATH##*/} online"
733
734 fi
735
736can process the event further.
737
738When changes to the CPUs in the system occur, the sysfs file
739/sys/devices/system/cpu/crash_hotplug contains '1' if the kernel
740updates the kdump capture kernel list of CPUs itself (via elfcorehdr and
741other relevant kexec segment), or '0' if userspace must update the kdump
742capture kernel list of CPUs.
743
744The availability depends on the CONFIG_HOTPLUG_CPU kernel configuration
745option.
746
747To skip userspace processing of CPU hot un/plug events for kdump
748(i.e. the unload-then-reload to obtain a current list of CPUs), this sysfs
749file can be used in a udev rule as follows:
750
751 SUBSYSTEM=="cpu", ATTRS{crash_hotplug}=="1", GOTO="kdump_reload_end"
752
753For a CPU hot un/plug event, if the architecture supports kernel updates
754of the elfcorehdr (which contains the list of CPUs) and other relevant
755kexec segments, then the rule skips the unload-then-reload of the kdump
756capture kernel.
757
758Kernel Inline Documentations Reference
759======================================
760
761.. kernel-doc:: include/linux/cpuhotplug.h
1=========================
2CPU hotplug in the Kernel
3=========================
4
5:Date: September, 2021
6:Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
7 Rusty Russell <rusty@rustcorp.com.au>,
8 Srivatsa Vaddagiri <vatsa@in.ibm.com>,
9 Ashok Raj <ashok.raj@intel.com>,
10 Joel Schopp <jschopp@austin.ibm.com>,
11 Thomas Gleixner <tglx@linutronix.de>
12
13Introduction
14============
15
16Modern advances in system architectures have introduced advanced error
17reporting and correction capabilities in processors. There are couple OEMS that
18support NUMA hardware which are hot pluggable as well, where physical node
19insertion and removal require support for CPU hotplug.
20
21Such advances require CPUs available to a kernel to be removed either for
22provisioning reasons, or for RAS purposes to keep an offending CPU off
23system execution path. Hence the need for CPU hotplug support in the
24Linux kernel.
25
26A more novel use of CPU-hotplug support is its use today in suspend resume
27support for SMP. Dual-core and HT support makes even a laptop run SMP kernels
28which didn't support these methods.
29
30
31Command Line Switches
32=====================
33``maxcpus=n``
34 Restrict boot time CPUs to *n*. Say if you have four CPUs, using
35 ``maxcpus=2`` will only boot two. You can choose to bring the
36 other CPUs later online.
37
38``nr_cpus=n``
39 Restrict the total amount of CPUs the kernel will support. If the number
40 supplied here is lower than the number of physically available CPUs, then
41 those CPUs can not be brought online later.
42
43``additional_cpus=n``
44 Use this to limit hotpluggable CPUs. This option sets
45 ``cpu_possible_mask = cpu_present_mask + additional_cpus``
46
47 This option is limited to the IA64 architecture.
48
49``possible_cpus=n``
50 This option sets ``possible_cpus`` bits in ``cpu_possible_mask``.
51
52 This option is limited to the X86 and S390 architecture.
53
54``cpu0_hotplug``
55 Allow to shutdown CPU0.
56
57 This option is limited to the X86 architecture.
58
59CPU maps
60========
61
62``cpu_possible_mask``
63 Bitmap of possible CPUs that can ever be available in the
64 system. This is used to allocate some boot time memory for per_cpu variables
65 that aren't designed to grow/shrink as CPUs are made available or removed.
66 Once set during boot time discovery phase, the map is static, i.e no bits
67 are added or removed anytime. Trimming it accurately for your system needs
68 upfront can save some boot time memory.
69
70``cpu_online_mask``
71 Bitmap of all CPUs currently online. Its set in ``__cpu_up()``
72 after a CPU is available for kernel scheduling and ready to receive
73 interrupts from devices. Its cleared when a CPU is brought down using
74 ``__cpu_disable()``, before which all OS services including interrupts are
75 migrated to another target CPU.
76
77``cpu_present_mask``
78 Bitmap of CPUs currently present in the system. Not all
79 of them may be online. When physical hotplug is processed by the relevant
80 subsystem (e.g ACPI) can change and new bit either be added or removed
81 from the map depending on the event is hot-add/hot-remove. There are currently
82 no locking rules as of now. Typical usage is to init topology during boot,
83 at which time hotplug is disabled.
84
85You really don't need to manipulate any of the system CPU maps. They should
86be read-only for most use. When setting up per-cpu resources almost always use
87``cpu_possible_mask`` or ``for_each_possible_cpu()`` to iterate. To macro
88``for_each_cpu()`` can be used to iterate over a custom CPU mask.
89
90Never use anything other than ``cpumask_t`` to represent bitmap of CPUs.
91
92
93Using CPU hotplug
94=================
95
96The kernel option *CONFIG_HOTPLUG_CPU* needs to be enabled. It is currently
97available on multiple architectures including ARM, MIPS, PowerPC and X86. The
98configuration is done via the sysfs interface::
99
100 $ ls -lh /sys/devices/system/cpu
101 total 0
102 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu0
103 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu1
104 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu2
105 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu3
106 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu4
107 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu5
108 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu6
109 drwxr-xr-x 9 root root 0 Dec 21 16:33 cpu7
110 drwxr-xr-x 2 root root 0 Dec 21 16:33 hotplug
111 -r--r--r-- 1 root root 4.0K Dec 21 16:33 offline
112 -r--r--r-- 1 root root 4.0K Dec 21 16:33 online
113 -r--r--r-- 1 root root 4.0K Dec 21 16:33 possible
114 -r--r--r-- 1 root root 4.0K Dec 21 16:33 present
115
116The files *offline*, *online*, *possible*, *present* represent the CPU masks.
117Each CPU folder contains an *online* file which controls the logical on (1) and
118off (0) state. To logically shutdown CPU4::
119
120 $ echo 0 > /sys/devices/system/cpu/cpu4/online
121 smpboot: CPU 4 is now offline
122
123Once the CPU is shutdown, it will be removed from */proc/interrupts*,
124*/proc/cpuinfo* and should also not be shown visible by the *top* command. To
125bring CPU4 back online::
126
127 $ echo 1 > /sys/devices/system/cpu/cpu4/online
128 smpboot: Booting Node 0 Processor 4 APIC 0x1
129
130The CPU is usable again. This should work on all CPUs. CPU0 is often special
131and excluded from CPU hotplug. On X86 the kernel option
132*CONFIG_BOOTPARAM_HOTPLUG_CPU0* has to be enabled in order to be able to
133shutdown CPU0. Alternatively the kernel command option *cpu0_hotplug* can be
134used. Some known dependencies of CPU0:
135
136* Resume from hibernate/suspend. Hibernate/suspend will fail if CPU0 is offline.
137* PIC interrupts. CPU0 can't be removed if a PIC interrupt is detected.
138
139Please let Fenghua Yu <fenghua.yu@intel.com> know if you find any dependencies
140on CPU0.
141
142The CPU hotplug coordination
143============================
144
145The offline case
146----------------
147
148Once a CPU has been logically shutdown the teardown callbacks of registered
149hotplug states will be invoked, starting with ``CPUHP_ONLINE`` and terminating
150at state ``CPUHP_OFFLINE``. This includes:
151
152* If tasks are frozen due to a suspend operation then *cpuhp_tasks_frozen*
153 will be set to true.
154* All processes are migrated away from this outgoing CPU to new CPUs.
155 The new CPU is chosen from each process' current cpuset, which may be
156 a subset of all online CPUs.
157* All interrupts targeted to this CPU are migrated to a new CPU
158* timers are also migrated to a new CPU
159* Once all services are migrated, kernel calls an arch specific routine
160 ``__cpu_disable()`` to perform arch specific cleanup.
161
162
163The CPU hotplug API
164===================
165
166CPU hotplug state machine
167-------------------------
168
169CPU hotplug uses a trivial state machine with a linear state space from
170CPUHP_OFFLINE to CPUHP_ONLINE. Each state has a startup and a teardown
171callback.
172
173When a CPU is onlined, the startup callbacks are invoked sequentially until
174the state CPUHP_ONLINE is reached. They can also be invoked when the
175callbacks of a state are set up or an instance is added to a multi-instance
176state.
177
178When a CPU is offlined the teardown callbacks are invoked in the reverse
179order sequentially until the state CPUHP_OFFLINE is reached. They can also
180be invoked when the callbacks of a state are removed or an instance is
181removed from a multi-instance state.
182
183If a usage site requires only a callback in one direction of the hotplug
184operations (CPU online or CPU offline) then the other not-required callback
185can be set to NULL when the state is set up.
186
187The state space is divided into three sections:
188
189* The PREPARE section
190
191 The PREPARE section covers the state space from CPUHP_OFFLINE to
192 CPUHP_BRINGUP_CPU.
193
194 The startup callbacks in this section are invoked before the CPU is
195 started during a CPU online operation. The teardown callbacks are invoked
196 after the CPU has become dysfunctional during a CPU offline operation.
197
198 The callbacks are invoked on a control CPU as they can't obviously run on
199 the hotplugged CPU which is either not yet started or has become
200 dysfunctional already.
201
202 The startup callbacks are used to setup resources which are required to
203 bring a CPU successfully online. The teardown callbacks are used to free
204 resources or to move pending work to an online CPU after the hotplugged
205 CPU became dysfunctional.
206
207 The startup callbacks are allowed to fail. If a callback fails, the CPU
208 online operation is aborted and the CPU is brought down to the previous
209 state (usually CPUHP_OFFLINE) again.
210
211 The teardown callbacks in this section are not allowed to fail.
212
213* The STARTING section
214
215 The STARTING section covers the state space between CPUHP_BRINGUP_CPU + 1
216 and CPUHP_AP_ONLINE.
217
218 The startup callbacks in this section are invoked on the hotplugged CPU
219 with interrupts disabled during a CPU online operation in the early CPU
220 setup code. The teardown callbacks are invoked with interrupts disabled
221 on the hotplugged CPU during a CPU offline operation shortly before the
222 CPU is completely shut down.
223
224 The callbacks in this section are not allowed to fail.
225
226 The callbacks are used for low level hardware initialization/shutdown and
227 for core subsystems.
228
229* The ONLINE section
230
231 The ONLINE section covers the state space between CPUHP_AP_ONLINE + 1 and
232 CPUHP_ONLINE.
233
234 The startup callbacks in this section are invoked on the hotplugged CPU
235 during a CPU online operation. The teardown callbacks are invoked on the
236 hotplugged CPU during a CPU offline operation.
237
238 The callbacks are invoked in the context of the per CPU hotplug thread,
239 which is pinned on the hotplugged CPU. The callbacks are invoked with
240 interrupts and preemption enabled.
241
242 The callbacks are allowed to fail. When a callback fails the hotplug
243 operation is aborted and the CPU is brought back to the previous state.
244
245CPU online/offline operations
246-----------------------------
247
248A successful online operation looks like this::
249
250 [CPUHP_OFFLINE]
251 [CPUHP_OFFLINE + 1]->startup() -> success
252 [CPUHP_OFFLINE + 2]->startup() -> success
253 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
254 ...
255 [CPUHP_BRINGUP_CPU]->startup() -> success
256 === End of PREPARE section
257 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
258 ...
259 [CPUHP_AP_ONLINE]->startup() -> success
260 === End of STARTUP section
261 [CPUHP_AP_ONLINE + 1]->startup() -> success
262 ...
263 [CPUHP_ONLINE - 1]->startup() -> success
264 [CPUHP_ONLINE]
265
266A successful offline operation looks like this::
267
268 [CPUHP_ONLINE]
269 [CPUHP_ONLINE - 1]->teardown() -> success
270 ...
271 [CPUHP_AP_ONLINE + 1]->teardown() -> success
272 === Start of STARTUP section
273 [CPUHP_AP_ONLINE]->teardown() -> success
274 ...
275 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
276 ...
277 === Start of PREPARE section
278 [CPUHP_BRINGUP_CPU]->teardown()
279 [CPUHP_OFFLINE + 3]->teardown()
280 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
281 [CPUHP_OFFLINE + 1]->teardown()
282 [CPUHP_OFFLINE]
283
284A failed online operation looks like this::
285
286 [CPUHP_OFFLINE]
287 [CPUHP_OFFLINE + 1]->startup() -> success
288 [CPUHP_OFFLINE + 2]->startup() -> success
289 [CPUHP_OFFLINE + 3] -> skipped because startup == NULL
290 ...
291 [CPUHP_BRINGUP_CPU]->startup() -> success
292 === End of PREPARE section
293 [CPUHP_BRINGUP_CPU + 1]->startup() -> success
294 ...
295 [CPUHP_AP_ONLINE]->startup() -> success
296 === End of STARTUP section
297 [CPUHP_AP_ONLINE + 1]->startup() -> success
298 ---
299 [CPUHP_AP_ONLINE + N]->startup() -> fail
300 [CPUHP_AP_ONLINE + (N - 1)]->teardown()
301 ...
302 [CPUHP_AP_ONLINE + 1]->teardown()
303 === Start of STARTUP section
304 [CPUHP_AP_ONLINE]->teardown()
305 ...
306 [CPUHP_BRINGUP_ONLINE - 1]->teardown()
307 ...
308 === Start of PREPARE section
309 [CPUHP_BRINGUP_CPU]->teardown()
310 [CPUHP_OFFLINE + 3]->teardown()
311 [CPUHP_OFFLINE + 2] -> skipped because teardown == NULL
312 [CPUHP_OFFLINE + 1]->teardown()
313 [CPUHP_OFFLINE]
314
315A failed offline operation looks like this::
316
317 [CPUHP_ONLINE]
318 [CPUHP_ONLINE - 1]->teardown() -> success
319 ...
320 [CPUHP_ONLINE - N]->teardown() -> fail
321 [CPUHP_ONLINE - (N - 1)]->startup()
322 ...
323 [CPUHP_ONLINE - 1]->startup()
324 [CPUHP_ONLINE]
325
326Recursive failures cannot be handled sensibly. Look at the following
327example of a recursive fail due to a failed offline operation: ::
328
329 [CPUHP_ONLINE]
330 [CPUHP_ONLINE - 1]->teardown() -> success
331 ...
332 [CPUHP_ONLINE - N]->teardown() -> fail
333 [CPUHP_ONLINE - (N - 1)]->startup() -> success
334 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
335
336The CPU hotplug state machine stops right here and does not try to go back
337down again because that would likely result in an endless loop::
338
339 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
340 [CPUHP_ONLINE - N]->teardown() -> fail
341 [CPUHP_ONLINE - (N - 1)]->startup() -> success
342 [CPUHP_ONLINE - (N - 2)]->startup() -> fail
343 [CPUHP_ONLINE - (N - 1)]->teardown() -> success
344 [CPUHP_ONLINE - N]->teardown() -> fail
345
346Lather, rinse and repeat. In this case the CPU left in state::
347
348 [CPUHP_ONLINE - (N - 1)]
349
350which at least lets the system make progress and gives the user a chance to
351debug or even resolve the situation.
352
353Allocating a state
354------------------
355
356There are two ways to allocate a CPU hotplug state:
357
358* Static allocation
359
360 Static allocation has to be used when the subsystem or driver has
361 ordering requirements versus other CPU hotplug states. E.g. the PERF core
362 startup callback has to be invoked before the PERF driver startup
363 callbacks during a CPU online operation. During a CPU offline operation
364 the driver teardown callbacks have to be invoked before the core teardown
365 callback. The statically allocated states are described by constants in
366 the cpuhp_state enum which can be found in include/linux/cpuhotplug.h.
367
368 Insert the state into the enum at the proper place so the ordering
369 requirements are fulfilled. The state constant has to be used for state
370 setup and removal.
371
372 Static allocation is also required when the state callbacks are not set
373 up at runtime and are part of the initializer of the CPU hotplug state
374 array in kernel/cpu.c.
375
376* Dynamic allocation
377
378 When there are no ordering requirements for the state callbacks then
379 dynamic allocation is the preferred method. The state number is allocated
380 by the setup function and returned to the caller on success.
381
382 Only the PREPARE and ONLINE sections provide a dynamic allocation
383 range. The STARTING section does not as most of the callbacks in that
384 section have explicit ordering requirements.
385
386Setup of a CPU hotplug state
387----------------------------
388
389The core code provides the following functions to setup a state:
390
391* cpuhp_setup_state(state, name, startup, teardown)
392* cpuhp_setup_state_nocalls(state, name, startup, teardown)
393* cpuhp_setup_state_cpuslocked(state, name, startup, teardown)
394* cpuhp_setup_state_nocalls_cpuslocked(state, name, startup, teardown)
395
396For cases where a driver or a subsystem has multiple instances and the same
397CPU hotplug state callbacks need to be invoked for each instance, the CPU
398hotplug core provides multi-instance support. The advantage over driver
399specific instance lists is that the instance related functions are fully
400serialized against CPU hotplug operations and provide the automatic
401invocations of the state callbacks on add and removal. To set up such a
402multi-instance state the following function is available:
403
404* cpuhp_setup_state_multi(state, name, startup, teardown)
405
406The @state argument is either a statically allocated state or one of the
407constants for dynamically allocated states - CPUHP_PREPARE_DYN,
408CPUHP_ONLINE_DYN - depending on the state section (PREPARE, ONLINE) for
409which a dynamic state should be allocated.
410
411The @name argument is used for sysfs output and for instrumentation. The
412naming convention is "subsys:mode" or "subsys/driver:mode",
413e.g. "perf:mode" or "perf/x86:mode". The common mode names are:
414
415======== =======================================================
416prepare For states in the PREPARE section
417
418dead For states in the PREPARE section which do not provide
419 a startup callback
420
421starting For states in the STARTING section
422
423dying For states in the STARTING section which do not provide
424 a startup callback
425
426online For states in the ONLINE section
427
428offline For states in the ONLINE section which do not provide
429 a startup callback
430======== =======================================================
431
432As the @name argument is only used for sysfs and instrumentation other mode
433descriptors can be used as well if they describe the nature of the state
434better than the common ones.
435
436Examples for @name arguments: "perf/online", "perf/x86:prepare",
437"RCU/tree:dying", "sched/waitempty"
438
439The @startup argument is a function pointer to the callback which should be
440invoked during a CPU online operation. If the usage site does not require a
441startup callback set the pointer to NULL.
442
443The @teardown argument is a function pointer to the callback which should
444be invoked during a CPU offline operation. If the usage site does not
445require a teardown callback set the pointer to NULL.
446
447The functions differ in the way how the installed callbacks are treated:
448
449 * cpuhp_setup_state_nocalls(), cpuhp_setup_state_nocalls_cpuslocked()
450 and cpuhp_setup_state_multi() only install the callbacks
451
452 * cpuhp_setup_state() and cpuhp_setup_state_cpuslocked() install the
453 callbacks and invoke the @startup callback (if not NULL) for all online
454 CPUs which have currently a state greater than the newly installed
455 state. Depending on the state section the callback is either invoked on
456 the current CPU (PREPARE section) or on each online CPU (ONLINE
457 section) in the context of the CPU's hotplug thread.
458
459 If a callback fails for CPU N then the teardown callback for CPU
460 0 .. N-1 is invoked to rollback the operation. The state setup fails,
461 the callbacks for the state are not installed and in case of dynamic
462 allocation the allocated state is freed.
463
464The state setup and the callback invocations are serialized against CPU
465hotplug operations. If the setup function has to be called from a CPU
466hotplug read locked region, then the _cpuslocked() variants have to be
467used. These functions cannot be used from within CPU hotplug callbacks.
468
469The function return values:
470 ======== ===================================================================
471 0 Statically allocated state was successfully set up
472
473 >0 Dynamically allocated state was successfully set up.
474
475 The returned number is the state number which was allocated. If
476 the state callbacks have to be removed later, e.g. module
477 removal, then this number has to be saved by the caller and used
478 as @state argument for the state remove function. For
479 multi-instance states the dynamically allocated state number is
480 also required as @state argument for the instance add/remove
481 operations.
482
483 <0 Operation failed
484 ======== ===================================================================
485
486Removal of a CPU hotplug state
487------------------------------
488
489To remove a previously set up state, the following functions are provided:
490
491* cpuhp_remove_state(state)
492* cpuhp_remove_state_nocalls(state)
493* cpuhp_remove_state_nocalls_cpuslocked(state)
494* cpuhp_remove_multi_state(state)
495
496The @state argument is either a statically allocated state or the state
497number which was allocated in the dynamic range by cpuhp_setup_state*(). If
498the state is in the dynamic range, then the state number is freed and
499available for dynamic allocation again.
500
501The functions differ in the way how the installed callbacks are treated:
502
503 * cpuhp_remove_state_nocalls(), cpuhp_remove_state_nocalls_cpuslocked()
504 and cpuhp_remove_multi_state() only remove the callbacks.
505
506 * cpuhp_remove_state() removes the callbacks and invokes the teardown
507 callback (if not NULL) for all online CPUs which have currently a state
508 greater than the removed state. Depending on the state section the
509 callback is either invoked on the current CPU (PREPARE section) or on
510 each online CPU (ONLINE section) in the context of the CPU's hotplug
511 thread.
512
513 In order to complete the removal, the teardown callback should not fail.
514
515The state removal and the callback invocations are serialized against CPU
516hotplug operations. If the remove function has to be called from a CPU
517hotplug read locked region, then the _cpuslocked() variants have to be
518used. These functions cannot be used from within CPU hotplug callbacks.
519
520If a multi-instance state is removed then the caller has to remove all
521instances first.
522
523Multi-Instance state instance management
524----------------------------------------
525
526Once the multi-instance state is set up, instances can be added to the
527state:
528
529 * cpuhp_state_add_instance(state, node)
530 * cpuhp_state_add_instance_nocalls(state, node)
531
532The @state argument is either a statically allocated state or the state
533number which was allocated in the dynamic range by cpuhp_setup_state_multi().
534
535The @node argument is a pointer to an hlist_node which is embedded in the
536instance's data structure. The pointer is handed to the multi-instance
537state callbacks and can be used by the callback to retrieve the instance
538via container_of().
539
540The functions differ in the way how the installed callbacks are treated:
541
542 * cpuhp_state_add_instance_nocalls() and only adds the instance to the
543 multi-instance state's node list.
544
545 * cpuhp_state_add_instance() adds the instance and invokes the startup
546 callback (if not NULL) associated with @state for all online CPUs which
547 have currently a state greater than @state. The callback is only
548 invoked for the to be added instance. Depending on the state section
549 the callback is either invoked on the current CPU (PREPARE section) or
550 on each online CPU (ONLINE section) in the context of the CPU's hotplug
551 thread.
552
553 If a callback fails for CPU N then the teardown callback for CPU
554 0 .. N-1 is invoked to rollback the operation, the function fails and
555 the instance is not added to the node list of the multi-instance state.
556
557To remove an instance from the state's node list these functions are
558available:
559
560 * cpuhp_state_remove_instance(state, node)
561 * cpuhp_state_remove_instance_nocalls(state, node)
562
563The arguments are the same as for the cpuhp_state_add_instance*()
564variants above.
565
566The functions differ in the way how the installed callbacks are treated:
567
568 * cpuhp_state_remove_instance_nocalls() only removes the instance from the
569 state's node list.
570
571 * cpuhp_state_remove_instance() removes the instance and invokes the
572 teardown callback (if not NULL) associated with @state for all online
573 CPUs which have currently a state greater than @state. The callback is
574 only invoked for the to be removed instance. Depending on the state
575 section the callback is either invoked on the current CPU (PREPARE
576 section) or on each online CPU (ONLINE section) in the context of the
577 CPU's hotplug thread.
578
579 In order to complete the removal, the teardown callback should not fail.
580
581The node list add/remove operations and the callback invocations are
582serialized against CPU hotplug operations. These functions cannot be used
583from within CPU hotplug callbacks and CPU hotplug read locked regions.
584
585Examples
586--------
587
588Setup and teardown a statically allocated state in the STARTING section for
589notifications on online and offline operations::
590
591 ret = cpuhp_setup_state(CPUHP_SUBSYS_STARTING, "subsys:starting", subsys_cpu_starting, subsys_cpu_dying);
592 if (ret < 0)
593 return ret;
594 ....
595 cpuhp_remove_state(CPUHP_SUBSYS_STARTING);
596
597Setup and teardown a dynamically allocated state in the ONLINE section
598for notifications on offline operations::
599
600 state = cpuhp_setup_state(CPUHP_ONLINE_DYN, "subsys:offline", NULL, subsys_cpu_offline);
601 if (state < 0)
602 return state;
603 ....
604 cpuhp_remove_state(state);
605
606Setup and teardown a dynamically allocated state in the ONLINE section
607for notifications on online operations without invoking the callbacks::
608
609 state = cpuhp_setup_state_nocalls(CPUHP_ONLINE_DYN, "subsys:online", subsys_cpu_online, NULL);
610 if (state < 0)
611 return state;
612 ....
613 cpuhp_remove_state_nocalls(state);
614
615Setup, use and teardown a dynamically allocated multi-instance state in the
616ONLINE section for notifications on online and offline operation::
617
618 state = cpuhp_setup_state_multi(CPUHP_ONLINE_DYN, "subsys:online", subsys_cpu_online, subsys_cpu_offline);
619 if (state < 0)
620 return state;
621 ....
622 ret = cpuhp_state_add_instance(state, &inst1->node);
623 if (ret)
624 return ret;
625 ....
626 ret = cpuhp_state_add_instance(state, &inst2->node);
627 if (ret)
628 return ret;
629 ....
630 cpuhp_remove_instance(state, &inst1->node);
631 ....
632 cpuhp_remove_instance(state, &inst2->node);
633 ....
634 remove_multi_state(state);
635
636
637Testing of hotplug states
638=========================
639
640One way to verify whether a custom state is working as expected or not is to
641shutdown a CPU and then put it online again. It is also possible to put the CPU
642to certain state (for instance *CPUHP_AP_ONLINE*) and then go back to
643*CPUHP_ONLINE*. This would simulate an error one state after *CPUHP_AP_ONLINE*
644which would lead to rollback to the online state.
645
646All registered states are enumerated in ``/sys/devices/system/cpu/hotplug/states`` ::
647
648 $ tail /sys/devices/system/cpu/hotplug/states
649 138: mm/vmscan:online
650 139: mm/vmstat:online
651 140: lib/percpu_cnt:online
652 141: acpi/cpu-drv:online
653 142: base/cacheinfo:online
654 143: virtio/net:online
655 144: x86/mce:online
656 145: printk:online
657 168: sched:active
658 169: online
659
660To rollback CPU4 to ``lib/percpu_cnt:online`` and back online just issue::
661
662 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
663 169
664 $ echo 140 > /sys/devices/system/cpu/cpu4/hotplug/target
665 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
666 140
667
668It is important to note that the teardown callback of state 140 have been
669invoked. And now get back online::
670
671 $ echo 169 > /sys/devices/system/cpu/cpu4/hotplug/target
672 $ cat /sys/devices/system/cpu/cpu4/hotplug/state
673 169
674
675With trace events enabled, the individual steps are visible, too::
676
677 # TASK-PID CPU# TIMESTAMP FUNCTION
678 # | | | | |
679 bash-394 [001] 22.976: cpuhp_enter: cpu: 0004 target: 140 step: 169 (cpuhp_kick_ap_work)
680 cpuhp/4-31 [004] 22.977: cpuhp_enter: cpu: 0004 target: 140 step: 168 (sched_cpu_deactivate)
681 cpuhp/4-31 [004] 22.990: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
682 cpuhp/4-31 [004] 22.991: cpuhp_enter: cpu: 0004 target: 140 step: 144 (mce_cpu_pre_down)
683 cpuhp/4-31 [004] 22.992: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
684 cpuhp/4-31 [004] 22.993: cpuhp_multi_enter: cpu: 0004 target: 140 step: 143 (virtnet_cpu_down_prep)
685 cpuhp/4-31 [004] 22.994: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
686 cpuhp/4-31 [004] 22.995: cpuhp_enter: cpu: 0004 target: 140 step: 142 (cacheinfo_cpu_pre_down)
687 cpuhp/4-31 [004] 22.996: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
688 bash-394 [001] 22.997: cpuhp_exit: cpu: 0004 state: 140 step: 169 ret: 0
689 bash-394 [005] 95.540: cpuhp_enter: cpu: 0004 target: 169 step: 140 (cpuhp_kick_ap_work)
690 cpuhp/4-31 [004] 95.541: cpuhp_enter: cpu: 0004 target: 169 step: 141 (acpi_soft_cpu_online)
691 cpuhp/4-31 [004] 95.542: cpuhp_exit: cpu: 0004 state: 141 step: 141 ret: 0
692 cpuhp/4-31 [004] 95.543: cpuhp_enter: cpu: 0004 target: 169 step: 142 (cacheinfo_cpu_online)
693 cpuhp/4-31 [004] 95.544: cpuhp_exit: cpu: 0004 state: 142 step: 142 ret: 0
694 cpuhp/4-31 [004] 95.545: cpuhp_multi_enter: cpu: 0004 target: 169 step: 143 (virtnet_cpu_online)
695 cpuhp/4-31 [004] 95.546: cpuhp_exit: cpu: 0004 state: 143 step: 143 ret: 0
696 cpuhp/4-31 [004] 95.547: cpuhp_enter: cpu: 0004 target: 169 step: 144 (mce_cpu_online)
697 cpuhp/4-31 [004] 95.548: cpuhp_exit: cpu: 0004 state: 144 step: 144 ret: 0
698 cpuhp/4-31 [004] 95.549: cpuhp_enter: cpu: 0004 target: 169 step: 145 (console_cpu_notify)
699 cpuhp/4-31 [004] 95.550: cpuhp_exit: cpu: 0004 state: 145 step: 145 ret: 0
700 cpuhp/4-31 [004] 95.551: cpuhp_enter: cpu: 0004 target: 169 step: 168 (sched_cpu_activate)
701 cpuhp/4-31 [004] 95.552: cpuhp_exit: cpu: 0004 state: 168 step: 168 ret: 0
702 bash-394 [005] 95.553: cpuhp_exit: cpu: 0004 state: 169 step: 140 ret: 0
703
704As it an be seen, CPU4 went down until timestamp 22.996 and then back up until
70595.552. All invoked callbacks including their return codes are visible in the
706trace.
707
708Architecture's requirements
709===========================
710
711The following functions and configurations are required:
712
713``CONFIG_HOTPLUG_CPU``
714 This entry needs to be enabled in Kconfig
715
716``__cpu_up()``
717 Arch interface to bring up a CPU
718
719``__cpu_disable()``
720 Arch interface to shutdown a CPU, no more interrupts can be handled by the
721 kernel after the routine returns. This includes the shutdown of the timer.
722
723``__cpu_die()``
724 This actually supposed to ensure death of the CPU. Actually look at some
725 example code in other arch that implement CPU hotplug. The processor is taken
726 down from the ``idle()`` loop for that specific architecture. ``__cpu_die()``
727 typically waits for some per_cpu state to be set, to ensure the processor dead
728 routine is called to be sure positively.
729
730User Space Notification
731=======================
732
733After CPU successfully onlined or offline udev events are sent. A udev rule like::
734
735 SUBSYSTEM=="cpu", DRIVERS=="processor", DEVPATH=="/devices/system/cpu/*", RUN+="the_hotplug_receiver.sh"
736
737will receive all events. A script like::
738
739 #!/bin/sh
740
741 if [ "${ACTION}" = "offline" ]
742 then
743 echo "CPU ${DEVPATH##*/} offline"
744
745 elif [ "${ACTION}" = "online" ]
746 then
747 echo "CPU ${DEVPATH##*/} online"
748
749 fi
750
751can process the event further.
752
753Kernel Inline Documentations Reference
754======================================
755
756.. kernel-doc:: include/linux/cpuhotplug.h