Loading...
1=================================================
2Using kgdb, kdb and the kernel debugger internals
3=================================================
4
5:Author: Jason Wessel
6
7Introduction
8============
9
10The kernel has two different debugger front ends (kdb and kgdb) which
11interface to the debug core. It is possible to use either of the
12debugger front ends and dynamically transition between them if you
13configure the kernel properly at compile and runtime.
14
15Kdb is simplistic shell-style interface which you can use on a system
16console with a keyboard or serial console. You can use it to inspect
17memory, registers, process lists, dmesg, and even set breakpoints to
18stop in a certain location. Kdb is not a source level debugger, although
19you can set breakpoints and execute some basic kernel run control. Kdb
20is mainly aimed at doing some analysis to aid in development or
21diagnosing kernel problems. You can access some symbols by name in
22kernel built-ins or in kernel modules if the code was built with
23``CONFIG_KALLSYMS``.
24
25Kgdb is intended to be used as a source level debugger for the Linux
26kernel. It is used along with gdb to debug a Linux kernel. The
27expectation is that gdb can be used to "break in" to the kernel to
28inspect memory, variables and look through call stack information
29similar to the way an application developer would use gdb to debug an
30application. It is possible to place breakpoints in kernel code and
31perform some limited execution stepping.
32
33Two machines are required for using kgdb. One of these machines is a
34development machine and the other is the target machine. The kernel to
35be debugged runs on the target machine. The development machine runs an
36instance of gdb against the vmlinux file which contains the symbols (not
37a boot image such as bzImage, zImage, uImage...). In gdb the developer
38specifies the connection parameters and connects to kgdb. The type of
39connection a developer makes with gdb depends on the availability of
40kgdb I/O modules compiled as built-ins or loadable kernel modules in the
41test machine's kernel.
42
43Compiling a kernel
44==================
45
46- In order to enable compilation of kdb, you must first enable kgdb.
47
48- The kgdb test compile options are described in the kgdb test suite
49 chapter.
50
51Kernel config options for kgdb
52------------------------------
53
54To enable ``CONFIG_KGDB`` you should look under
55:menuselection:`Kernel hacking --> Kernel debugging` and select
56:menuselection:`KGDB: kernel debugger`.
57
58While it is not a hard requirement that you have symbols in your vmlinux
59file, gdb tends not to be very useful without the symbolic data, so you
60will want to turn on ``CONFIG_DEBUG_INFO`` which is called
61:menuselection:`Compile the kernel with debug info` in the config menu.
62
63It is advised, but not required, that you turn on the
64``CONFIG_FRAME_POINTER`` kernel option which is called :menuselection:`Compile
65the kernel with frame pointers` in the config menu. This option inserts code
66into the compiled executable which saves the frame information in registers
67or on the stack at different points which allows a debugger such as gdb to
68more accurately construct stack back traces while debugging the kernel.
69
70If the architecture that you are using supports the kernel option
71``CONFIG_STRICT_KERNEL_RWX``, you should consider turning it off. This
72option will prevent the use of software breakpoints because it marks
73certain regions of the kernel's memory space as read-only. If kgdb
74supports it for the architecture you are using, you can use hardware
75breakpoints if you desire to run with the ``CONFIG_STRICT_KERNEL_RWX``
76option turned on, else you need to turn off this option.
77
78Next you should choose one or more I/O drivers to interconnect the debugging
79host and debugged target. Early boot debugging requires a KGDB I/O
80driver that supports early debugging and the driver must be built into
81the kernel directly. Kgdb I/O driver configuration takes place via
82kernel or module parameters which you can learn more about in the
83section that describes the parameter kgdboc.
84
85Here is an example set of ``.config`` symbols to enable or disable for kgdb::
86
87 # CONFIG_STRICT_KERNEL_RWX is not set
88 CONFIG_FRAME_POINTER=y
89 CONFIG_KGDB=y
90 CONFIG_KGDB_SERIAL_CONSOLE=y
91
92Kernel config options for kdb
93-----------------------------
94
95Kdb is quite a bit more complex than the simple gdbstub sitting on top
96of the kernel's debug core. Kdb must implement a shell, and also adds
97some helper functions in other parts of the kernel, responsible for
98printing out interesting data such as what you would see if you ran
99``lsmod``, or ``ps``. In order to build kdb into the kernel you follow the
100same steps as you would for kgdb.
101
102The main config option for kdb is ``CONFIG_KGDB_KDB`` which is called
103:menuselection:`KGDB_KDB: include kdb frontend for kgdb` in the config menu.
104In theory you would have already also selected an I/O driver such as the
105``CONFIG_KGDB_SERIAL_CONSOLE`` interface if you plan on using kdb on a
106serial port, when you were configuring kgdb.
107
108If you want to use a PS/2-style keyboard with kdb, you would select
109``CONFIG_KDB_KEYBOARD`` which is called :menuselection:`KGDB_KDB: keyboard as
110input device` in the config menu. The ``CONFIG_KDB_KEYBOARD`` option is not
111used for anything in the gdb interface to kgdb. The ``CONFIG_KDB_KEYBOARD``
112option only works with kdb.
113
114Here is an example set of ``.config`` symbols to enable/disable kdb::
115
116 # CONFIG_STRICT_KERNEL_RWX is not set
117 CONFIG_FRAME_POINTER=y
118 CONFIG_KGDB=y
119 CONFIG_KGDB_SERIAL_CONSOLE=y
120 CONFIG_KGDB_KDB=y
121 CONFIG_KDB_KEYBOARD=y
122
123Kernel Debugger Boot Arguments
124==============================
125
126This section describes the various runtime kernel parameters that affect
127the configuration of the kernel debugger. The following chapter covers
128using kdb and kgdb as well as providing some examples of the
129configuration parameters.
130
131Kernel parameter: kgdboc
132------------------------
133
134The kgdboc driver was originally an abbreviation meant to stand for
135"kgdb over console". Today it is the primary mechanism to configure how
136to communicate from gdb to kgdb as well as the devices you want to use
137to interact with the kdb shell.
138
139For kgdb/gdb, kgdboc is designed to work with a single serial port. It
140is intended to cover the circumstance where you want to use a serial
141console as your primary console as well as using it to perform kernel
142debugging. It is also possible to use kgdb on a serial port which is not
143designated as a system console. Kgdboc may be configured as a kernel
144built-in or a kernel loadable module. You can only make use of
145``kgdbwait`` and early debugging if you build kgdboc into the kernel as
146a built-in.
147
148Optionally you can elect to activate kms (Kernel Mode Setting)
149integration. When you use kms with kgdboc and you have a video driver
150that has atomic mode setting hooks, it is possible to enter the debugger
151on the graphics console. When the kernel execution is resumed, the
152previous graphics mode will be restored. This integration can serve as a
153useful tool to aid in diagnosing crashes or doing analysis of memory
154with kdb while allowing the full graphics console applications to run.
155
156kgdboc arguments
157~~~~~~~~~~~~~~~~
158
159Usage::
160
161 kgdboc=[kms][[,]kbd][[,]serial_device][,baud]
162
163The order listed above must be observed if you use any of the optional
164configurations together.
165
166Abbreviations:
167
168- kms = Kernel Mode Setting
169
170- kbd = Keyboard
171
172You can configure kgdboc to use the keyboard, and/or a serial device
173depending on if you are using kdb and/or kgdb, in one of the following
174scenarios. The order listed above must be observed if you use any of the
175optional configurations together. Using kms + only gdb is generally not
176a useful combination.
177
178Using loadable module or built-in
179^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
180
1811. As a kernel built-in:
182
183 Use the kernel boot argument::
184
185 kgdboc=<tty-device>,[baud]
186
1872. As a kernel loadable module:
188
189 Use the command::
190
191 modprobe kgdboc kgdboc=<tty-device>,[baud]
192
193 Here are two examples of how you might format the kgdboc string. The
194 first is for an x86 target using the first serial port. The second
195 example is for the ARM Versatile AB using the second serial port.
196
197 1. ``kgdboc=ttyS0,115200``
198
199 2. ``kgdboc=ttyAMA1,115200``
200
201Configure kgdboc at runtime with sysfs
202^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
203
204At run time you can enable or disable kgdboc by writing parameters
205into sysfs. Here are two examples:
206
2071. Enable kgdboc on ttyS0::
208
209 echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc
210
2112. Disable kgdboc::
212
213 echo "" > /sys/module/kgdboc/parameters/kgdboc
214
215.. note::
216
217 You do not need to specify the baud if you are configuring the
218 console on tty which is already configured or open.
219
220More examples
221^^^^^^^^^^^^^
222
223You can configure kgdboc to use the keyboard, and/or a serial device
224depending on if you are using kdb and/or kgdb, in one of the following
225scenarios.
226
2271. kdb and kgdb over only a serial port::
228
229 kgdboc=<serial_device>[,baud]
230
231 Example::
232
233 kgdboc=ttyS0,115200
234
2352. kdb and kgdb with keyboard and a serial port::
236
237 kgdboc=kbd,<serial_device>[,baud]
238
239 Example::
240
241 kgdboc=kbd,ttyS0,115200
242
2433. kdb with a keyboard::
244
245 kgdboc=kbd
246
2474. kdb with kernel mode setting::
248
249 kgdboc=kms,kbd
250
2515. kdb with kernel mode setting and kgdb over a serial port::
252
253 kgdboc=kms,kbd,ttyS0,115200
254
255.. note::
256
257 Kgdboc does not support interrupting the target via the gdb remote
258 protocol. You must manually send a :kbd:`SysRq-G` unless you have a proxy
259 that splits console output to a terminal program. A console proxy has a
260 separate TCP port for the debugger and a separate TCP port for the
261 "human" console. The proxy can take care of sending the :kbd:`SysRq-G`
262 for you.
263
264When using kgdboc with no debugger proxy, you can end up connecting the
265debugger at one of two entry points. If an exception occurs after you
266have loaded kgdboc, a message should print on the console stating it is
267waiting for the debugger. In this case you disconnect your terminal
268program and then connect the debugger in its place. If you want to
269interrupt the target system and forcibly enter a debug session you have
270to issue a :kbd:`Sysrq` sequence and then type the letter :kbd:`g`. Then you
271disconnect the terminal session and connect gdb. Your options if you
272don't like this are to hack gdb to send the :kbd:`SysRq-G` for you as well as
273on the initial connect, or to use a debugger proxy that allows an
274unmodified gdb to do the debugging.
275
276Kernel parameter: ``kgdboc_earlycon``
277-------------------------------------
278
279If you specify the kernel parameter ``kgdboc_earlycon`` and your serial
280driver registers a boot console that supports polling (doesn't need
281interrupts and implements a nonblocking read() function) kgdb will attempt
282to work using the boot console until it can transition to the regular
283tty driver specified by the ``kgdboc`` parameter.
284
285Normally there is only one boot console (especially that implements the
286read() function) so just adding ``kgdboc_earlycon`` on its own is
287sufficient to make this work. If you have more than one boot console you
288can add the boot console's name to differentiate. Note that names that
289are registered through the boot console layer and the tty layer are not
290the same for the same port.
291
292For instance, on one board to be explicit you might do::
293
294 kgdboc_earlycon=qcom_geni kgdboc=ttyMSM0
295
296If the only boot console on the device was "qcom_geni", you could simplify::
297
298 kgdboc_earlycon kgdboc=ttyMSM0
299
300Kernel parameter: ``kgdbwait``
301------------------------------
302
303The Kernel command line option ``kgdbwait`` makes kgdb wait for a
304debugger connection during booting of a kernel. You can only use this
305option if you compiled a kgdb I/O driver into the kernel and you
306specified the I/O driver configuration as a kernel command line option.
307The kgdbwait parameter should always follow the configuration parameter
308for the kgdb I/O driver in the kernel command line else the I/O driver
309will not be configured prior to asking the kernel to use it to wait.
310
311The kernel will stop and wait as early as the I/O driver and
312architecture allows when you use this option. If you build the kgdb I/O
313driver as a loadable kernel module kgdbwait will not do anything.
314
315Kernel parameter: ``kgdbcon``
316-----------------------------
317
318The ``kgdbcon`` feature allows you to see printk() messages inside gdb
319while gdb is connected to the kernel. Kdb does not make use of the kgdbcon
320feature.
321
322Kgdb supports using the gdb serial protocol to send console messages to
323the debugger when the debugger is connected and running. There are two
324ways to activate this feature.
325
3261. Activate with the kernel command line option::
327
328 kgdbcon
329
3302. Use sysfs before configuring an I/O driver::
331
332 echo 1 > /sys/module/debug_core/parameters/kgdb_use_con
333
334.. note::
335
336 If you do this after you configure the kgdb I/O driver, the
337 setting will not take effect until the next point the I/O is
338 reconfigured.
339
340.. important::
341
342 You cannot use kgdboc + kgdbcon on a tty that is an
343 active system console. An example of incorrect usage is::
344
345 console=ttyS0,115200 kgdboc=ttyS0 kgdbcon
346
347It is possible to use this option with kgdboc on a tty that is not a
348system console.
349
350Run time parameter: ``kgdbreboot``
351----------------------------------
352
353The kgdbreboot feature allows you to change how the debugger deals with
354the reboot notification. You have 3 choices for the behavior. The
355default behavior is always set to 0.
356
357.. tabularcolumns:: |p{0.4cm}|p{11.5cm}|p{5.6cm}|
358
359.. flat-table::
360 :widths: 1 10 8
361
362 * - 1
363 - ``echo -1 > /sys/module/debug_core/parameters/kgdbreboot``
364 - Ignore the reboot notification entirely.
365
366 * - 2
367 - ``echo 0 > /sys/module/debug_core/parameters/kgdbreboot``
368 - Send the detach message to any attached debugger client.
369
370 * - 3
371 - ``echo 1 > /sys/module/debug_core/parameters/kgdbreboot``
372 - Enter the debugger on reboot notify.
373
374Kernel parameter: ``nokaslr``
375-----------------------------
376
377If the architecture that you are using enables KASLR by default,
378you should consider turning it off. KASLR randomizes the
379virtual address where the kernel image is mapped and confuses
380gdb which resolves addresses of kernel symbols from the symbol table
381of vmlinux.
382
383Using kdb
384=========
385
386Quick start for kdb on a serial port
387------------------------------------
388
389This is a quick example of how to use kdb.
390
3911. Configure kgdboc at boot using kernel parameters::
392
393 console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr
394
395 OR
396
397 Configure kgdboc after the kernel has booted; assuming you are using
398 a serial port console::
399
400 echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc
401
4022. Enter the kernel debugger manually or by waiting for an oops or
403 fault. There are several ways you can enter the kernel debugger
404 manually; all involve using the :kbd:`SysRq-G`, which means you must have
405 enabled ``CONFIG_MAGIC_SYSRQ=y`` in your kernel config.
406
407 - When logged in as root or with a super user session you can run::
408
409 echo g > /proc/sysrq-trigger
410
411 - Example using minicom 2.2
412
413 Press: :kbd:`CTRL-A` :kbd:`f` :kbd:`g`
414
415 - When you have telneted to a terminal server that supports sending
416 a remote break
417
418 Press: :kbd:`CTRL-]`
419
420 Type in: ``send break``
421
422 Press: :kbd:`Enter` :kbd:`g`
423
4243. From the kdb prompt you can run the ``help`` command to see a complete
425 list of the commands that are available.
426
427 Some useful commands in kdb include:
428
429 =========== =================================================================
430 ``lsmod`` Shows where kernel modules are loaded
431 ``ps`` Displays only the active processes
432 ``ps A`` Shows all the processes
433 ``summary`` Shows kernel version info and memory usage
434 ``bt`` Get a backtrace of the current process using dump_stack()
435 ``dmesg`` View the kernel syslog buffer
436 ``go`` Continue the system
437 =========== =================================================================
438
4394. When you are done using kdb you need to consider rebooting the system
440 or using the ``go`` command to resuming normal kernel execution. If you
441 have paused the kernel for a lengthy period of time, applications
442 that rely on timely networking or anything to do with real wall clock
443 time could be adversely affected, so you should take this into
444 consideration when using the kernel debugger.
445
446Quick start for kdb using a keyboard connected console
447------------------------------------------------------
448
449This is a quick example of how to use kdb with a keyboard.
450
4511. Configure kgdboc at boot using kernel parameters::
452
453 kgdboc=kbd
454
455 OR
456
457 Configure kgdboc after the kernel has booted::
458
459 echo kbd > /sys/module/kgdboc/parameters/kgdboc
460
4612. Enter the kernel debugger manually or by waiting for an oops or
462 fault. There are several ways you can enter the kernel debugger
463 manually; all involve using the :kbd:`SysRq-G`, which means you must have
464 enabled ``CONFIG_MAGIC_SYSRQ=y`` in your kernel config.
465
466 - When logged in as root or with a super user session you can run::
467
468 echo g > /proc/sysrq-trigger
469
470 - Example using a laptop keyboard:
471
472 Press and hold down: :kbd:`Alt`
473
474 Press and hold down: :kbd:`Fn`
475
476 Press and release the key with the label: :kbd:`SysRq`
477
478 Release: :kbd:`Fn`
479
480 Press and release: :kbd:`g`
481
482 Release: :kbd:`Alt`
483
484 - Example using a PS/2 101-key keyboard
485
486 Press and hold down: :kbd:`Alt`
487
488 Press and release the key with the label: :kbd:`SysRq`
489
490 Press and release: :kbd:`g`
491
492 Release: :kbd:`Alt`
493
4943. Now type in a kdb command such as ``help``, ``dmesg``, ``bt`` or ``go`` to
495 continue kernel execution.
496
497Using kgdb / gdb
498================
499
500In order to use kgdb you must activate it by passing configuration
501information to one of the kgdb I/O drivers. If you do not pass any
502configuration information kgdb will not do anything at all. Kgdb will
503only actively hook up to the kernel trap hooks if a kgdb I/O driver is
504loaded and configured. If you unconfigure a kgdb I/O driver, kgdb will
505unregister all the kernel hook points.
506
507All kgdb I/O drivers can be reconfigured at run time, if
508``CONFIG_SYSFS`` and ``CONFIG_MODULES`` are enabled, by echo'ing a new
509config string to ``/sys/module/<driver>/parameter/<option>``. The driver
510can be unconfigured by passing an empty string. You cannot change the
511configuration while the debugger is attached. Make sure to detach the
512debugger with the ``detach`` command prior to trying to unconfigure a
513kgdb I/O driver.
514
515Connecting with gdb to a serial port
516------------------------------------
517
5181. Configure kgdboc
519
520 Configure kgdboc at boot using kernel parameters::
521
522 kgdboc=ttyS0,115200
523
524 OR
525
526 Configure kgdboc after the kernel has booted::
527
528 echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc
529
5302. Stop kernel execution (break into the debugger)
531
532 In order to connect to gdb via kgdboc, the kernel must first be
533 stopped. There are several ways to stop the kernel which include
534 using kgdbwait as a boot argument, via a :kbd:`SysRq-G`, or running the
535 kernel until it takes an exception where it waits for the debugger to
536 attach.
537
538 - When logged in as root or with a super user session you can run::
539
540 echo g > /proc/sysrq-trigger
541
542 - Example using minicom 2.2
543
544 Press: :kbd:`CTRL-A` :kbd:`f` :kbd:`g`
545
546 - When you have telneted to a terminal server that supports sending
547 a remote break
548
549 Press: :kbd:`CTRL-]`
550
551 Type in: ``send break``
552
553 Press: :kbd:`Enter` :kbd:`g`
554
5553. Connect from gdb
556
557 Example (using a directly connected port)::
558
559 % gdb ./vmlinux
560 (gdb) set serial baud 115200
561 (gdb) target remote /dev/ttyS0
562
563
564 Example (kgdb to a terminal server on TCP port 2012)::
565
566 % gdb ./vmlinux
567 (gdb) target remote 192.168.2.2:2012
568
569
570 Once connected, you can debug a kernel the way you would debug an
571 application program.
572
573 If you are having problems connecting or something is going seriously
574 wrong while debugging, it will most often be the case that you want
575 to enable gdb to be verbose about its target communications. You do
576 this prior to issuing the ``target remote`` command by typing in::
577
578 set debug remote 1
579
580Remember if you continue in gdb, and need to "break in" again, you need
581to issue an other :kbd:`SysRq-G`. It is easy to create a simple entry point by
582putting a breakpoint at ``sys_sync`` and then you can run ``sync`` from a
583shell or script to break into the debugger.
584
585kgdb and kdb interoperability
586=============================
587
588It is possible to transition between kdb and kgdb dynamically. The debug
589core will remember which you used the last time and automatically start
590in the same mode.
591
592Switching between kdb and kgdb
593------------------------------
594
595Switching from kgdb to kdb
596~~~~~~~~~~~~~~~~~~~~~~~~~~
597
598There are two ways to switch from kgdb to kdb: you can use gdb to issue
599a maintenance packet, or you can blindly type the command ``$3#33``.
600Whenever the kernel debugger stops in kgdb mode it will print the
601message ``KGDB or $3#33 for KDB``. It is important to note that you have
602to type the sequence correctly in one pass. You cannot type a backspace
603or delete because kgdb will interpret that as part of the debug stream.
604
6051. Change from kgdb to kdb by blindly typing::
606
607 $3#33
608
6092. Change from kgdb to kdb with gdb::
610
611 maintenance packet 3
612
613 .. note::
614
615 Now you must kill gdb. Typically you press :kbd:`CTRL-Z` and issue
616 the command::
617
618 kill -9 %
619
620Change from kdb to kgdb
621~~~~~~~~~~~~~~~~~~~~~~~
622
623There are two ways you can change from kdb to kgdb. You can manually
624enter kgdb mode by issuing the kgdb command from the kdb shell prompt,
625or you can connect gdb while the kdb shell prompt is active. The kdb
626shell looks for the typical first commands that gdb would issue with the
627gdb remote protocol and if it sees one of those commands it
628automatically changes into kgdb mode.
629
6301. From kdb issue the command::
631
632 kgdb
633
6342. At the kdb prompt, disconnect the terminal program and connect gdb in
635 its place.
636
637Running kdb commands from gdb
638-----------------------------
639
640It is possible to run a limited set of kdb commands from gdb, using the
641gdb monitor command. You don't want to execute any of the run control or
642breakpoint operations, because it can disrupt the state of the kernel
643debugger. You should be using gdb for breakpoints and run control
644operations if you have gdb connected. The more useful commands to run
645are things like lsmod, dmesg, ps or possibly some of the memory
646information commands. To see all the kdb commands you can run
647``monitor help``.
648
649Example::
650
651 (gdb) monitor ps
652 1 idle process (state I) and
653 27 sleeping system daemon (state M) processes suppressed,
654 use 'ps A' to see all.
655 Task Addr Pid Parent [*] cpu State Thread Command
656
657 0xc78291d0 1 0 0 0 S 0xc7829404 init
658 0xc7954150 942 1 0 0 S 0xc7954384 dropbear
659 0xc78789c0 944 1 0 0 S 0xc7878bf4 sh
660 (gdb)
661
662kgdb Test Suite
663===============
664
665When kgdb is enabled in the kernel config you can also elect to enable
666the config parameter ``KGDB_TESTS``. Turning this on will enable a special
667kgdb I/O module which is designed to test the kgdb internal functions.
668
669The kgdb tests are mainly intended for developers to test the kgdb
670internals as well as a tool for developing a new kgdb architecture
671specific implementation. These tests are not really for end users of the
672Linux kernel. The primary source of documentation would be to look in
673the ``drivers/misc/kgdbts.c`` file.
674
675The kgdb test suite can also be configured at compile time to run the
676core set of tests by setting the kernel config parameter
677``KGDB_TESTS_ON_BOOT``. This particular option is aimed at automated
678regression testing and does not require modifying the kernel boot config
679arguments. If this is turned on, the kgdb test suite can be disabled by
680specifying ``kgdbts=`` as a kernel boot argument.
681
682Kernel Debugger Internals
683=========================
684
685Architecture Specifics
686----------------------
687
688The kernel debugger is organized into a number of components:
689
6901. The debug core
691
692 The debug core is found in ``kernel/debugger/debug_core.c``. It
693 contains:
694
695 - A generic OS exception handler which includes sync'ing the
696 processors into a stopped state on an multi-CPU system.
697
698 - The API to talk to the kgdb I/O drivers
699
700 - The API to make calls to the arch-specific kgdb implementation
701
702 - The logic to perform safe memory reads and writes to memory while
703 using the debugger
704
705 - A full implementation for software breakpoints unless overridden
706 by the arch
707
708 - The API to invoke either the kdb or kgdb frontend to the debug
709 core.
710
711 - The structures and callback API for atomic kernel mode setting.
712
713 .. note:: kgdboc is where the kms callbacks are invoked.
714
7152. kgdb arch-specific implementation
716
717 This implementation is generally found in ``arch/*/kernel/kgdb.c``. As
718 an example, ``arch/x86/kernel/kgdb.c`` contains the specifics to
719 implement HW breakpoint as well as the initialization to dynamically
720 register and unregister for the trap handlers on this architecture.
721 The arch-specific portion implements:
722
723 - contains an arch-specific trap catcher which invokes
724 kgdb_handle_exception() to start kgdb about doing its work
725
726 - translation to and from gdb specific packet format to struct pt_regs
727
728 - Registration and unregistration of architecture specific trap
729 hooks
730
731 - Any special exception handling and cleanup
732
733 - NMI exception handling and cleanup
734
735 - (optional) HW breakpoints
736
7373. gdbstub frontend (aka kgdb)
738
739 The gdbstub is located in ``kernel/debug/gdbstub.c``. It contains:
740
741 - All the logic to implement the gdb serial protocol
742
7434. kdb frontend
744
745 The kdb debugger shell is broken down into a number of components.
746 The kdb core is located in kernel/debug/kdb. There are a number of
747 helper functions in some of the other kernel components to make it
748 possible for kdb to examine and report information about the kernel
749 without taking locks that could cause a kernel deadlock. The kdb core
750 implements the following functionality.
751
752 - A simple shell
753
754 - The kdb core command set
755
756 - A registration API to register additional kdb shell commands.
757
758 - A good example of a self-contained kdb module is the ``ftdump``
759 command for dumping the ftrace buffer. See:
760 ``kernel/trace/trace_kdb.c``
761
762 - For an example of how to dynamically register a new kdb command
763 you can build the kdb_hello.ko kernel module from
764 ``samples/kdb/kdb_hello.c``. To build this example you can set
765 ``CONFIG_SAMPLES=y`` and ``CONFIG_SAMPLE_KDB=m`` in your kernel
766 config. Later run ``modprobe kdb_hello`` and the next time you
767 enter the kdb shell, you can run the ``hello`` command.
768
769 - The implementation for kdb_printf() which emits messages directly
770 to I/O drivers, bypassing the kernel log.
771
772 - SW / HW breakpoint management for the kdb shell
773
7745. kgdb I/O driver
775
776 Each kgdb I/O driver has to provide an implementation for the
777 following:
778
779 - configuration via built-in or module
780
781 - dynamic configuration and kgdb hook registration calls
782
783 - read and write character interface
784
785 - A cleanup handler for unconfiguring from the kgdb core
786
787 - (optional) Early debug methodology
788
789 Any given kgdb I/O driver has to operate very closely with the
790 hardware and must do it in such a way that does not enable interrupts
791 or change other parts of the system context without completely
792 restoring them. The kgdb core will repeatedly "poll" a kgdb I/O
793 driver for characters when it needs input. The I/O driver is expected
794 to return immediately if there is no data available. Doing so allows
795 for the future possibility to touch watchdog hardware in such a way
796 as to have a target system not reset when these are enabled.
797
798If you are intent on adding kgdb architecture specific support for a new
799architecture, the architecture should define ``HAVE_ARCH_KGDB`` in the
800architecture specific Kconfig file. This will enable kgdb for the
801architecture, and at that point you must create an architecture specific
802kgdb implementation.
803
804There are a few flags which must be set on every architecture in their
805``asm/kgdb.h`` file. These are:
806
807- ``NUMREGBYTES``:
808 The size in bytes of all of the registers, so that we
809 can ensure they will all fit into a packet.
810
811- ``BUFMAX``:
812 The size in bytes of the buffer GDB will read into. This must
813 be larger than NUMREGBYTES.
814
815- ``CACHE_FLUSH_IS_SAFE``:
816 Set to 1 if it is always safe to call
817 flush_cache_range or flush_icache_range. On some architectures,
818 these functions may not be safe to call on SMP since we keep other
819 CPUs in a holding pattern.
820
821There are also the following functions for the common backend, found in
822``kernel/kgdb.c``, that must be supplied by the architecture-specific
823backend unless marked as (optional), in which case a default function
824maybe used if the architecture does not need to provide a specific
825implementation.
826
827.. kernel-doc:: include/linux/kgdb.h
828 :internal:
829
830kgdboc internals
831----------------
832
833kgdboc and uarts
834~~~~~~~~~~~~~~~~
835
836The kgdboc driver is actually a very thin driver that relies on the
837underlying low level to the hardware driver having "polling hooks" to
838which the tty driver is attached. In the initial implementation of
839kgdboc the serial_core was changed to expose a low level UART hook for
840doing polled mode reading and writing of a single character while in an
841atomic context. When kgdb makes an I/O request to the debugger, kgdboc
842invokes a callback in the serial core which in turn uses the callback in
843the UART driver.
844
845When using kgdboc with a UART, the UART driver must implement two
846callbacks in the struct uart_ops.
847Example from ``drivers/8250.c``::
848
849
850 #ifdef CONFIG_CONSOLE_POLL
851 .poll_get_char = serial8250_get_poll_char,
852 .poll_put_char = serial8250_put_poll_char,
853 #endif
854
855
856Any implementation specifics around creating a polling driver use the
857``#ifdef CONFIG_CONSOLE_POLL``, as shown above. Keep in mind that
858polling hooks have to be implemented in such a way that they can be
859called from an atomic context and have to restore the state of the UART
860chip on return such that the system can return to normal when the
861debugger detaches. You need to be very careful with any kind of lock you
862consider, because failing here is most likely going to mean pressing the
863reset button.
864
865kgdboc and keyboards
866~~~~~~~~~~~~~~~~~~~~~~~~
867
868The kgdboc driver contains logic to configure communications with an
869attached keyboard. The keyboard infrastructure is only compiled into the
870kernel when ``CONFIG_KDB_KEYBOARD=y`` is set in the kernel configuration.
871
872The core polled keyboard driver for PS/2 type keyboards is in
873``drivers/char/kdb_keyboard.c``. This driver is hooked into the debug core
874when kgdboc populates the callback in the array called
875:c:expr:`kdb_poll_funcs[]`. The kdb_get_kbd_char() is the top-level
876function which polls hardware for single character input.
877
878kgdboc and kms
879~~~~~~~~~~~~~~~~~~
880
881The kgdboc driver contains logic to request the graphics display to
882switch to a text context when you are using ``kgdboc=kms,kbd``, provided
883that you have a video driver which has a frame buffer console and atomic
884kernel mode setting support.
885
886Every time the kernel debugger is entered it calls
887kgdboc_pre_exp_handler() which in turn calls con_debug_enter()
888in the virtual console layer. On resuming kernel execution, the kernel
889debugger calls kgdboc_post_exp_handler() which in turn calls
890con_debug_leave().
891
892Any video driver that wants to be compatible with the kernel debugger
893and the atomic kms callbacks must implement the ``mode_set_base_atomic``,
894``fb_debug_enter`` and ``fb_debug_leave operations``. For the
895``fb_debug_enter`` and ``fb_debug_leave`` the option exists to use the
896generic drm fb helper functions or implement something custom for the
897hardware. The following example shows the initialization of the
898.mode_set_base_atomic operation in
899drivers/gpu/drm/i915/intel_display.c::
900
901
902 static const struct drm_crtc_helper_funcs intel_helper_funcs = {
903 [...]
904 .mode_set_base_atomic = intel_pipe_set_base_atomic,
905 [...]
906 };
907
908
909Here is an example of how the i915 driver initializes the
910fb_debug_enter and fb_debug_leave functions to use the generic drm
911helpers in ``drivers/gpu/drm/i915/intel_fb.c``::
912
913
914 static struct fb_ops intelfb_ops = {
915 [...]
916 .fb_debug_enter = drm_fb_helper_debug_enter,
917 .fb_debug_leave = drm_fb_helper_debug_leave,
918 [...]
919 };
920
921
922Credits
923=======
924
925The following people have contributed to this document:
926
9271. Amit Kale <amitkale@linsyssoft.com>
928
9292. Tom Rini <trini@kernel.crashing.org>
930
931In March 2008 this document was completely rewritten by:
932
933- Jason Wessel <jason.wessel@windriver.com>
934
935In Jan 2010 this document was updated to include kdb.
936
937- Jason Wessel <jason.wessel@windriver.com>
1=================================================
2Using kgdb, kdb and the kernel debugger internals
3=================================================
4
5:Author: Jason Wessel
6
7Introduction
8============
9
10The kernel has two different debugger front ends (kdb and kgdb) which
11interface to the debug core. It is possible to use either of the
12debugger front ends and dynamically transition between them if you
13configure the kernel properly at compile and runtime.
14
15Kdb is simplistic shell-style interface which you can use on a system
16console with a keyboard or serial console. You can use it to inspect
17memory, registers, process lists, dmesg, and even set breakpoints to
18stop in a certain location. Kdb is not a source level debugger, although
19you can set breakpoints and execute some basic kernel run control. Kdb
20is mainly aimed at doing some analysis to aid in development or
21diagnosing kernel problems. You can access some symbols by name in
22kernel built-ins or in kernel modules if the code was built with
23``CONFIG_KALLSYMS``.
24
25Kgdb is intended to be used as a source level debugger for the Linux
26kernel. It is used along with gdb to debug a Linux kernel. The
27expectation is that gdb can be used to "break in" to the kernel to
28inspect memory, variables and look through call stack information
29similar to the way an application developer would use gdb to debug an
30application. It is possible to place breakpoints in kernel code and
31perform some limited execution stepping.
32
33Two machines are required for using kgdb. One of these machines is a
34development machine and the other is the target machine. The kernel to
35be debugged runs on the target machine. The development machine runs an
36instance of gdb against the vmlinux file which contains the symbols (not
37a boot image such as bzImage, zImage, uImage...). In gdb the developer
38specifies the connection parameters and connects to kgdb. The type of
39connection a developer makes with gdb depends on the availability of
40kgdb I/O modules compiled as built-ins or loadable kernel modules in the
41test machine's kernel.
42
43Compiling a kernel
44==================
45
46- In order to enable compilation of kdb, you must first enable kgdb.
47
48- The kgdb test compile options are described in the kgdb test suite
49 chapter.
50
51Kernel config options for kgdb
52------------------------------
53
54To enable ``CONFIG_KGDB`` you should look under
55:menuselection:`Kernel hacking --> Kernel debugging` and select
56:menuselection:`KGDB: kernel debugger`.
57
58While it is not a hard requirement that you have symbols in your vmlinux
59file, gdb tends not to be very useful without the symbolic data, so you
60will want to turn on ``CONFIG_DEBUG_INFO`` which is called
61:menuselection:`Compile the kernel with debug info` in the config menu.
62
63It is advised, but not required, that you turn on the
64``CONFIG_FRAME_POINTER`` kernel option which is called :menuselection:`Compile
65the kernel with frame pointers` in the config menu. This option inserts code
66to into the compiled executable which saves the frame information in
67registers or on the stack at different points which allows a debugger
68such as gdb to more accurately construct stack back traces while
69debugging the kernel.
70
71If the architecture that you are using supports the kernel option
72``CONFIG_STRICT_KERNEL_RWX``, you should consider turning it off. This
73option will prevent the use of software breakpoints because it marks
74certain regions of the kernel's memory space as read-only. If kgdb
75supports it for the architecture you are using, you can use hardware
76breakpoints if you desire to run with the ``CONFIG_STRICT_KERNEL_RWX``
77option turned on, else you need to turn off this option.
78
79Next you should choose one of more I/O drivers to interconnect debugging
80host and debugged target. Early boot debugging requires a KGDB I/O
81driver that supports early debugging and the driver must be built into
82the kernel directly. Kgdb I/O driver configuration takes place via
83kernel or module parameters which you can learn more about in the in the
84section that describes the parameter kgdboc.
85
86Here is an example set of ``.config`` symbols to enable or disable for kgdb::
87
88 # CONFIG_STRICT_KERNEL_RWX is not set
89 CONFIG_FRAME_POINTER=y
90 CONFIG_KGDB=y
91 CONFIG_KGDB_SERIAL_CONSOLE=y
92
93Kernel config options for kdb
94-----------------------------
95
96Kdb is quite a bit more complex than the simple gdbstub sitting on top
97of the kernel's debug core. Kdb must implement a shell, and also adds
98some helper functions in other parts of the kernel, responsible for
99printing out interesting data such as what you would see if you ran
100``lsmod``, or ``ps``. In order to build kdb into the kernel you follow the
101same steps as you would for kgdb.
102
103The main config option for kdb is ``CONFIG_KGDB_KDB`` which is called
104:menuselection:`KGDB_KDB: include kdb frontend for kgdb` in the config menu.
105In theory you would have already also selected an I/O driver such as the
106``CONFIG_KGDB_SERIAL_CONSOLE`` interface if you plan on using kdb on a
107serial port, when you were configuring kgdb.
108
109If you want to use a PS/2-style keyboard with kdb, you would select
110``CONFIG_KDB_KEYBOARD`` which is called :menuselection:`KGDB_KDB: keyboard as
111input device` in the config menu. The ``CONFIG_KDB_KEYBOARD`` option is not
112used for anything in the gdb interface to kgdb. The ``CONFIG_KDB_KEYBOARD``
113option only works with kdb.
114
115Here is an example set of ``.config`` symbols to enable/disable kdb::
116
117 # CONFIG_STRICT_KERNEL_RWX is not set
118 CONFIG_FRAME_POINTER=y
119 CONFIG_KGDB=y
120 CONFIG_KGDB_SERIAL_CONSOLE=y
121 CONFIG_KGDB_KDB=y
122 CONFIG_KDB_KEYBOARD=y
123
124Kernel Debugger Boot Arguments
125==============================
126
127This section describes the various runtime kernel parameters that affect
128the configuration of the kernel debugger. The following chapter covers
129using kdb and kgdb as well as providing some examples of the
130configuration parameters.
131
132Kernel parameter: kgdboc
133------------------------
134
135The kgdboc driver was originally an abbreviation meant to stand for
136"kgdb over console". Today it is the primary mechanism to configure how
137to communicate from gdb to kgdb as well as the devices you want to use
138to interact with the kdb shell.
139
140For kgdb/gdb, kgdboc is designed to work with a single serial port. It
141is intended to cover the circumstance where you want to use a serial
142console as your primary console as well as using it to perform kernel
143debugging. It is also possible to use kgdb on a serial port which is not
144designated as a system console. Kgdboc may be configured as a kernel
145built-in or a kernel loadable module. You can only make use of
146``kgdbwait`` and early debugging if you build kgdboc into the kernel as
147a built-in.
148
149Optionally you can elect to activate kms (Kernel Mode Setting)
150integration. When you use kms with kgdboc and you have a video driver
151that has atomic mode setting hooks, it is possible to enter the debugger
152on the graphics console. When the kernel execution is resumed, the
153previous graphics mode will be restored. This integration can serve as a
154useful tool to aid in diagnosing crashes or doing analysis of memory
155with kdb while allowing the full graphics console applications to run.
156
157kgdboc arguments
158~~~~~~~~~~~~~~~~
159
160Usage::
161
162 kgdboc=[kms][[,]kbd][[,]serial_device][,baud]
163
164The order listed above must be observed if you use any of the optional
165configurations together.
166
167Abbreviations:
168
169- kms = Kernel Mode Setting
170
171- kbd = Keyboard
172
173You can configure kgdboc to use the keyboard, and/or a serial device
174depending on if you are using kdb and/or kgdb, in one of the following
175scenarios. The order listed above must be observed if you use any of the
176optional configurations together. Using kms + only gdb is generally not
177a useful combination.
178
179Using loadable module or built-in
180^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
181
1821. As a kernel built-in:
183
184 Use the kernel boot argument::
185
186 kgdboc=<tty-device>,[baud]
187
1882. As a kernel loadable module:
189
190 Use the command::
191
192 modprobe kgdboc kgdboc=<tty-device>,[baud]
193
194 Here are two examples of how you might format the kgdboc string. The
195 first is for an x86 target using the first serial port. The second
196 example is for the ARM Versatile AB using the second serial port.
197
198 1. ``kgdboc=ttyS0,115200``
199
200 2. ``kgdboc=ttyAMA1,115200``
201
202Configure kgdboc at runtime with sysfs
203^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
204
205At run time you can enable or disable kgdboc by echoing a parameters
206into the sysfs. Here are two examples:
207
2081. Enable kgdboc on ttyS0::
209
210 echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc
211
2122. Disable kgdboc::
213
214 echo "" > /sys/module/kgdboc/parameters/kgdboc
215
216.. note::
217
218 You do not need to specify the baud if you are configuring the
219 console on tty which is already configured or open.
220
221More examples
222^^^^^^^^^^^^^
223
224You can configure kgdboc to use the keyboard, and/or a serial device
225depending on if you are using kdb and/or kgdb, in one of the following
226scenarios.
227
2281. kdb and kgdb over only a serial port::
229
230 kgdboc=<serial_device>[,baud]
231
232 Example::
233
234 kgdboc=ttyS0,115200
235
2362. kdb and kgdb with keyboard and a serial port::
237
238 kgdboc=kbd,<serial_device>[,baud]
239
240 Example::
241
242 kgdboc=kbd,ttyS0,115200
243
2443. kdb with a keyboard::
245
246 kgdboc=kbd
247
2484. kdb with kernel mode setting::
249
250 kgdboc=kms,kbd
251
2525. kdb with kernel mode setting and kgdb over a serial port::
253
254 kgdboc=kms,kbd,ttyS0,115200
255
256.. note::
257
258 Kgdboc does not support interrupting the target via the gdb remote
259 protocol. You must manually send a :kbd:`SysRq-G` unless you have a proxy
260 that splits console output to a terminal program. A console proxy has a
261 separate TCP port for the debugger and a separate TCP port for the
262 "human" console. The proxy can take care of sending the :kbd:`SysRq-G`
263 for you.
264
265When using kgdboc with no debugger proxy, you can end up connecting the
266debugger at one of two entry points. If an exception occurs after you
267have loaded kgdboc, a message should print on the console stating it is
268waiting for the debugger. In this case you disconnect your terminal
269program and then connect the debugger in its place. If you want to
270interrupt the target system and forcibly enter a debug session you have
271to issue a :kbd:`Sysrq` sequence and then type the letter :kbd:`g`. Then you
272disconnect the terminal session and connect gdb. Your options if you
273don't like this are to hack gdb to send the :kbd:`SysRq-G` for you as well as
274on the initial connect, or to use a debugger proxy that allows an
275unmodified gdb to do the debugging.
276
277Kernel parameter: ``kgdboc_earlycon``
278-------------------------------------
279
280If you specify the kernel parameter ``kgdboc_earlycon`` and your serial
281driver registers a boot console that supports polling (doesn't need
282interrupts and implements a nonblocking read() function) kgdb will attempt
283to work using the boot console until it can transition to the regular
284tty driver specified by the ``kgdboc`` parameter.
285
286Normally there is only one boot console (especially that implements the
287read() function) so just adding ``kgdboc_earlycon`` on its own is
288sufficient to make this work. If you have more than one boot console you
289can add the boot console's name to differentiate. Note that names that
290are registered through the boot console layer and the tty layer are not
291the same for the same port.
292
293For instance, on one board to be explicit you might do::
294
295 kgdboc_earlycon=qcom_geni kgdboc=ttyMSM0
296
297If the only boot console on the device was "qcom_geni", you could simplify::
298
299 kgdboc_earlycon kgdboc=ttyMSM0
300
301Kernel parameter: ``kgdbwait``
302------------------------------
303
304The Kernel command line option ``kgdbwait`` makes kgdb wait for a
305debugger connection during booting of a kernel. You can only use this
306option if you compiled a kgdb I/O driver into the kernel and you
307specified the I/O driver configuration as a kernel command line option.
308The kgdbwait parameter should always follow the configuration parameter
309for the kgdb I/O driver in the kernel command line else the I/O driver
310will not be configured prior to asking the kernel to use it to wait.
311
312The kernel will stop and wait as early as the I/O driver and
313architecture allows when you use this option. If you build the kgdb I/O
314driver as a loadable kernel module kgdbwait will not do anything.
315
316Kernel parameter: ``kgdbcon``
317-----------------------------
318
319The ``kgdbcon`` feature allows you to see printk() messages inside gdb
320while gdb is connected to the kernel. Kdb does not make use of the kgdbcon
321feature.
322
323Kgdb supports using the gdb serial protocol to send console messages to
324the debugger when the debugger is connected and running. There are two
325ways to activate this feature.
326
3271. Activate with the kernel command line option::
328
329 kgdbcon
330
3312. Use sysfs before configuring an I/O driver::
332
333 echo 1 > /sys/module/kgdb/parameters/kgdb_use_con
334
335.. note::
336
337 If you do this after you configure the kgdb I/O driver, the
338 setting will not take effect until the next point the I/O is
339 reconfigured.
340
341.. important::
342
343 You cannot use kgdboc + kgdbcon on a tty that is an
344 active system console. An example of incorrect usage is::
345
346 console=ttyS0,115200 kgdboc=ttyS0 kgdbcon
347
348It is possible to use this option with kgdboc on a tty that is not a
349system console.
350
351Run time parameter: ``kgdbreboot``
352----------------------------------
353
354The kgdbreboot feature allows you to change how the debugger deals with
355the reboot notification. You have 3 choices for the behavior. The
356default behavior is always set to 0.
357
358.. tabularcolumns:: |p{0.4cm}|p{11.5cm}|p{5.6cm}|
359
360.. flat-table::
361 :widths: 1 10 8
362
363 * - 1
364 - ``echo -1 > /sys/module/debug_core/parameters/kgdbreboot``
365 - Ignore the reboot notification entirely.
366
367 * - 2
368 - ``echo 0 > /sys/module/debug_core/parameters/kgdbreboot``
369 - Send the detach message to any attached debugger client.
370
371 * - 3
372 - ``echo 1 > /sys/module/debug_core/parameters/kgdbreboot``
373 - Enter the debugger on reboot notify.
374
375Kernel parameter: ``nokaslr``
376-----------------------------
377
378If the architecture that you are using enable KASLR by default,
379you should consider turning it off. KASLR randomizes the
380virtual address where the kernel image is mapped and confuse
381gdb which resolve kernel symbol address from symbol table
382of vmlinux.
383
384Using kdb
385=========
386
387Quick start for kdb on a serial port
388------------------------------------
389
390This is a quick example of how to use kdb.
391
3921. Configure kgdboc at boot using kernel parameters::
393
394 console=ttyS0,115200 kgdboc=ttyS0,115200 nokaslr
395
396 OR
397
398 Configure kgdboc after the kernel has booted; assuming you are using
399 a serial port console::
400
401 echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc
402
4032. Enter the kernel debugger manually or by waiting for an oops or
404 fault. There are several ways you can enter the kernel debugger
405 manually; all involve using the :kbd:`SysRq-G`, which means you must have
406 enabled ``CONFIG_MAGIC_SysRq=y`` in your kernel config.
407
408 - When logged in as root or with a super user session you can run::
409
410 echo g > /proc/sysrq-trigger
411
412 - Example using minicom 2.2
413
414 Press: :kbd:`CTRL-A` :kbd:`f` :kbd:`g`
415
416 - When you have telneted to a terminal server that supports sending
417 a remote break
418
419 Press: :kbd:`CTRL-]`
420
421 Type in: ``send break``
422
423 Press: :kbd:`Enter` :kbd:`g`
424
4253. From the kdb prompt you can run the ``help`` command to see a complete
426 list of the commands that are available.
427
428 Some useful commands in kdb include:
429
430 =========== =================================================================
431 ``lsmod`` Shows where kernel modules are loaded
432 ``ps`` Displays only the active processes
433 ``ps A`` Shows all the processes
434 ``summary`` Shows kernel version info and memory usage
435 ``bt`` Get a backtrace of the current process using dump_stack()
436 ``dmesg`` View the kernel syslog buffer
437 ``go`` Continue the system
438 =========== =================================================================
439
4404. When you are done using kdb you need to consider rebooting the system
441 or using the ``go`` command to resuming normal kernel execution. If you
442 have paused the kernel for a lengthy period of time, applications
443 that rely on timely networking or anything to do with real wall clock
444 time could be adversely affected, so you should take this into
445 consideration when using the kernel debugger.
446
447Quick start for kdb using a keyboard connected console
448------------------------------------------------------
449
450This is a quick example of how to use kdb with a keyboard.
451
4521. Configure kgdboc at boot using kernel parameters::
453
454 kgdboc=kbd
455
456 OR
457
458 Configure kgdboc after the kernel has booted::
459
460 echo kbd > /sys/module/kgdboc/parameters/kgdboc
461
4622. Enter the kernel debugger manually or by waiting for an oops or
463 fault. There are several ways you can enter the kernel debugger
464 manually; all involve using the :kbd:`SysRq-G`, which means you must have
465 enabled ``CONFIG_MAGIC_SysRq=y`` in your kernel config.
466
467 - When logged in as root or with a super user session you can run::
468
469 echo g > /proc/sysrq-trigger
470
471 - Example using a laptop keyboard:
472
473 Press and hold down: :kbd:`Alt`
474
475 Press and hold down: :kbd:`Fn`
476
477 Press and release the key with the label: :kbd:`SysRq`
478
479 Release: :kbd:`Fn`
480
481 Press and release: :kbd:`g`
482
483 Release: :kbd:`Alt`
484
485 - Example using a PS/2 101-key keyboard
486
487 Press and hold down: :kbd:`Alt`
488
489 Press and release the key with the label: :kbd:`SysRq`
490
491 Press and release: :kbd:`g`
492
493 Release: :kbd:`Alt`
494
4953. Now type in a kdb command such as ``help``, ``dmesg``, ``bt`` or ``go`` to
496 continue kernel execution.
497
498Using kgdb / gdb
499================
500
501In order to use kgdb you must activate it by passing configuration
502information to one of the kgdb I/O drivers. If you do not pass any
503configuration information kgdb will not do anything at all. Kgdb will
504only actively hook up to the kernel trap hooks if a kgdb I/O driver is
505loaded and configured. If you unconfigure a kgdb I/O driver, kgdb will
506unregister all the kernel hook points.
507
508All kgdb I/O drivers can be reconfigured at run time, if
509``CONFIG_SYSFS`` and ``CONFIG_MODULES`` are enabled, by echo'ing a new
510config string to ``/sys/module/<driver>/parameter/<option>``. The driver
511can be unconfigured by passing an empty string. You cannot change the
512configuration while the debugger is attached. Make sure to detach the
513debugger with the ``detach`` command prior to trying to unconfigure a
514kgdb I/O driver.
515
516Connecting with gdb to a serial port
517------------------------------------
518
5191. Configure kgdboc
520
521 Configure kgdboc at boot using kernel parameters::
522
523 kgdboc=ttyS0,115200
524
525 OR
526
527 Configure kgdboc after the kernel has booted::
528
529 echo ttyS0 > /sys/module/kgdboc/parameters/kgdboc
530
5312. Stop kernel execution (break into the debugger)
532
533 In order to connect to gdb via kgdboc, the kernel must first be
534 stopped. There are several ways to stop the kernel which include
535 using kgdbwait as a boot argument, via a :kbd:`SysRq-G`, or running the
536 kernel until it takes an exception where it waits for the debugger to
537 attach.
538
539 - When logged in as root or with a super user session you can run::
540
541 echo g > /proc/sysrq-trigger
542
543 - Example using minicom 2.2
544
545 Press: :kbd:`CTRL-A` :kbd:`f` :kbd:`g`
546
547 - When you have telneted to a terminal server that supports sending
548 a remote break
549
550 Press: :kbd:`CTRL-]`
551
552 Type in: ``send break``
553
554 Press: :kbd:`Enter` :kbd:`g`
555
5563. Connect from gdb
557
558 Example (using a directly connected port)::
559
560 % gdb ./vmlinux
561 (gdb) set remotebaud 115200
562 (gdb) target remote /dev/ttyS0
563
564
565 Example (kgdb to a terminal server on TCP port 2012)::
566
567 % gdb ./vmlinux
568 (gdb) target remote 192.168.2.2:2012
569
570
571 Once connected, you can debug a kernel the way you would debug an
572 application program.
573
574 If you are having problems connecting or something is going seriously
575 wrong while debugging, it will most often be the case that you want
576 to enable gdb to be verbose about its target communications. You do
577 this prior to issuing the ``target remote`` command by typing in::
578
579 set debug remote 1
580
581Remember if you continue in gdb, and need to "break in" again, you need
582to issue an other :kbd:`SysRq-G`. It is easy to create a simple entry point by
583putting a breakpoint at ``sys_sync`` and then you can run ``sync`` from a
584shell or script to break into the debugger.
585
586kgdb and kdb interoperability
587=============================
588
589It is possible to transition between kdb and kgdb dynamically. The debug
590core will remember which you used the last time and automatically start
591in the same mode.
592
593Switching between kdb and kgdb
594------------------------------
595
596Switching from kgdb to kdb
597~~~~~~~~~~~~~~~~~~~~~~~~~~
598
599There are two ways to switch from kgdb to kdb: you can use gdb to issue
600a maintenance packet, or you can blindly type the command ``$3#33``.
601Whenever the kernel debugger stops in kgdb mode it will print the
602message ``KGDB or $3#33 for KDB``. It is important to note that you have
603to type the sequence correctly in one pass. You cannot type a backspace
604or delete because kgdb will interpret that as part of the debug stream.
605
6061. Change from kgdb to kdb by blindly typing::
607
608 $3#33
609
6102. Change from kgdb to kdb with gdb::
611
612 maintenance packet 3
613
614 .. note::
615
616 Now you must kill gdb. Typically you press :kbd:`CTRL-Z` and issue
617 the command::
618
619 kill -9 %
620
621Change from kdb to kgdb
622~~~~~~~~~~~~~~~~~~~~~~~
623
624There are two ways you can change from kdb to kgdb. You can manually
625enter kgdb mode by issuing the kgdb command from the kdb shell prompt,
626or you can connect gdb while the kdb shell prompt is active. The kdb
627shell looks for the typical first commands that gdb would issue with the
628gdb remote protocol and if it sees one of those commands it
629automatically changes into kgdb mode.
630
6311. From kdb issue the command::
632
633 kgdb
634
635 Now disconnect your terminal program and connect gdb in its place
636
6372. At the kdb prompt, disconnect the terminal program and connect gdb in
638 its place.
639
640Running kdb commands from gdb
641-----------------------------
642
643It is possible to run a limited set of kdb commands from gdb, using the
644gdb monitor command. You don't want to execute any of the run control or
645breakpoint operations, because it can disrupt the state of the kernel
646debugger. You should be using gdb for breakpoints and run control
647operations if you have gdb connected. The more useful commands to run
648are things like lsmod, dmesg, ps or possibly some of the memory
649information commands. To see all the kdb commands you can run
650``monitor help``.
651
652Example::
653
654 (gdb) monitor ps
655 1 idle process (state I) and
656 27 sleeping system daemon (state M) processes suppressed,
657 use 'ps A' to see all.
658 Task Addr Pid Parent [*] cpu State Thread Command
659
660 0xc78291d0 1 0 0 0 S 0xc7829404 init
661 0xc7954150 942 1 0 0 S 0xc7954384 dropbear
662 0xc78789c0 944 1 0 0 S 0xc7878bf4 sh
663 (gdb)
664
665kgdb Test Suite
666===============
667
668When kgdb is enabled in the kernel config you can also elect to enable
669the config parameter ``KGDB_TESTS``. Turning this on will enable a special
670kgdb I/O module which is designed to test the kgdb internal functions.
671
672The kgdb tests are mainly intended for developers to test the kgdb
673internals as well as a tool for developing a new kgdb architecture
674specific implementation. These tests are not really for end users of the
675Linux kernel. The primary source of documentation would be to look in
676the ``drivers/misc/kgdbts.c`` file.
677
678The kgdb test suite can also be configured at compile time to run the
679core set of tests by setting the kernel config parameter
680``KGDB_TESTS_ON_BOOT``. This particular option is aimed at automated
681regression testing and does not require modifying the kernel boot config
682arguments. If this is turned on, the kgdb test suite can be disabled by
683specifying ``kgdbts=`` as a kernel boot argument.
684
685Kernel Debugger Internals
686=========================
687
688Architecture Specifics
689----------------------
690
691The kernel debugger is organized into a number of components:
692
6931. The debug core
694
695 The debug core is found in ``kernel/debugger/debug_core.c``. It
696 contains:
697
698 - A generic OS exception handler which includes sync'ing the
699 processors into a stopped state on an multi-CPU system.
700
701 - The API to talk to the kgdb I/O drivers
702
703 - The API to make calls to the arch-specific kgdb implementation
704
705 - The logic to perform safe memory reads and writes to memory while
706 using the debugger
707
708 - A full implementation for software breakpoints unless overridden
709 by the arch
710
711 - The API to invoke either the kdb or kgdb frontend to the debug
712 core.
713
714 - The structures and callback API for atomic kernel mode setting.
715
716 .. note:: kgdboc is where the kms callbacks are invoked.
717
7182. kgdb arch-specific implementation
719
720 This implementation is generally found in ``arch/*/kernel/kgdb.c``. As
721 an example, ``arch/x86/kernel/kgdb.c`` contains the specifics to
722 implement HW breakpoint as well as the initialization to dynamically
723 register and unregister for the trap handlers on this architecture.
724 The arch-specific portion implements:
725
726 - contains an arch-specific trap catcher which invokes
727 kgdb_handle_exception() to start kgdb about doing its work
728
729 - translation to and from gdb specific packet format to :c:type:`pt_regs`
730
731 - Registration and unregistration of architecture specific trap
732 hooks
733
734 - Any special exception handling and cleanup
735
736 - NMI exception handling and cleanup
737
738 - (optional) HW breakpoints
739
7403. gdbstub frontend (aka kgdb)
741
742 The gdbstub is located in ``kernel/debug/gdbstub.c``. It contains:
743
744 - All the logic to implement the gdb serial protocol
745
7464. kdb frontend
747
748 The kdb debugger shell is broken down into a number of components.
749 The kdb core is located in kernel/debug/kdb. There are a number of
750 helper functions in some of the other kernel components to make it
751 possible for kdb to examine and report information about the kernel
752 without taking locks that could cause a kernel deadlock. The kdb core
753 contains implements the following functionality.
754
755 - A simple shell
756
757 - The kdb core command set
758
759 - A registration API to register additional kdb shell commands.
760
761 - A good example of a self-contained kdb module is the ``ftdump``
762 command for dumping the ftrace buffer. See:
763 ``kernel/trace/trace_kdb.c``
764
765 - For an example of how to dynamically register a new kdb command
766 you can build the kdb_hello.ko kernel module from
767 ``samples/kdb/kdb_hello.c``. To build this example you can set
768 ``CONFIG_SAMPLES=y`` and ``CONFIG_SAMPLE_KDB=m`` in your kernel
769 config. Later run ``modprobe kdb_hello`` and the next time you
770 enter the kdb shell, you can run the ``hello`` command.
771
772 - The implementation for kdb_printf() which emits messages directly
773 to I/O drivers, bypassing the kernel log.
774
775 - SW / HW breakpoint management for the kdb shell
776
7775. kgdb I/O driver
778
779 Each kgdb I/O driver has to provide an implementation for the
780 following:
781
782 - configuration via built-in or module
783
784 - dynamic configuration and kgdb hook registration calls
785
786 - read and write character interface
787
788 - A cleanup handler for unconfiguring from the kgdb core
789
790 - (optional) Early debug methodology
791
792 Any given kgdb I/O driver has to operate very closely with the
793 hardware and must do it in such a way that does not enable interrupts
794 or change other parts of the system context without completely
795 restoring them. The kgdb core will repeatedly "poll" a kgdb I/O
796 driver for characters when it needs input. The I/O driver is expected
797 to return immediately if there is no data available. Doing so allows
798 for the future possibility to touch watchdog hardware in such a way
799 as to have a target system not reset when these are enabled.
800
801If you are intent on adding kgdb architecture specific support for a new
802architecture, the architecture should define ``HAVE_ARCH_KGDB`` in the
803architecture specific Kconfig file. This will enable kgdb for the
804architecture, and at that point you must create an architecture specific
805kgdb implementation.
806
807There are a few flags which must be set on every architecture in their
808``asm/kgdb.h`` file. These are:
809
810- ``NUMREGBYTES``:
811 The size in bytes of all of the registers, so that we
812 can ensure they will all fit into a packet.
813
814- ``BUFMAX``:
815 The size in bytes of the buffer GDB will read into. This must
816 be larger than NUMREGBYTES.
817
818- ``CACHE_FLUSH_IS_SAFE``:
819 Set to 1 if it is always safe to call
820 flush_cache_range or flush_icache_range. On some architectures,
821 these functions may not be safe to call on SMP since we keep other
822 CPUs in a holding pattern.
823
824There are also the following functions for the common backend, found in
825``kernel/kgdb.c``, that must be supplied by the architecture-specific
826backend unless marked as (optional), in which case a default function
827maybe used if the architecture does not need to provide a specific
828implementation.
829
830.. kernel-doc:: include/linux/kgdb.h
831 :internal:
832
833kgdboc internals
834----------------
835
836kgdboc and uarts
837~~~~~~~~~~~~~~~~
838
839The kgdboc driver is actually a very thin driver that relies on the
840underlying low level to the hardware driver having "polling hooks" to
841which the tty driver is attached. In the initial implementation of
842kgdboc the serial_core was changed to expose a low level UART hook for
843doing polled mode reading and writing of a single character while in an
844atomic context. When kgdb makes an I/O request to the debugger, kgdboc
845invokes a callback in the serial core which in turn uses the callback in
846the UART driver.
847
848When using kgdboc with a UART, the UART driver must implement two
849callbacks in the :c:type:`struct uart_ops <uart_ops>`.
850Example from ``drivers/8250.c``::
851
852
853 #ifdef CONFIG_CONSOLE_POLL
854 .poll_get_char = serial8250_get_poll_char,
855 .poll_put_char = serial8250_put_poll_char,
856 #endif
857
858
859Any implementation specifics around creating a polling driver use the
860``#ifdef CONFIG_CONSOLE_POLL``, as shown above. Keep in mind that
861polling hooks have to be implemented in such a way that they can be
862called from an atomic context and have to restore the state of the UART
863chip on return such that the system can return to normal when the
864debugger detaches. You need to be very careful with any kind of lock you
865consider, because failing here is most likely going to mean pressing the
866reset button.
867
868kgdboc and keyboards
869~~~~~~~~~~~~~~~~~~~~~~~~
870
871The kgdboc driver contains logic to configure communications with an
872attached keyboard. The keyboard infrastructure is only compiled into the
873kernel when ``CONFIG_KDB_KEYBOARD=y`` is set in the kernel configuration.
874
875The core polled keyboard driver for PS/2 type keyboards is in
876``drivers/char/kdb_keyboard.c``. This driver is hooked into the debug core
877when kgdboc populates the callback in the array called
878:c:type:`kdb_poll_funcs[]`. The kdb_get_kbd_char() is the top-level
879function which polls hardware for single character input.
880
881kgdboc and kms
882~~~~~~~~~~~~~~~~~~
883
884The kgdboc driver contains logic to request the graphics display to
885switch to a text context when you are using ``kgdboc=kms,kbd``, provided
886that you have a video driver which has a frame buffer console and atomic
887kernel mode setting support.
888
889Every time the kernel debugger is entered it calls
890kgdboc_pre_exp_handler() which in turn calls con_debug_enter()
891in the virtual console layer. On resuming kernel execution, the kernel
892debugger calls kgdboc_post_exp_handler() which in turn calls
893con_debug_leave().
894
895Any video driver that wants to be compatible with the kernel debugger
896and the atomic kms callbacks must implement the ``mode_set_base_atomic``,
897``fb_debug_enter`` and ``fb_debug_leave operations``. For the
898``fb_debug_enter`` and ``fb_debug_leave`` the option exists to use the
899generic drm fb helper functions or implement something custom for the
900hardware. The following example shows the initialization of the
901.mode_set_base_atomic operation in
902drivers/gpu/drm/i915/intel_display.c::
903
904
905 static const struct drm_crtc_helper_funcs intel_helper_funcs = {
906 [...]
907 .mode_set_base_atomic = intel_pipe_set_base_atomic,
908 [...]
909 };
910
911
912Here is an example of how the i915 driver initializes the
913fb_debug_enter and fb_debug_leave functions to use the generic drm
914helpers in ``drivers/gpu/drm/i915/intel_fb.c``::
915
916
917 static struct fb_ops intelfb_ops = {
918 [...]
919 .fb_debug_enter = drm_fb_helper_debug_enter,
920 .fb_debug_leave = drm_fb_helper_debug_leave,
921 [...]
922 };
923
924
925Credits
926=======
927
928The following people have contributed to this document:
929
9301. Amit Kale <amitkale@linsyssoft.com>
931
9322. Tom Rini <trini@kernel.crashing.org>
933
934In March 2008 this document was completely rewritten by:
935
936- Jason Wessel <jason.wessel@windriver.com>
937
938In Jan 2010 this document was updated to include kdb.
939
940- Jason Wessel <jason.wessel@windriver.com>