KGDB --- a remote gdb stub for SunOS kernel debugging ----------------------------------------------------- by John Heidemann v1.8/12-Sep-94 Executive Summary ----------------- This is a source-level kernel debugger for SunOS 4.1 based on GNU gdb (versions 4.7 through 4.11, not 4.13 or later) and Sun's kadb. It requires Sun kernel source code to build and two machines to operate. See the end of this file for a list of changes between kgdb releases. Installation ------------ 0. Read the rest of this file. 1. Get binaries for the appropriate Sun kernel modules described below in ``What Is Not Here''. 2. Edit the Makefile according to your local configuration. 3. Patch your version of gdb with one of the included patches; build and install it. See the GDB documentation for detailed instructions. 3. Build kgdb and bsd_strip by typing: make 4. Configure and build a debugging kernel, patching the Makefile as described below in ``Building A Debugging Kernel''. 5. To use kgdb, see ``Using Kgdb'' below. The most recent version of kgdb can be obtained by anonymous ftp to ftp.cs.ucla.edu in the directory pub/ficus/johnh. What Is Here ------------ These files contain part of a remote gdb stub (analogous to kadb) to allow gdb to perform remote kernel debugging. They also contain bsd_strip which is needed to build runnable SunOS debugging kernels. What Is Not Here ---------------- The (physically) largest piece of missing code is gdb. You will need to acquire gdb 4.8 and apply the patches (in gdb.4.8.patch) to it to serve as the main debugger. Gdb is available by anonymous ftp from prep.ai.mit.edu, among other places. Also, the included code provides only the gdb-specific portions of this debugging stub. To use it you will need to link with low-level routines and startup code provided with your SunOS kernel source release. Required .o's and libraries are: /sys/boot/sun4c/libkadb.a /sys/boot/lib/sun4c/libkadb.a /sys/sun4c/debug/locore.o /sys/sun4c/debug/machdep.o /sys/sun4c/debug/map.o /sys/sun4c/debug/clock.o /sys/debug/main.o /sys/debug/support.o You should compile these routines and adjust the kgdb Makefile before trying to build kgdb. Building A Debugging Kernel --------------------------- Configure a SunOS kernel as normal. Before building any .o's, you'll need to patch the kernel's Makefile to make it compile for debugging. Two changes are required. Change the line: CFLAGS=-O ${COPTS} to: CFLAGS=-g ${COPTS} And add to the ``vmunix:'' target after ``@chmod 755 vmunix'': @echo making debugging kernel @cp vmunix vmunix.gdb @/usr/dist/bin/bsd_strip -d vmunix There will now be two kernels, one with debugging symbols (vmunix.gdb) and one without (vmunix). Vmunix.gdb contains symbolic debugging information for use by gdb. Vmunix lacks this bulky information but can be booted by a debugging slave machine (which finds the size of a kernel with debugging symbols too hefty for its tastes). Using Kgdb ---------- Remote debugging requires two machines: a master and a slave. Gdb runs on the master, controlling the slave's kernel via a serial connection. First, start gdb on the master. Cd to the build directory and type ``gdb ./vmunix.gdb''. Next boot kgdb on your slave machine. Kgdb will load your debugging kernel (configured as described above) on the slave. When it comes up, you can L1-A and it should say something like: kgdb: waiting for debugger. stopped at ffc01048 At that point, you need to attach the gdb you've already started to the slave. The gdb prompt will be ``(gdb)''. Type ``target remote /dev/MASTER_SIDE_SERIAL_PORT_DEVICE''. The slave will say: Remote debugging using /dev/whatever You can then set breakpoints and use all gdb features. Remote debugging can be slow. (This is a combination of the mechanism required to do breakpoints on the SPARC architecture and partly that serial communication is fairly slow.) To make debugging work faster, minimize the number of breakpoints you keep active. If you wish to stop debugging, you must gracefully shut down gdb. On the slave, do L1-A to return control to gdb. Remove any breakpoints you have set, then type ``detach''. Gdb will release the kernel. Alternatively, the gdb ``kill'' command will drop the slave into the monitor PROM, from which you can reboot. If your kernel has crashed but you do not wish to go to the trouble of starting up gdb, you can simply type control-C on the slave at the "kgdb: waiting for debugger" prompt and it will drop into the monitor. Alternatively, if you did L1-A and you just want to continue without starting up gdb, type control-Q. Caveats ------- This version has only been tested with SunOS 4.1.1 and the sun4c kernel architecture. It requires PROM version 2.0 or better. If you get it working on other releases, please let me know. The code assumes that debugging information will be carried at 9600 baud through serial port ``b''. To change this, patch kgdb_device in kgdb_utils.c. Common Problems --------------- Missing header files during kgdb make: This problem can occur if you haven't built gdb before you try to build kgdb. Some gdb header files are constructed during the gdb build process and are missing until after gdb has been built. This problem can also result from compiling kgdb against different versions of gdb than I've tested. The gdb headers often move around. If this is the case you must either match kgdb/gdb versions o adjust the compiler include path (INCS) in the kgdb Makefile. Incompatible version of gdb: Kgdb currently works with gdb version 4.7 through 4.11. It does not work with more recent versions of gdb (like 4.13) because of incompatible changes to the serial-line protocol. At some point in the future I will probably move kgdb forward, but not necessarily soon (I want to get my degree and 4.11 works fine for us here). As a work-around, a pre-patched version of gdb-4.11 is available as ftp://ftp.cs.ucla.edu/pub/ficus/johnh/patched-gdb-4.11.tar.gz. Credits ------- This work was inspired by the CSRG at Berkeley and their kernel debugger in BSD 4.4. The bsd_strip program is their strip, verbatim. Thanks guys! Gdb is great. What more can I say. Thanks to the Gnu people. Michael Tuciarone of Auspex gave me some great advice about how to do serial I/O from a stand-alone program. Thanks, Tooch. I have talked to Peter van der Linden at Sun who was (at one time) involved with their kdbx about doing debugging over the Ethernet. Dante De Lucia and David Peterson have a prototype of an older version of kgdb which directly accesses the network; their work is not integrated with this release. Bugs and To Do's ---------------- Some users have experienced problems of incorrect reporting of local function variables. This problem should now be fixed (as of v1.3p2). Let me know if it this problem re-occurs. The current patches to gdb are gross. A new target (sunos4kern) should be created to do thing the Right way. Anyone who wants to send me a clean set of patches fixing this would be appreciated. It'd be a lot nicer if debugging packets went out over the Ethernet rather than the serial ports. I'm interested in hearing about the Right way to do this from a stand-alone program. As described in the ``Credits'' section, Dante De Lucia and David Peterson have done some work in this area, but I don't have time to merge the versions (I'd really like to graduate). Parties interested in continuing with this work should contact me. What's New With Release 1.4 --------------------------- Release 1.4 includes patches against gdb 4.8 instead of gdb 4.7. Also, control-C will now cause the kgdb stub to drop into the monitor from the "kgdb: waiting for debugger" prompt. No other significant changes have been made. What's New With Release 1.5 --------------------------- There are times when you want to be able to halt and resume a machine without necessarily bothering to bring up a full fledged gdb process. Control-Q from the "waiting for debugger" prompt will now allow you to simply continue execution; no gdb process is required. What's New With Release 1.6 --------------------------- This version incorporates code from Geoff Kuenning to directly read and write the serial chips. This *significantly* speeds up remote debugging (to the point where emacs' gdb mode is reasonable to use). Thank you, Geoff! The kioecho protocol has been removed from the gdb.4.x.patch file; Geoff's changes make it unnecessary. Patches are included for most gdb releases through 4.11. Release 1.6 kgdb also handles interrupt characters arriving over the debugging line and resets to a known state. What's New With Release 1.7 --------------------------- This version adds the header file kgdb.h, missing from release 1.6. What's New With Release 1.8 --------------------------- Information added about problems with gdb-4.13. The Makefile now builds tar and tar.gz distributions with the standard format (all files in a ./kgdb-$VERSION subdirectory). Ispell Wordlist --------------- % LocalWords: mit chmod dist Auspex Sep SPARC INCS der kdbx sunos kioecho sparc % LocalWords: KGDB gdb kadb kgdb bsd ftp ai libkadb RockyMountain rahul gz % LocalWords: lib locore machdep CFLAGS COPTS vmunix Ispell Wordlist % LocalWords: cp usr slave's Cd ffc dev utils CSRG % LocalWords: Tuciarone Tooch Do's LocalWords Conclusion ---------- Good luck. If you find this useful, drop me a postcard. I know *I* don't plan on going back to kadb. -John Heidemann 4860E Boelter Hall UCLA Los Angeles, CA 90024