date:20200501

Re: Symbol debugging support for kernel modules in crash dumps

2020-05-01 Thread Paul Goyette


Very cool!

On Fri, 1 May 2020, Christos Zoulas wrote:



Hi,

I just added symbol debugging support for modules in kernel dumps.
Things are not perfect because of what I call "current thread
confusion" in the kvm target, but as you see in the following
session it works just fine if you follow the right steps. First of
all you need a build from HEAD that has the capability to build
.debug files for kernel modules.  Once that's done, you are all
set; see how it works (comments prefixed by )

Enjoy,

christos

$ gdb /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb
GNU gdb (GDB) 8.3
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64--netbsd".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
   .

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb...
(gdb) target kvm netbsd.22.core
0x80224375 in cpu_reboot (howto=howto@entry=260,
   bootstr=bootstr@entry=0x0) at ../../../../arch/amd64/amd64/machdep.c:718
warning: Source file is more recent than executable.
718 if (s != IPL_NONE)

 Ok we got a stacktrace here, but we don't have a current thread...
 So we set it...

(gdb) info thread
 Id   Target Id Frame
* 2.1   0x80224375 in cpu_reboot (
   howto=howto@entry=260, bootstr=bootstr@entry=0x0)
   at ../../../../arch/amd64/amd64/machdep.c:718

No selected thread.  See `help thread'.
(gdb) thread 2.1

[Switching to thread 2.1 ()]
#0  0x80224375 in ?? ()

 Note that here we lost all symbol table access when we switched threads
 let's load it again..

(gdb) add-symbol-file /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb
add symbol table from file "/usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb"
(y or n) y
Reading symbols from /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb...

 OK, lets load our modules

(gdb) source /usr/src/sys/gdbscripts/modload
(gdb) modload
add symbol table from file "/stand/amd64/9.99.59/modules/ping/ping.kmod" at
   .text_addr = 0x8266e000
   .data_addr = 0x8266b000
   .rodata_addr = 0x8266c000
add symbol table from file 
"/stand/amd64/9.99.59/modules/nfsserver/nfsserver.kmod" at
   .text_addr = 0x82a64000
   .data_addr = 0x82669000
   .rodata_addr = 0x8298e000
add symbol table from file 
"/stand/amd64/9.99.59/modules/npf_ext_log/npf_ext_log.kmod" at
   .text_addr = 0x82668000
   .data_addr = 0x82667000
   .rodata_addr = 0x82969000
add symbol table from file 
"/stand/amd64/9.99.59/modules/npf_alg_icmp/npf_alg_icmp.kmod" at
   .text_addr = 0x82666000
   .data_addr = 0x82665000
   .rodata_addr = 0x82952000
add symbol table from file "/stand/amd64/9.99.59/modules/bpfjit/bpfjit.kmod" at
   .text_addr = 0x82661000
   .data_addr = 0x0
   .rodata_addr = 0x828dd000
add symbol table from file "/stand/amd64/9.99.59/modules/sljit/sljit.kmod" at
   .text_addr = 0x82945000
   .data_addr = 0x82664000
   .rodata_addr = 0x828f9000
add symbol table from file 
"/stand/amd64/9.99.59/modules/if_npflog/if_npflog.kmod" at
   .text_addr = 0x8266
   .data_addr = 0x8265f000
   .rodata_addr = 0x828ca000
add symbol table from file "/stand/amd64/9.99.59/modules/npf/npf.kmod" at
   .text_addr = 0x82648000
   .data_addr = 0x82647000
   .rodata_addr = 0x826d6000
add symbol table from file "/stand/amd64/9.99.59/modules/bpf/bpf.kmod" at
   .text_addr = 0x82622000
   .data_addr = 0x82621000
   .rodata_addr = 0x826a3000
add symbol table from file 
"/stand/amd64/9.99.59/modules/bpf_filter/bpf_filter.kmod" at
   .text_addr = 0x8263c000
   .data_addr = 0x0
   .rodata_addr = 0x82627000
add symbol table from file 
"/stand/amd64/9.99.59/modules/scsiverbose/scsiverbose.kmod" at
   .text_addr = 0x826a2000
   .data_addr = 0x82686000
   .rodata_addr = 0x82687000
add symbol table from file 
"/stand/amd64/9.99.59/modules/usbverbose/usbverbose.kmod" at
   .text_addr = 0x82685000
   .data_addr = 0x8267
   .rodata_addr = 0x82671000
add symbol table from file 
"/stand/amd64/9.99.59/modules/miiverbose/miiverbose.kmod" at
   .text_addr =

Re: Symbol debugging support for kernel modules in crash dumps

2020-05-01 Thread David Young

Fantastic! Thanks.

Dave

Spilling kerrectud by iPhone

> On May 1, 2020, at 6:34 PM, Christos Zoulas  wrote:
> 
> 
> Hi,
> 
> I just added symbol debugging support for modules in kernel dumps.
> Things are not perfect because of what I call "current thread
> confusion" in the kvm target, but as you see in the following
> session it works just fine if you follow the right steps. First of
> all you need a build from HEAD that has the capability to build
> .debug files for kernel modules.  Once that's done, you are all
> set; see how it works (comments prefixed by )
> 
> Enjoy,
> 
> christos
> 
> $ gdb /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb
> GNU gdb (GDB) 8.3
> Copyright (C) 2019 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> Type "show copying" and "show warranty" for details.
> This GDB was configured as "x86_64--netbsd".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> .
> Find the GDB manual and other documentation resources online at:
>.
> 
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb...
> (gdb) target kvm netbsd.22.core
> 0x80224375 in cpu_reboot (howto=howto@entry=260, 
>bootstr=bootstr@entry=0x0) at ../../../../arch/amd64/amd64/machdep.c:718
> warning: Source file is more recent than executable.
> 718 if (s != IPL_NONE)
> 
>  Ok we got a stacktrace here, but we don't have a current thread...
>  So we set it...
> 
> (gdb) info thread
>  Id   Target Id Frame 
> * 2.1   0x80224375 in cpu_reboot (
>howto=howto@entry=260, bootstr=bootstr@entry=0x0)
>at ../../../../arch/amd64/amd64/machdep.c:718
> 
> No selected thread.  See `help thread'.
> (gdb) thread 2.1
> 
> [Switching to thread 2.1 ()]
> #0  0x80224375 in ?? ()
> 
>  Note that here we lost all symbol table access when we switched threads
>  let's load it again..
> 
> (gdb) add-symbol-file /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb
> add symbol table from file "/usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb"
> (y or n) y
> Reading symbols from /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb...
> 
>  OK, lets load our modules
> 
> (gdb) source /usr/src/sys/gdbscripts/modload 
> (gdb) modload
> add symbol table from file "/stand/amd64/9.99.59/modules/ping/ping.kmod" at
>.text_addr = 0x8266e000
>.data_addr = 0x8266b000
>.rodata_addr = 0x8266c000
> add symbol table from file 
> "/stand/amd64/9.99.59/modules/nfsserver/nfsserver.kmod" at
>.text_addr = 0x82a64000
>.data_addr = 0x82669000
>.rodata_addr = 0x8298e000
> add symbol table from file 
> "/stand/amd64/9.99.59/modules/npf_ext_log/npf_ext_log.kmod" at
>.text_addr = 0x82668000
>.data_addr = 0x82667000
>.rodata_addr = 0x82969000
> add symbol table from file 
> "/stand/amd64/9.99.59/modules/npf_alg_icmp/npf_alg_icmp.kmod" at
>.text_addr = 0x82666000
>.data_addr = 0x82665000
>.rodata_addr = 0x82952000
> add symbol table from file "/stand/amd64/9.99.59/modules/bpfjit/bpfjit.kmod" 
> at
>.text_addr = 0x82661000
>.data_addr = 0x0
>.rodata_addr = 0x828dd000
> add symbol table from file "/stand/amd64/9.99.59/modules/sljit/sljit.kmod" at
>.text_addr = 0x82945000
>.data_addr = 0x82664000
>.rodata_addr = 0x828f9000
> add symbol table from file 
> "/stand/amd64/9.99.59/modules/if_npflog/if_npflog.kmod" at
>.text_addr = 0x8266
>.data_addr = 0x8265f000
>.rodata_addr = 0x828ca000
> add symbol table from file "/stand/amd64/9.99.59/modules/npf/npf.kmod" at
>.text_addr = 0x82648000
>.data_addr = 0x82647000
>.rodata_addr = 0x826d6000
> add symbol table from file "/stand/amd64/9.99.59/modules/bpf/bpf.kmod" at
>.text_addr = 0x82622000
>.data_addr = 0x82621000
>.rodata_addr = 0x826a3000
> add symbol table from file 
> "/stand/amd64/9.99.59/modules/bpf_filter/bpf_filter.kmod" at
>.text_addr = 0x8263c000
>.data_addr = 0x0
>.rodata_addr = 0x82627000
> add symbol table from file 
> "/stand/amd64/9.99.59/modules/scsiverbose/scsiverbose.kmod" at
>.text_addr = 0x826a2000
>.data_addr = 0x82686000
>.rodata_addr = 0x82687000
> add symbol table from file 
>

Symbol debugging support for kernel modules in crash dumps

2020-05-01 Thread Christos Zoulas



Hi,

I just added symbol debugging support for modules in kernel dumps.
Things are not perfect because of what I call "current thread
confusion" in the kvm target, but as you see in the following
session it works just fine if you follow the right steps. First of
all you need a build from HEAD that has the capability to build
.debug files for kernel modules.  Once that's done, you are all
set; see how it works (comments prefixed by )

Enjoy,

christos

$ gdb /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb
GNU gdb (GDB) 8.3
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64--netbsd".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb...
(gdb) target kvm netbsd.22.core
0x80224375 in cpu_reboot (howto=howto@entry=260, 
bootstr=bootstr@entry=0x0) at ../../../../arch/amd64/amd64/machdep.c:718
warning: Source file is more recent than executable.
718 if (s != IPL_NONE)

 Ok we got a stacktrace here, but we don't have a current thread...
 So we set it...

(gdb) info thread
  Id   Target Id Frame 
* 2.1   0x80224375 in cpu_reboot (
howto=howto@entry=260, bootstr=bootstr@entry=0x0)
at ../../../../arch/amd64/amd64/machdep.c:718

No selected thread.  See `help thread'.
(gdb) thread 2.1

[Switching to thread 2.1 ()]
#0  0x80224375 in ?? ()

 Note that here we lost all symbol table access when we switched threads
 let's load it again..

(gdb) add-symbol-file /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb
add symbol table from file "/usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb"
(y or n) y
Reading symbols from /usr/src/sys/arch/amd64/compile/QUASAR/netbsd.gdb...

 OK, lets load our modules

(gdb) source /usr/src/sys/gdbscripts/modload 
(gdb) modload
add symbol table from file "/stand/amd64/9.99.59/modules/ping/ping.kmod" at
.text_addr = 0x8266e000
.data_addr = 0x8266b000
.rodata_addr = 0x8266c000
add symbol table from file 
"/stand/amd64/9.99.59/modules/nfsserver/nfsserver.kmod" at
.text_addr = 0x82a64000
.data_addr = 0x82669000
.rodata_addr = 0x8298e000
add symbol table from file 
"/stand/amd64/9.99.59/modules/npf_ext_log/npf_ext_log.kmod" at
.text_addr = 0x82668000
.data_addr = 0x82667000
.rodata_addr = 0x82969000
add symbol table from file 
"/stand/amd64/9.99.59/modules/npf_alg_icmp/npf_alg_icmp.kmod" at
.text_addr = 0x82666000
.data_addr = 0x82665000
.rodata_addr = 0x82952000
add symbol table from file "/stand/amd64/9.99.59/modules/bpfjit/bpfjit.kmod" at
.text_addr = 0x82661000
.data_addr = 0x0
.rodata_addr = 0x828dd000
add symbol table from file "/stand/amd64/9.99.59/modules/sljit/sljit.kmod" at
.text_addr = 0x82945000
.data_addr = 0x82664000
.rodata_addr = 0x828f9000
add symbol table from file 
"/stand/amd64/9.99.59/modules/if_npflog/if_npflog.kmod" at
.text_addr = 0x8266
.data_addr = 0x8265f000
.rodata_addr = 0x828ca000
add symbol table from file "/stand/amd64/9.99.59/modules/npf/npf.kmod" at
.text_addr = 0x82648000
.data_addr = 0x82647000
.rodata_addr = 0x826d6000
add symbol table from file "/stand/amd64/9.99.59/modules/bpf/bpf.kmod" at
.text_addr = 0x82622000
.data_addr = 0x82621000
.rodata_addr = 0x826a3000
add symbol table from file 
"/stand/amd64/9.99.59/modules/bpf_filter/bpf_filter.kmod" at
.text_addr = 0x8263c000
.data_addr = 0x0
.rodata_addr = 0x82627000
add symbol table from file 
"/stand/amd64/9.99.59/modules/scsiverbose/scsiverbose.kmod" at
.text_addr = 0x826a2000
.data_addr = 0x82686000
.rodata_addr = 0x82687000
add symbol table from file 
"/stand/amd64/9.99.59/modules/usbverbose/usbverbose.kmod" at
.text_addr = 0x82685000
.data_addr = 0x8267
.rodata_addr = 0x82671000
add symbol table from file 
"/stand/amd64/9.99.59/modules/miiverbose/miiverbose.kmod" at
.text_addr = 0x82646000

getrandom and getentropy

2020-05-01 Thread Taylor R Campbell

[please followup to tech-userlevel@ to keep discussion in one place]

Traditionally NetBSD has implemented three ways to get unpredictable
independent uniform random bytes out of the kernel's entropy pool:

- /dev/urandom -- never blocks, uses whatever is in the entropy pool,
  returns short reads only if interrupted

- /dev/random -- may block, may be truncated to 16 bytes or otherwise
  short reads; if system has partial entropy, blocks until full
  entropy; traditionally also sometimes blocks randomly at other times
  according to the obsolete `entropy depletion' semantics

- sysctl kern.arandom -- never blocks, uses whatever is in the entropy
  pool like /dev/urandom, returns up to 256 bytes.  (There's also
  kern.urandom, but it only provides 32 bits at a time and to my
  knowledge has never been documented.)

I propose that we additionally adopt getrandom and getentropy, two C
APIs the world is converging on.  For getentropy, this can be a simple
userland wrapper in libc; for getrandom, this requires a new path into
the kernel, and I propose that we adopt getrandom as both a system
call and a public libc stub, like Linux/glibc and FreeBSD both did.

The getentropy and getrandom APIs, described below, may not be
perfect, and for several years I was reluctant to adopt them, but I've
come to the conclusion that adopting them is a good idea, for reasons
below, and that there's no clearly better alternative ways to achieve
the same desiderata.


DETAILS

The usage model for getentropy, from OpenBSD [1], is:

if (getentropy(buf, buflen) == -1)
err(...);

The semantics is essentially the same as sysctl kern.arandom: never
blocks, uses whatever is in the entropy pool like /dev/urandom,
returns up to 256 bytes (fails with EIO if buflen>256).  Once the
system entropy pool is seeded, this is good, although it provides no
way to wait until the entropy pool is seeded.

getrandom, from Linux [2], has three usage models:

/*
 * Recommended default usage -- may block once at boot time;
 * otherwise never blocks.  Limited only to 33554431 bytes.
 */
if (getrandom(buf, buflen, 0) == -1)
err(...);

/*
 * Never blocks, like /dev/urandom, sysctl kern.arandom, and
 * getentropy.  Limited only to 33554431 bytes.  Called
 * GRND_INSECURE because it is safe only after you have
 * already verified the system entropy pool is seeded -- e.g.,
 * by calling getrandom(0,0,0) once or reading a single byte
 * from /dev/random; if you _don't_ wait for the pool to be
 * seeded, then it can ruin the security of the entropy pool
 * for _everyone_.
 */
if (getrandom(buf, buflen, GRND_INSECURE) == -1)
err(...);

/*
 * May block, may return very short reads, like /dev/random;
 * limited to 512 bytes.  NOT RECOMMENDED -- provided only for
 * source compatibility with applications written for Linux,
 * FreeBSD, or Solaris.
 */
nread = getrandom(buf, buflen, GRND_RANDOM);
if (nread == -1)
err(...);
if ((size_t)nread < buflen)
/* start over and try to read some more, maybe */

I drafted a man page for getrandom that I think is considerably
clearer than the Linux manual; you can read it here:
 (the
mdoc source is also in the attached patch).

The option GRND_NONBLOCK may also be bitwise-OR'd into the third
argument to make it fail with EAGAIN rather than blocking in the event
it would have blocked.  For example, `if (getrandom(0,0,GRND_NONBLOCK)
== -1 && errno == EAGAIN) ...' serves to test whether the entropy pool
is not yet seeded.


RATIONALE

### Why adopt anything?  Why not leave things as they are?

1. getrandom and getentropy are easier to use than /dev/u?random or a
   sysctl, and a lot of software has already adopted them to make
   getting cryptographic key material much simpler than it was in the
   past.  They serve a real need in practice.

2. Like sysctl kern.arandom, getrandom and getentropy don't need /dev
   to be populated (so they can be used in an empty chroot) and don't
   require opening and closing a file descriptor.

   I see very little value in using the file system to control access
   to the random number generator -- after all, the process can always
   use sysctl kern.arandom.  I see much greater value in making it
   more easily accessible to applications to reduce the temptation to
   use garbage like srand(time()).

3. getrandom and getentropy are more portable than sysctl
   kern.arandom:

   . Linux/glibc [2], FreeBSD [3], and Solaris (Oracle [4] and illumos
 [5]) all support getrandom.

   . glibc [6], OpenBSD [1], FreeBSD [7], and Oracle Solaris [4] (not
 sure about illumos) all support getentropy.

   Only NetBSD still supports kern.arandom, to my

Re: Symbol debugging support for kernel modules in crash dumps

Re: Symbol debugging support for kernel modules in crash dumps

Symbol debugging support for kernel modules in crash dumps

getrandom and getentropy

4 matches

Site Navigation

Mail list logo

Footer information