Re: kernel panic by enabling net.inet.ip.random_id

[Gleb Smirnoff]

On Wed, Jan 06, 2016 at 01:24:53PM -0500, Shawn Webb wrote:
S> That's what gets toggled via the sysctl. I think I figured out that I
S> need to call ip_initid() in the SYSINIT. Compiling and testing now.

Right, that's the problem. You actually want VNET_SYSINIT().

Please next time you report a panic, don't say "I'm on latest HEAD on
amd64 in bhyve" when you are actually running a modified version. It
took me about 10 minutes pondering into your backtrace in initial
post and into ip_id.c. Then I went on reading thread further and
discovered that I was reading wrong ip_id.c

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: kernel panic by enabling net.inet.ip.random_id

[Shawn Webb]

(kgdb) list *(0x80b5de9e)
0x80b5de9e is in ip_fillid (/usr/src/sys/netinet/ip_id.c:237).
warning: Source file is more recent than executable.

232 new_id = 0;
233 do {
234 if (new_id != 0)
235 V_random_id_collisions++;
236 arc4rand(_id, sizeof(new_id), 0);
237 } while (bit_test(V_id_bits, new_id) || new_id == 0);
238 bit_clear(V_id_bits, V_id_array[V_array_ptr]);
239 bit_set(V_id_bits, new_id);
240 V_id_array[V_array_ptr] = new_id;
241 V_array_ptr++;

This is the change I made to ip_id.c that caused the underlying kernel
panic:
https://github.com/HardenedBSD/hardenedBSD/commit/52d5a93b92097e7a79be8d2e0eb9c1a58b8337d1

Ideally, we should be able to just toggle that variable and all would be
well. But it seems with the VIMAGE work, something is preventing that.

Thanks,

Shawn

On Tue, Jan 05, 2016 at 06:22:34PM -0800, Adrian Chadd wrote:
> try list *(0x[address]) .
> 
> That line is mtx_unlock(), which makes no sense (as mtx_lock succeeded fine.)
> 
> 
> -a
> 
> 
> On 5 January 2016 at 18:13, Shawn Webb  wrote:
> > Thanks for the quick reply! Here's some more debugging output:
> >
> > === Begin Log ===
> > (kgdb) bt
> > #0  doadump (textdump=0) at pcpu.h:221
> > #1  0x8037c78b in db_dump (dummy=, 
> > dummy2=false, dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
> > #2  0x8037c57e in db_command (cmd_table=0x0) at 
> > /usr/src/sys/ddb/db_command.c:440
> > #3  0x8037c314 in db_command_loop () at 
> > /usr/src/sys/ddb/db_command.c:493
> > #4  0x8037edab in db_trap (type=, code=0) at 
> > /usr/src/sys/ddb/db_main.c:251
> > #5  0x80a5c563 in kdb_trap (type=12, code=0, tf= > out>) at /usr/src/sys/kern/subr_kdb.c:654
> > #6  0x80e6b7e1 in trap_fatal (frame=0xfe02c33894d0, eva= > optimized out>) at /usr/src/sys/amd64/amd64/trap.c:829
> > #7  0x80e6ba2d in trap_pfault (frame=0xfe02c33894d0, 
> > usermode=) at /usr/src/sys/amd64/amd64/trap.c:684
> > #8  0x80e6b15f in trap (frame=0xfe02c33894d0) at 
> > /usr/src/sys/amd64/amd64/trap.c:435
> > #9  0x80e4af97 in calltrap () at 
> > /usr/src/sys/amd64/amd64/exception.S:234
> > #10 0x80b5de9e in ip_fillid (ip=0xf8000ef8cb88) at 
> > /usr/src/sys/netinet/ip_id.c:237
> > #11 0x80b6c41b in ip_output (m=, opt= > optimized out>, ro=, flags=0, imo=0x0, 
> > inp=0xf8000e66e960) at /usr/src/sys/netinet/ip_output.c:268
> > #12 0x80bf0612 in udp_send (so=, flags= > optimized out>, m=, addr=0x0, control= > out>, td=0xf8000ef8cb88) at /usr/src/sys/netinet/udp_usrreq.c:1517
> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
> > uio=, top=0xf8000ef8cb00, control=0x0, 
> > flags=, td=0x81bef2ec) at 
> > /usr/src/sys/kern/uipc_socket.c:1164
> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
> > uio=, top=0xf8000ef8cb00, control=0x0, 
> > flags=, td=0x81bef2ec) at 
> > /usr/src/sys/kern/uipc_socket.c:1164
> > #14 0x80aaa03b in kern_sendit (td=0xf8000e4cd9c0, s=6, 
> > mp=, flags=0, control=0x0, segflg=UIO_USERSPACE) at 
> > /usr/src/sys/kern/uipc_syscalls.c:906
> > #15 0x80aaa336 in sendit (td=0xf8000e4cd9c0, s= > out>, mp=0xfe02c3389970, flags=3980) at 
> > /usr/src/sys/kern/uipc_syscalls.c:833
> > #16 0x80aaa1fd in sys_sendto (td=0x0, uap=) at 
> > /usr/src/sys/kern/uipc_syscalls.c:957
> > #17 0x80e6bfdb in amd64_syscall (td=0xf8000e4cd9c0, traced=0) 
> > at subr_syscall.c:135
> > #18 0x80e4b27b in Xfast_syscall () at 
> > /usr/src/sys/amd64/amd64/exception.S:394
> > #19 0x03e339782e8a in ?? ()
> > (kgdb) x/i 0x80b5de9e
> > 0x80b5de9e : movzbl (%rax,%rcx,1),%esi
> > (kgdb) info reg
> > rax0x0  0
> > rbx0x0  0
> > rcx0x0  0
> > rdx0x0  0
> > rsi0x0  0
> > rdi0x0  0
> > rbp0xfe02c3388fe0   0xfe02c3388fe0
> > rsp0xfe02c3388fc8   0xfe02c3388fc8
> > r8 0x0  0
> > r9 0x0  0
> > r100x0  0
> > r110x0  0
> > r120x817c0b80   -2122577024
> > r130x817c1470   -2122574736
> > r140x1  1
> > r150x4  4
> > rip0x80a1fae3   0x80a1fae3 
> > eflags 0x0  0
> > cs 0x0  0
> > ss 0x0  0
> > ds 0x0  0
> > es 0x0  0
> > fs 0x0  0
> > gs 0x0  0
> > === End Log ===
> >
> > Thanks,
> >
> > Shawn
> >
> > On Tue, Jan 05, 2016 at 06:06:41PM -0800, Adrian Chadd wrote:
> >> looks like a null 

Re: kernel panic by enabling net.inet.ip.random_id

[Adrian Chadd]

Why'd you condition the vimage definition? :)



-a


On 6 January 2016 at 06:46, Shawn Webb  wrote:
> (kgdb) list *(0x80b5de9e)
> 0x80b5de9e is in ip_fillid (/usr/src/sys/netinet/ip_id.c:237).
> warning: Source file is more recent than executable.
>
> 232 new_id = 0;
> 233 do {
> 234 if (new_id != 0)
> 235 V_random_id_collisions++;
> 236 arc4rand(_id, sizeof(new_id), 0);
> 237 } while (bit_test(V_id_bits, new_id) || new_id == 0);
> 238 bit_clear(V_id_bits, V_id_array[V_array_ptr]);
> 239 bit_set(V_id_bits, new_id);
> 240 V_id_array[V_array_ptr] = new_id;
> 241 V_array_ptr++;
>
> This is the change I made to ip_id.c that caused the underlying kernel
> panic:
> https://github.com/HardenedBSD/hardenedBSD/commit/52d5a93b92097e7a79be8d2e0eb9c1a58b8337d1
>
> Ideally, we should be able to just toggle that variable and all would be
> well. But it seems with the VIMAGE work, something is preventing that.
>
> Thanks,
>
> Shawn
>
> On Tue, Jan 05, 2016 at 06:22:34PM -0800, Adrian Chadd wrote:
>> try list *(0x[address]) .
>>
>> That line is mtx_unlock(), which makes no sense (as mtx_lock succeeded fine.)
>>
>>
>> -a
>>
>>
>> On 5 January 2016 at 18:13, Shawn Webb  wrote:
>> > Thanks for the quick reply! Here's some more debugging output:
>> >
>> > === Begin Log ===
>> > (kgdb) bt
>> > #0  doadump (textdump=0) at pcpu.h:221
>> > #1  0x8037c78b in db_dump (dummy=, 
>> > dummy2=false, dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
>> > #2  0x8037c57e in db_command (cmd_table=0x0) at 
>> > /usr/src/sys/ddb/db_command.c:440
>> > #3  0x8037c314 in db_command_loop () at 
>> > /usr/src/sys/ddb/db_command.c:493
>> > #4  0x8037edab in db_trap (type=, code=0) at 
>> > /usr/src/sys/ddb/db_main.c:251
>> > #5  0x80a5c563 in kdb_trap (type=12, code=0, tf=> > out>) at /usr/src/sys/kern/subr_kdb.c:654
>> > #6  0x80e6b7e1 in trap_fatal (frame=0xfe02c33894d0, eva=> > optimized out>) at /usr/src/sys/amd64/amd64/trap.c:829
>> > #7  0x80e6ba2d in trap_pfault (frame=0xfe02c33894d0, 
>> > usermode=) at /usr/src/sys/amd64/amd64/trap.c:684
>> > #8  0x80e6b15f in trap (frame=0xfe02c33894d0) at 
>> > /usr/src/sys/amd64/amd64/trap.c:435
>> > #9  0x80e4af97 in calltrap () at 
>> > /usr/src/sys/amd64/amd64/exception.S:234
>> > #10 0x80b5de9e in ip_fillid (ip=0xf8000ef8cb88) at 
>> > /usr/src/sys/netinet/ip_id.c:237
>> > #11 0x80b6c41b in ip_output (m=, opt=> > optimized out>, ro=, flags=0, imo=0x0, 
>> > inp=0xf8000e66e960) at /usr/src/sys/netinet/ip_output.c:268
>> > #12 0x80bf0612 in udp_send (so=, flags=> > optimized out>, m=, addr=0x0, control=> > optimized out>, td=0xf8000ef8cb88) at 
>> > /usr/src/sys/netinet/udp_usrreq.c:1517
>> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
>> > uio=, top=0xf8000ef8cb00, control=0x0, 
>> > flags=, td=0x81bef2ec) at 
>> > /usr/src/sys/kern/uipc_socket.c:1164
>> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
>> > uio=, top=0xf8000ef8cb00, control=0x0, 
>> > flags=, td=0x81bef2ec) at 
>> > /usr/src/sys/kern/uipc_socket.c:1164
>> > #14 0x80aaa03b in kern_sendit (td=0xf8000e4cd9c0, s=6, 
>> > mp=, flags=0, control=0x0, segflg=UIO_USERSPACE) at 
>> > /usr/src/sys/kern/uipc_syscalls.c:906
>> > #15 0x80aaa336 in sendit (td=0xf8000e4cd9c0, s=> > optimized out>, mp=0xfe02c3389970, flags=3980) at 
>> > /usr/src/sys/kern/uipc_syscalls.c:833
>> > #16 0x80aaa1fd in sys_sendto (td=0x0, uap=) 
>> > at /usr/src/sys/kern/uipc_syscalls.c:957
>> > #17 0x80e6bfdb in amd64_syscall (td=0xf8000e4cd9c0, traced=0) 
>> > at subr_syscall.c:135
>> > #18 0x80e4b27b in Xfast_syscall () at 
>> > /usr/src/sys/amd64/amd64/exception.S:394
>> > #19 0x03e339782e8a in ?? ()
>> > (kgdb) x/i 0x80b5de9e
>> > 0x80b5de9e : movzbl (%rax,%rcx,1),%esi
>> > (kgdb) info reg
>> > rax0x0  0
>> > rbx0x0  0
>> > rcx0x0  0
>> > rdx0x0  0
>> > rsi0x0  0
>> > rdi0x0  0
>> > rbp0xfe02c3388fe0   0xfe02c3388fe0
>> > rsp0xfe02c3388fc8   0xfe02c3388fc8
>> > r8 0x0  0
>> > r9 0x0  0
>> > r100x0  0
>> > r110x0  0
>> > r120x817c0b80   -2122577024
>> > r130x817c1470   -2122574736
>> > r140x1  1
>> > r150x4  4
>> > rip0x80a1fae3   0x80a1fae3 
>> > eflags 0x0  0
>> > cs 0x0  0
>> > ss   

Re: kernel panic by enabling net.inet.ip.random_id

[Shawn Webb]

That's what gets toggled via the sysctl. I think I figured out that I
need to call ip_initid() in the SYSINIT. Compiling and testing now.

Thanks,

Shawn

On Wed, Jan 06, 2016 at 10:15:08AM -0800, Adrian Chadd wrote:
> Why'd you condition the vimage definition? :)
> 
> 
> 
> -a
> 
> 
> On 6 January 2016 at 06:46, Shawn Webb  wrote:
> > (kgdb) list *(0x80b5de9e)
> > 0x80b5de9e is in ip_fillid (/usr/src/sys/netinet/ip_id.c:237).
> > warning: Source file is more recent than executable.
> >
> > 232 new_id = 0;
> > 233 do {
> > 234 if (new_id != 0)
> > 235 V_random_id_collisions++;
> > 236 arc4rand(_id, sizeof(new_id), 0);
> > 237 } while (bit_test(V_id_bits, new_id) || new_id == 0);
> > 238 bit_clear(V_id_bits, V_id_array[V_array_ptr]);
> > 239 bit_set(V_id_bits, new_id);
> > 240 V_id_array[V_array_ptr] = new_id;
> > 241 V_array_ptr++;
> >
> > This is the change I made to ip_id.c that caused the underlying kernel
> > panic:
> > https://github.com/HardenedBSD/hardenedBSD/commit/52d5a93b92097e7a79be8d2e0eb9c1a58b8337d1
> >
> > Ideally, we should be able to just toggle that variable and all would be
> > well. But it seems with the VIMAGE work, something is preventing that.
> >
> > Thanks,
> >
> > Shawn
> >
> > On Tue, Jan 05, 2016 at 06:22:34PM -0800, Adrian Chadd wrote:
> >> try list *(0x[address]) .
> >>
> >> That line is mtx_unlock(), which makes no sense (as mtx_lock succeeded 
> >> fine.)
> >>
> >>
> >> -a
> >>
> >>
> >> On 5 January 2016 at 18:13, Shawn Webb  wrote:
> >> > Thanks for the quick reply! Here's some more debugging output:
> >> >
> >> > === Begin Log ===
> >> > (kgdb) bt
> >> > #0  doadump (textdump=0) at pcpu.h:221
> >> > #1  0x8037c78b in db_dump (dummy=, 
> >> > dummy2=false, dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
> >> > #2  0x8037c57e in db_command (cmd_table=0x0) at 
> >> > /usr/src/sys/ddb/db_command.c:440
> >> > #3  0x8037c314 in db_command_loop () at 
> >> > /usr/src/sys/ddb/db_command.c:493
> >> > #4  0x8037edab in db_trap (type=, code=0) 
> >> > at /usr/src/sys/ddb/db_main.c:251
> >> > #5  0x80a5c563 in kdb_trap (type=12, code=0, tf= >> > out>) at /usr/src/sys/kern/subr_kdb.c:654
> >> > #6  0x80e6b7e1 in trap_fatal (frame=0xfe02c33894d0, 
> >> > eva=) at /usr/src/sys/amd64/amd64/trap.c:829
> >> > #7  0x80e6ba2d in trap_pfault (frame=0xfe02c33894d0, 
> >> > usermode=) at /usr/src/sys/amd64/amd64/trap.c:684
> >> > #8  0x80e6b15f in trap (frame=0xfe02c33894d0) at 
> >> > /usr/src/sys/amd64/amd64/trap.c:435
> >> > #9  0x80e4af97 in calltrap () at 
> >> > /usr/src/sys/amd64/amd64/exception.S:234
> >> > #10 0x80b5de9e in ip_fillid (ip=0xf8000ef8cb88) at 
> >> > /usr/src/sys/netinet/ip_id.c:237
> >> > #11 0x80b6c41b in ip_output (m=, opt= >> > optimized out>, ro=, flags=0, imo=0x0, 
> >> > inp=0xf8000e66e960) at /usr/src/sys/netinet/ip_output.c:268
> >> > #12 0x80bf0612 in udp_send (so=, 
> >> > flags=, m=, addr=0x0, 
> >> > control=, td=0xf8000ef8cb88) at 
> >> > /usr/src/sys/netinet/udp_usrreq.c:1517
> >> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
> >> > uio=, top=0xf8000ef8cb00, control=0x0, 
> >> > flags=, td=0x81bef2ec) at 
> >> > /usr/src/sys/kern/uipc_socket.c:1164
> >> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
> >> > uio=, top=0xf8000ef8cb00, control=0x0, 
> >> > flags=, td=0x81bef2ec) at 
> >> > /usr/src/sys/kern/uipc_socket.c:1164
> >> > #14 0x80aaa03b in kern_sendit (td=0xf8000e4cd9c0, s=6, 
> >> > mp=, flags=0, control=0x0, segflg=UIO_USERSPACE) at 
> >> > /usr/src/sys/kern/uipc_syscalls.c:906
> >> > #15 0x80aaa336 in sendit (td=0xf8000e4cd9c0, s= >> > optimized out>, mp=0xfe02c3389970, flags=3980) at 
> >> > /usr/src/sys/kern/uipc_syscalls.c:833
> >> > #16 0x80aaa1fd in sys_sendto (td=0x0, uap=) 
> >> > at /usr/src/sys/kern/uipc_syscalls.c:957
> >> > #17 0x80e6bfdb in amd64_syscall (td=0xf8000e4cd9c0, 
> >> > traced=0) at subr_syscall.c:135
> >> > #18 0x80e4b27b in Xfast_syscall () at 
> >> > /usr/src/sys/amd64/amd64/exception.S:394
> >> > #19 0x03e339782e8a in ?? ()
> >> > (kgdb) x/i 0x80b5de9e
> >> > 0x80b5de9e : movzbl (%rax,%rcx,1),%esi
> >> > (kgdb) info reg
> >> > rax0x0  0
> >> > rbx0x0  0
> >> > rcx0x0  0
> >> > rdx0x0  0
> >> > rsi0x0  0
> >> > rdi0x0  0
> >> > rbp0xfe02c3388fe0   0xfe02c3388fe0
> >> > rsp0xfe02c3388fc8   0xfe02c3388fc8
> >> > r8 0x0  0
> >> > r9

Re: kernel panic by enabling net.inet.ip.random_id

[Shawn Webb]

Yup, calling ip_initid() in the SYSINIT works! Thanks for the help.

Thanks,

Shawn

On Wed, Jan 06, 2016 at 01:24:53PM -0500, Shawn Webb wrote:
> That's what gets toggled via the sysctl. I think I figured out that I
> need to call ip_initid() in the SYSINIT. Compiling and testing now.
> 
> Thanks,
> 
> Shawn
> 
> On Wed, Jan 06, 2016 at 10:15:08AM -0800, Adrian Chadd wrote:
> > Why'd you condition the vimage definition? :)
> > 
> > 
> > 
> > -a
> > 
> > 
> > On 6 January 2016 at 06:46, Shawn Webb  wrote:
> > > (kgdb) list *(0x80b5de9e)
> > > 0x80b5de9e is in ip_fillid (/usr/src/sys/netinet/ip_id.c:237).
> > > warning: Source file is more recent than executable.
> > >
> > > 232 new_id = 0;
> > > 233 do {
> > > 234 if (new_id != 0)
> > > 235 V_random_id_collisions++;
> > > 236 arc4rand(_id, sizeof(new_id), 0);
> > > 237 } while (bit_test(V_id_bits, new_id) || new_id == 0);
> > > 238 bit_clear(V_id_bits, V_id_array[V_array_ptr]);
> > > 239 bit_set(V_id_bits, new_id);
> > > 240 V_id_array[V_array_ptr] = new_id;
> > > 241 V_array_ptr++;
> > >
> > > This is the change I made to ip_id.c that caused the underlying kernel
> > > panic:
> > > https://github.com/HardenedBSD/hardenedBSD/commit/52d5a93b92097e7a79be8d2e0eb9c1a58b8337d1
> > >
> > > Ideally, we should be able to just toggle that variable and all would be
> > > well. But it seems with the VIMAGE work, something is preventing that.
> > >
> > > Thanks,
> > >
> > > Shawn
> > >
> > > On Tue, Jan 05, 2016 at 06:22:34PM -0800, Adrian Chadd wrote:
> > >> try list *(0x[address]) .
> > >>
> > >> That line is mtx_unlock(), which makes no sense (as mtx_lock succeeded 
> > >> fine.)
> > >>
> > >>
> > >> -a
> > >>
> > >>
> > >> On 5 January 2016 at 18:13, Shawn Webb  
> > >> wrote:
> > >> > Thanks for the quick reply! Here's some more debugging output:
> > >> >
> > >> > === Begin Log ===
> > >> > (kgdb) bt
> > >> > #0  doadump (textdump=0) at pcpu.h:221
> > >> > #1  0x8037c78b in db_dump (dummy=, 
> > >> > dummy2=false, dummy3=0, dummy4=0x0) at 
> > >> > /usr/src/sys/ddb/db_command.c:533
> > >> > #2  0x8037c57e in db_command (cmd_table=0x0) at 
> > >> > /usr/src/sys/ddb/db_command.c:440
> > >> > #3  0x8037c314 in db_command_loop () at 
> > >> > /usr/src/sys/ddb/db_command.c:493
> > >> > #4  0x8037edab in db_trap (type=, code=0) 
> > >> > at /usr/src/sys/ddb/db_main.c:251
> > >> > #5  0x80a5c563 in kdb_trap (type=12, code=0, tf= > >> > optimized out>) at /usr/src/sys/kern/subr_kdb.c:654
> > >> > #6  0x80e6b7e1 in trap_fatal (frame=0xfe02c33894d0, 
> > >> > eva=) at /usr/src/sys/amd64/amd64/trap.c:829
> > >> > #7  0x80e6ba2d in trap_pfault (frame=0xfe02c33894d0, 
> > >> > usermode=) at /usr/src/sys/amd64/amd64/trap.c:684
> > >> > #8  0x80e6b15f in trap (frame=0xfe02c33894d0) at 
> > >> > /usr/src/sys/amd64/amd64/trap.c:435
> > >> > #9  0x80e4af97 in calltrap () at 
> > >> > /usr/src/sys/amd64/amd64/exception.S:234
> > >> > #10 0x80b5de9e in ip_fillid (ip=0xf8000ef8cb88) at 
> > >> > /usr/src/sys/netinet/ip_id.c:237
> > >> > #11 0x80b6c41b in ip_output (m=, 
> > >> > opt=, ro=, flags=0, imo=0x0, 
> > >> > inp=0xf8000e66e960) at /usr/src/sys/netinet/ip_output.c:268
> > >> > #12 0x80bf0612 in udp_send (so=, 
> > >> > flags=, m=, addr=0x0, 
> > >> > control=, td=0xf8000ef8cb88) at 
> > >> > /usr/src/sys/netinet/udp_usrreq.c:1517
> > >> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, 
> > >> > addr=0x0, uio=, top=0xf8000ef8cb00, 
> > >> > control=0x0, flags=, td=0x81bef2ec) at 
> > >> > /usr/src/sys/kern/uipc_socket.c:1164
> > >> > #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, 
> > >> > addr=0x0, uio=, top=0xf8000ef8cb00, 
> > >> > control=0x0, flags=, td=0x81bef2ec) at 
> > >> > /usr/src/sys/kern/uipc_socket.c:1164
> > >> > #14 0x80aaa03b in kern_sendit (td=0xf8000e4cd9c0, s=6, 
> > >> > mp=, flags=0, control=0x0, segflg=UIO_USERSPACE) 
> > >> > at /usr/src/sys/kern/uipc_syscalls.c:906
> > >> > #15 0x80aaa336 in sendit (td=0xf8000e4cd9c0, s= > >> > optimized out>, mp=0xfe02c3389970, flags=3980) at 
> > >> > /usr/src/sys/kern/uipc_syscalls.c:833
> > >> > #16 0x80aaa1fd in sys_sendto (td=0x0, uap= > >> > out>) at /usr/src/sys/kern/uipc_syscalls.c:957
> > >> > #17 0x80e6bfdb in amd64_syscall (td=0xf8000e4cd9c0, 
> > >> > traced=0) at subr_syscall.c:135
> > >> > #18 0x80e4b27b in Xfast_syscall () at 
> > >> > /usr/src/sys/amd64/amd64/exception.S:394
> > >> > #19 0x03e339782e8a in ?? ()
> > >> > (kgdb) x/i 0x80b5de9e
> > >> > 0x80b5de9e : movzbl (%rax,%rcx,1),%esi
> > >> > (kgdb) info reg
> 

Re: kernel panic by enabling net.inet.ip.random_id

[Adrian Chadd]

looks like a null pointer deference. What's kgdb show at that IP?


-a


On 5 January 2016 at 17:57, Shawn Webb  wrote:
> Hey All,
>
> Here's a kernel panic I'm experiencing by enabling net.inet.ip.random_id
> at boot.
>
> I'm on latest HEAD on amd64 in bhyve. I'll soon-ish be testing on native
> hardware with VIMAGE enabled.
>
> === Begin Log ===
> Kernel page fault with the following non-sleepable locks held:
> exclusive sleep mutex ip_id_mtx (ip_id_mtx) r = 0 (0x81c54830) locked 
> @ /usr/src/sys/netinet/ip_id.c:227
> stack backtrace:
> #0 0x80a79620 at witness_debugger+0x70
> #1 0x80a7a937 at witness_warn+0x3d7
> #2 0x80e6b887 at trap_pfault+0x57
> #3 0x80e6b15f at trap+0x4bf
> #4 0x80e4af97 at calltrap+0x8
> #5 0x80b6c41b at ip_output+0x16b
> #6 0x80b68e82 at icmp_reflect+0x5b2
> #7 0x80b6883f at icmp_error+0x46f
> #8 0x80beeb12 at udp_input+0x982
> #9 0x80b69d1d at ip_input+0x17d
> #10 0x80b08ba1 at netisr_dispatch_src+0x81
> #11 0x80afecce at ether_demux+0x15e
> #12 0x80affa14 at ether_nh_input+0x344
> #13 0x80b08ba1 at netisr_dispatch_src+0x81
> #14 0x80afefcf at ether_input+0x4f
> #15 0x8089a5c3 at vtnet_rxq_eof+0x823
> #16 0x8089b2ce at vtnet_rx_vq_intr+0x4e
> #17 0x809e9ba6 at intr_event_execute_handlers+0x96
>
>
> Fatal trap 12: page fault while in kernel mode
> cpuid = 6; apic id = 06
> fault virtual address   = 0x5bd
> fault code  = supervisor read data, page not present
> instruction pointer = 0x20:0x80b5de9e
> stack pointer   = 0x28:0xfe02b8d483e0
> frame pointer   = 0x28:0xfe02b8d48410
> code segment= base 0x0, limit 0xf, type 0x1b
> = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags= interrupt enabled, resume, IOPL = 0
> current process = 12 (irq265: virtio_pci0)
> [ thread pid 12 tid 100040 ]
> Stopped at  ip_fillid+0x8e: movzbl  (%rax,%rcx,1),%esi
> === End Log ===
>
> Thanks,
>
> --
> Shawn Webb
> HardenedBSD
>
> GPG Key ID:  0x6A84658F52456EEE
> GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: kernel panic by enabling net.inet.ip.random_id

[Shawn Webb]

Thanks for the quick reply! Here's some more debugging output:

=== Begin Log ===
(kgdb) bt
#0  doadump (textdump=0) at pcpu.h:221
#1  0x8037c78b in db_dump (dummy=, dummy2=false, 
dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
#2  0x8037c57e in db_command (cmd_table=0x0) at 
/usr/src/sys/ddb/db_command.c:440
#3  0x8037c314 in db_command_loop () at 
/usr/src/sys/ddb/db_command.c:493
#4  0x8037edab in db_trap (type=, code=0) at 
/usr/src/sys/ddb/db_main.c:251
#5  0x80a5c563 in kdb_trap (type=12, code=0, tf=) 
at /usr/src/sys/kern/subr_kdb.c:654
#6  0x80e6b7e1 in trap_fatal (frame=0xfe02c33894d0, eva=) at /usr/src/sys/amd64/amd64/trap.c:829
#7  0x80e6ba2d in trap_pfault (frame=0xfe02c33894d0, 
usermode=) at /usr/src/sys/amd64/amd64/trap.c:684
#8  0x80e6b15f in trap (frame=0xfe02c33894d0) at 
/usr/src/sys/amd64/amd64/trap.c:435
#9  0x80e4af97 in calltrap () at 
/usr/src/sys/amd64/amd64/exception.S:234
#10 0x80b5de9e in ip_fillid (ip=0xf8000ef8cb88) at 
/usr/src/sys/netinet/ip_id.c:237
#11 0x80b6c41b in ip_output (m=, opt=, ro=, flags=0, imo=0x0, 
inp=0xf8000e66e960) at /usr/src/sys/netinet/ip_output.c:268
#12 0x80bf0612 in udp_send (so=, flags=, m=, addr=0x0, control=, td=0xf8000ef8cb88) at /usr/src/sys/netinet/udp_usrreq.c:1517
#13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
uio=, top=0xf8000ef8cb00, control=0x0, flags=, td=0x81bef2ec) at /usr/src/sys/kern/uipc_socket.c:1164
#13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
uio=, top=0xf8000ef8cb00, control=0x0, flags=, td=0x81bef2ec) at /usr/src/sys/kern/uipc_socket.c:1164
#14 0x80aaa03b in kern_sendit (td=0xf8000e4cd9c0, s=6, mp=, flags=0, control=0x0, segflg=UIO_USERSPACE) at 
/usr/src/sys/kern/uipc_syscalls.c:906
#15 0x80aaa336 in sendit (td=0xf8000e4cd9c0, s=, mp=0xfe02c3389970, flags=3980) at 
/usr/src/sys/kern/uipc_syscalls.c:833
#16 0x80aaa1fd in sys_sendto (td=0x0, uap=) at 
/usr/src/sys/kern/uipc_syscalls.c:957
#17 0x80e6bfdb in amd64_syscall (td=0xf8000e4cd9c0, traced=0) at 
subr_syscall.c:135
#18 0x80e4b27b in Xfast_syscall () at 
/usr/src/sys/amd64/amd64/exception.S:394
#19 0x03e339782e8a in ?? ()
(kgdb) x/i 0x80b5de9e
0x80b5de9e : movzbl (%rax,%rcx,1),%esi
(kgdb) info reg
rax0x0  0
rbx0x0  0
rcx0x0  0
rdx0x0  0
rsi0x0  0
rdi0x0  0
rbp0xfe02c3388fe0   0xfe02c3388fe0
rsp0xfe02c3388fc8   0xfe02c3388fc8
r8 0x0  0
r9 0x0  0
r100x0  0
r110x0  0
r120x817c0b80   -2122577024
r130x817c1470   -2122574736
r140x1  1
r150x4  4
rip0x80a1fae3   0x80a1fae3 
eflags 0x0  0
cs 0x0  0
ss 0x0  0
ds 0x0  0
es 0x0  0
fs 0x0  0
gs 0x0  0
=== End Log ===

Thanks,

Shawn

On Tue, Jan 05, 2016 at 06:06:41PM -0800, Adrian Chadd wrote:
> looks like a null pointer deference. What's kgdb show at that IP?
> 
> 
> -a
> 
> 
> On 5 January 2016 at 17:57, Shawn Webb  wrote:
> > Hey All,
> >
> > Here's a kernel panic I'm experiencing by enabling net.inet.ip.random_id
> > at boot.
> >
> > I'm on latest HEAD on amd64 in bhyve. I'll soon-ish be testing on native
> > hardware with VIMAGE enabled.
> >
> > === Begin Log ===
> > Kernel page fault with the following non-sleepable locks held:
> > exclusive sleep mutex ip_id_mtx (ip_id_mtx) r = 0 (0x81c54830) 
> > locked @ /usr/src/sys/netinet/ip_id.c:227
> > stack backtrace:
> > #0 0x80a79620 at witness_debugger+0x70
> > #1 0x80a7a937 at witness_warn+0x3d7
> > #2 0x80e6b887 at trap_pfault+0x57
> > #3 0x80e6b15f at trap+0x4bf
> > #4 0x80e4af97 at calltrap+0x8
> > #5 0x80b6c41b at ip_output+0x16b
> > #6 0x80b68e82 at icmp_reflect+0x5b2
> > #7 0x80b6883f at icmp_error+0x46f
> > #8 0x80beeb12 at udp_input+0x982
> > #9 0x80b69d1d at ip_input+0x17d
> > #10 0x80b08ba1 at netisr_dispatch_src+0x81
> > #11 0x80afecce at ether_demux+0x15e
> > #12 0x80affa14 at ether_nh_input+0x344
> > #13 0x80b08ba1 at netisr_dispatch_src+0x81
> > #14 0x80afefcf at ether_input+0x4f
> > #15 0x8089a5c3 at vtnet_rxq_eof+0x823
> > #16 0x8089b2ce at vtnet_rx_vq_intr+0x4e
> > #17 0x809e9ba6 at intr_event_execute_handlers+0x96
> >
> >
> > Fatal trap 12: page fault while in kernel mode
> > cpuid = 6; apic id = 06
> > fault virtual address   = 0x5bd
> > fault code  

Re: kernel panic by enabling net.inet.ip.random_id

[Adrian Chadd]

try list *(0x[address]) .

That line is mtx_unlock(), which makes no sense (as mtx_lock succeeded fine.)


-a


On 5 January 2016 at 18:13, Shawn Webb  wrote:
> Thanks for the quick reply! Here's some more debugging output:
>
> === Begin Log ===
> (kgdb) bt
> #0  doadump (textdump=0) at pcpu.h:221
> #1  0x8037c78b in db_dump (dummy=, dummy2=false, 
> dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
> #2  0x8037c57e in db_command (cmd_table=0x0) at 
> /usr/src/sys/ddb/db_command.c:440
> #3  0x8037c314 in db_command_loop () at 
> /usr/src/sys/ddb/db_command.c:493
> #4  0x8037edab in db_trap (type=, code=0) at 
> /usr/src/sys/ddb/db_main.c:251
> #5  0x80a5c563 in kdb_trap (type=12, code=0, tf= out>) at /usr/src/sys/kern/subr_kdb.c:654
> #6  0x80e6b7e1 in trap_fatal (frame=0xfe02c33894d0, eva= optimized out>) at /usr/src/sys/amd64/amd64/trap.c:829
> #7  0x80e6ba2d in trap_pfault (frame=0xfe02c33894d0, 
> usermode=) at /usr/src/sys/amd64/amd64/trap.c:684
> #8  0x80e6b15f in trap (frame=0xfe02c33894d0) at 
> /usr/src/sys/amd64/amd64/trap.c:435
> #9  0x80e4af97 in calltrap () at 
> /usr/src/sys/amd64/amd64/exception.S:234
> #10 0x80b5de9e in ip_fillid (ip=0xf8000ef8cb88) at 
> /usr/src/sys/netinet/ip_id.c:237
> #11 0x80b6c41b in ip_output (m=, opt= optimized out>, ro=, flags=0, imo=0x0, 
> inp=0xf8000e66e960) at /usr/src/sys/netinet/ip_output.c:268
> #12 0x80bf0612 in udp_send (so=, flags= optimized out>, m=, addr=0x0, control= out>, td=0xf8000ef8cb88) at /usr/src/sys/netinet/udp_usrreq.c:1517
> #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
> uio=, top=0xf8000ef8cb00, control=0x0, flags= optimized out>, td=0x81bef2ec) at /usr/src/sys/kern/uipc_socket.c:1164
> #13 0x80aa3872 in sosend_dgram (so=0xf8000e6422e8, addr=0x0, 
> uio=, top=0xf8000ef8cb00, control=0x0, flags= optimized out>, td=0x81bef2ec) at /usr/src/sys/kern/uipc_socket.c:1164
> #14 0x80aaa03b in kern_sendit (td=0xf8000e4cd9c0, s=6, mp= optimized out>, flags=0, control=0x0, segflg=UIO_USERSPACE) at 
> /usr/src/sys/kern/uipc_syscalls.c:906
> #15 0x80aaa336 in sendit (td=0xf8000e4cd9c0, s= out>, mp=0xfe02c3389970, flags=3980) at 
> /usr/src/sys/kern/uipc_syscalls.c:833
> #16 0x80aaa1fd in sys_sendto (td=0x0, uap=) at 
> /usr/src/sys/kern/uipc_syscalls.c:957
> #17 0x80e6bfdb in amd64_syscall (td=0xf8000e4cd9c0, traced=0) at 
> subr_syscall.c:135
> #18 0x80e4b27b in Xfast_syscall () at 
> /usr/src/sys/amd64/amd64/exception.S:394
> #19 0x03e339782e8a in ?? ()
> (kgdb) x/i 0x80b5de9e
> 0x80b5de9e : movzbl (%rax,%rcx,1),%esi
> (kgdb) info reg
> rax0x0  0
> rbx0x0  0
> rcx0x0  0
> rdx0x0  0
> rsi0x0  0
> rdi0x0  0
> rbp0xfe02c3388fe0   0xfe02c3388fe0
> rsp0xfe02c3388fc8   0xfe02c3388fc8
> r8 0x0  0
> r9 0x0  0
> r100x0  0
> r110x0  0
> r120x817c0b80   -2122577024
> r130x817c1470   -2122574736
> r140x1  1
> r150x4  4
> rip0x80a1fae3   0x80a1fae3 
> eflags 0x0  0
> cs 0x0  0
> ss 0x0  0
> ds 0x0  0
> es 0x0  0
> fs 0x0  0
> gs 0x0  0
> === End Log ===
>
> Thanks,
>
> Shawn
>
> On Tue, Jan 05, 2016 at 06:06:41PM -0800, Adrian Chadd wrote:
>> looks like a null pointer deference. What's kgdb show at that IP?
>>
>>
>> -a
>>
>>
>> On 5 January 2016 at 17:57, Shawn Webb  wrote:
>> > Hey All,
>> >
>> > Here's a kernel panic I'm experiencing by enabling net.inet.ip.random_id
>> > at boot.
>> >
>> > I'm on latest HEAD on amd64 in bhyve. I'll soon-ish be testing on native
>> > hardware with VIMAGE enabled.
>> >
>> > === Begin Log ===
>> > Kernel page fault with the following non-sleepable locks held:
>> > exclusive sleep mutex ip_id_mtx (ip_id_mtx) r = 0 (0x81c54830) 
>> > locked @ /usr/src/sys/netinet/ip_id.c:227
>> > stack backtrace:
>> > #0 0x80a79620 at witness_debugger+0x70
>> > #1 0x80a7a937 at witness_warn+0x3d7
>> > #2 0x80e6b887 at trap_pfault+0x57
>> > #3 0x80e6b15f at trap+0x4bf
>> > #4 0x80e4af97 at calltrap+0x8
>> > #5 0x80b6c41b at ip_output+0x16b
>> > #6 0x80b68e82 at icmp_reflect+0x5b2
>> > #7 0x80b6883f at icmp_error+0x46f
>> > #8 0x80beeb12 at udp_input+0x982
>> > #9 0x80b69d1d at ip_input+0x17d
>> > #10 0x80b08ba1 at netisr_dispatch_src+0x81
>> > #11 0x80afecce at ether_demux+0x15e