Re: panic: sbappendstream 1 [head/amd64 @r293419]

2016-01-08 Thread Jonathan T. Looney 
On 1/8/16, 9:05 AM, "David Wolfskill"  wrote:

>After the first panic, I rebuilt the kernel without -DNO_CLEAN; the
>crash dump & other diagnostic info is from the clean build.
>
>January  8, 2016 at 05:57:27 AM PST
>
>FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1954
>r293419M/293420:1100093: Fri Jan  8 05:09:57 PST 2016
>r...@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64
>
>panic: sbappendstream 1
>
>...
>Unread portion of the kernel message buffer:
>panic: sbappendstream 1
>cpuid = 7
>KDB: stack backtrace:
>db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
>0xfe085e0595b0
>vpanic() at vpanic+0x182/frame 0xfe085e059630
>kassert_panic() at kassert_panic+0x126/frame 0xfe085e0596a0
>sbappendstream_locked() at sbappendstream_locked+0xa5/frame
>0xfe085e0596d0
>uipc_send() at uipc_send+0x942/frame 0xfe085e059780
>sosend_generic() at sosend_generic+0x42f/frame 0xfe085e059840
>kern_sendit() at kern_sendit+0x21b/frame 0xfe085e0598f0
>sendit() at sendit+0x126/frame 0xfe085e059940
>sys_sendmsg() at sys_sendmsg+0x61/frame 0xfe085e0599a0
>amd64_syscall() at amd64_syscall+0x2db/frame 0xfe085e059ab0
>Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe085e059ab0


The likely suspect here looks like r293405, which changed uipc_send() to
use sbappendstream_locked() instead of sbappend_locked().

However, I can't explain *why* that change is causing this problem without
further investigation.

Can you try reverting the change to see if that solves the problem you are
seeing?

Thanks!

Jonathan


>--- syscall (28, FreeBSD ELF64, sys_sendmsg), rip = 0x801270dfa, rsp =
>0x7fffa098, rbp = 0x7fffa0d0 ---
>KDB: enter: panic
>...
>Loaded symbols for /boot/kernel/autofs.ko
>#0  doadump (textdump=0) at pcpu.h:221
>221 pcpu.h: No such file or directory.
>in pcpu.h
>(kgdb) #0  doadump (textdump=0) at pcpu.h:221
>#1  0x8038205b in db_dump (dummy=,
>dummy2=false, 
>dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
>#2  0x80381e4e in db_command (cmd_table=0x0)
>at /usr/src/sys/ddb/db_command.c:440
>#3  0x80381be4 in db_command_loop ()
>at /usr/src/sys/ddb/db_command.c:493
>#4  0x8038467b in db_trap (type=, code=0)
>at /usr/src/sys/ddb/db_main.c:251
>#5  0x80a5cfe3 in kdb_trap (type=3, code=0, tf=out>)
>at /usr/src/sys/kern/subr_kdb.c:654
>#6  0x80e6a2a8 in trap (frame=0xfe085e0594e0)
>at /usr/src/sys/amd64/amd64/trap.c:549
>#7  0x80e4a317 in calltrap ()
>at /usr/src/sys/amd64/amd64/exception.S:234
>#8  0x80a5c6cb in kdb_enter (why=0x8137af3c "panic",
>msg=0x80 ) at cpufunc.h:63
>#9  0x80a1fb8f in vpanic (fmt=,
>ap=) at /usr/src/sys/kern/kern_shutdown.c:750
>#10 0x80a1f9e6 in kassert_panic (fmt=)
>at /usr/src/sys/kern/kern_shutdown.c:647
>#11 0x80aa3375 in sbappendstream_locked (sb=0xf80044212378,
>m=0xf800108c7200, flags=0) at /usr/src/sys/kern/uipc_sockbuf.c:642
>#12 0x80ab1a42 in uipc_send (so=0xf80044212000, flags=0,
>m=, nam=0x0, control=,
>td=0xf8001078e9a0) at /usr/src/sys/kern/uipc_usrreq.c:984
>#13 0x80aa5f5f in sosend_generic (so=0xf80044212000,
>addr=0x0, 
>uio=0xfe085e059890, top=,
>control=, flags=,
>td=0xfe085e059880) at /usr/src/sys/kern/uipc_socket.c:1349
>#14 0x80aac36b in kern_sendit (td=0xf8001078e9a0, s=6,
>mp=, flags=0, control=0x0, segflg=UIO_USERSPACE)
>at /usr/src/sys/kern/uipc_syscalls.c:906
>#15 0x80aac666 in sendit (td=0xf8001078e9a0,
>s=, mp=0xfe085e059958, flags=0)
>at /usr/src/sys/kern/uipc_syscalls.c:833
>#16 0x80aac6f1 in sys_sendmsg (td=0xf8001078e9a0,
>uap=0xfe085e059a40) at /usr/src/sys/kern/uipc_syscalls.c:1035
>#17 0x80e6b13b in amd64_syscall (td=0xf8001078e9a0, traced=0)
>at subr_syscall.c:135
>#18 0x80e4a5fb in Xfast_syscall ()
>at /usr/src/sys/amd64/amd64/exception.S:394
>#19 0x000801270dfa in ?? ()
>Previous frame inner to this frame (corrupt stack?)
>Current language:  auto; currently minimal
>(kgdb) 
>.
>
>As indicated above, this is with a GENERIC kernel.  My laptop (running
>a kernel built with the same sources, but a slightly customized kernel
>config) gets to the point of allowing me to login (via xdm), but when I
>fire off a command that creates xterms & tries to run tmux(1) in them,
>locks up (as far as I can tell), and a power-cycle is needed to recover.
>
>I can poke at the crash dump (given hints), make the dump and core.txt
>file
>available.
>
>Peace,
>david
>-- 
>David H. Wolfskill da...@catwhisker.org
>Those who would murder in the name of God or prophet are blasphemous
>cowards.
>
>See http://www.catwhisker.org/~david/publickey.gpg for my public key.

Re: panic: sbappendstream 1 [head/amd64 @r293419]

2016-01-08 Thread Gleb Smirnoff 
On Fri, Jan 08, 2016 at 09:34:23AM -0500, Jonathan T. Looney wrote:
J> The likely suspect here looks like r293405, which changed uipc_send() to
J> use sbappendstream_locked() instead of sbappend_locked().
J> 
J> However, I can't explain *why* that change is causing this problem without
J> further investigation.

That is because sbappendstream() has invariant that socket buffer doesn't
have any records in it. But, if control data is sent, which is possible
for AF_UNIX, a record is made. Thus, we can't used optimized version for
AF_UNIX, only for AF_INET.

-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

panic: sbappendstream 1 [head/amd64 @r293419]

2016-01-08 Thread David Wolfskill 
After the first panic, I rebuilt the kernel without -DNO_CLEAN; the
crash dump & other diagnostic info is from the clean build.

January  8, 2016 at 05:57:27 AM PST

FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1954  
r293419M/293420:1100093: Fri Jan  8 05:09:57 PST 2016 
r...@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64

panic: sbappendstream 1

...
Unread portion of the kernel message buffer:
panic: sbappendstream 1
cpuid = 7
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe085e0595b0
vpanic() at vpanic+0x182/frame 0xfe085e059630
kassert_panic() at kassert_panic+0x126/frame 0xfe085e0596a0
sbappendstream_locked() at sbappendstream_locked+0xa5/frame 0xfe085e0596d0
uipc_send() at uipc_send+0x942/frame 0xfe085e059780
sosend_generic() at sosend_generic+0x42f/frame 0xfe085e059840
kern_sendit() at kern_sendit+0x21b/frame 0xfe085e0598f0
sendit() at sendit+0x126/frame 0xfe085e059940
sys_sendmsg() at sys_sendmsg+0x61/frame 0xfe085e0599a0
amd64_syscall() at amd64_syscall+0x2db/frame 0xfe085e059ab0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe085e059ab0
--- syscall (28, FreeBSD ELF64, sys_sendmsg), rip = 0x801270dfa, rsp = 
0x7fffa098, rbp = 0x7fffa0d0 ---
KDB: enter: panic
...
Loaded symbols for /boot/kernel/autofs.ko
#0  doadump (textdump=0) at pcpu.h:221
221 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0  doadump (textdump=0) at pcpu.h:221
#1  0x8038205b in db_dump (dummy=, dummy2=false, 
dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
#2  0x80381e4e in db_command (cmd_table=0x0)
at /usr/src/sys/ddb/db_command.c:440
#3  0x80381be4 in db_command_loop ()
at /usr/src/sys/ddb/db_command.c:493
#4  0x8038467b in db_trap (type=, code=0)
at /usr/src/sys/ddb/db_main.c:251
#5  0x80a5cfe3 in kdb_trap (type=3, code=0, tf=)
at /usr/src/sys/kern/subr_kdb.c:654
#6  0x80e6a2a8 in trap (frame=0xfe085e0594e0)
at /usr/src/sys/amd64/amd64/trap.c:549
#7  0x80e4a317 in calltrap ()
at /usr/src/sys/amd64/amd64/exception.S:234
#8  0x80a5c6cb in kdb_enter (why=0x8137af3c "panic", 
msg=0x80 ) at cpufunc.h:63
#9  0x80a1fb8f in vpanic (fmt=, 
ap=) at /usr/src/sys/kern/kern_shutdown.c:750
#10 0x80a1f9e6 in kassert_panic (fmt=)
at /usr/src/sys/kern/kern_shutdown.c:647
#11 0x80aa3375 in sbappendstream_locked (sb=0xf80044212378, 
m=0xf800108c7200, flags=0) at /usr/src/sys/kern/uipc_sockbuf.c:642
#12 0x80ab1a42 in uipc_send (so=0xf80044212000, flags=0, 
m=, nam=0x0, control=, 
td=0xf8001078e9a0) at /usr/src/sys/kern/uipc_usrreq.c:984
#13 0x80aa5f5f in sosend_generic (so=0xf80044212000, addr=0x0, 
uio=0xfe085e059890, top=, 
control=, flags=, 
td=0xfe085e059880) at /usr/src/sys/kern/uipc_socket.c:1349
#14 0x80aac36b in kern_sendit (td=0xf8001078e9a0, s=6, 
mp=, flags=0, control=0x0, segflg=UIO_USERSPACE)
at /usr/src/sys/kern/uipc_syscalls.c:906
#15 0x80aac666 in sendit (td=0xf8001078e9a0, 
s=, mp=0xfe085e059958, flags=0)
at /usr/src/sys/kern/uipc_syscalls.c:833
#16 0x80aac6f1 in sys_sendmsg (td=0xf8001078e9a0, 
uap=0xfe085e059a40) at /usr/src/sys/kern/uipc_syscalls.c:1035
#17 0x80e6b13b in amd64_syscall (td=0xf8001078e9a0, traced=0)
at subr_syscall.c:135
#18 0x80e4a5fb in Xfast_syscall ()
at /usr/src/sys/amd64/amd64/exception.S:394
#19 0x000801270dfa in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb) 
.

As indicated above, this is with a GENERIC kernel.  My laptop (running
a kernel built with the same sources, but a slightly customized kernel
config) gets to the point of allowing me to login (via xdm), but when I
fire off a command that creates xterms & tries to run tmux(1) in them,
locks up (as far as I can tell), and a power-cycle is needed to recover.

I can poke at the crash dump (given hints), make the dump and core.txt file
available.

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


signature.asc
Description: PGP signature

Re: panic: sbappendstream 1 [head/amd64 @r293419]

2016-01-08 Thread David Wolfskill 
On Fri, Jan 08, 2016 at 06:05:18AM -0800, David Wolfskill wrote:
> After the first panic, I rebuilt the kernel without -DNO_CLEAN; the
> crash dump & other diagnostic info is from the clean build.
> 
> January  8, 2016 at 05:57:27 AM PST
> 
> FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1954  
> r293419M/293420:1100093: Fri Jan  8 05:09:57 PST 2016 
> r...@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64
> 
> panic: sbappendstream 1
> 

flo@ suggested reverting r293405; after doing that & rebuilding the
kernel, the build machine boots to multi-user mode and I'm able to
login:

FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1955  
r293419M/293420:1100093: Fri Jan  8 06:26:38 PST 2016 
r...@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64

I'll try that for my laptop, as well.

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


signature.asc
Description: PGP signature

Re: panic: sbappendstream 1 [head/amd64 @r293419]

2016-01-08 Thread David Wolfskill 
On Fri, Jan 08, 2016 at 06:33:40AM -0800, David Wolfskill wrote:
> ...
> > panic: sbappendstream 1
> > 
> 
> flo@ suggested reverting r293405; after doing that & rebuilding the
> kernel, the build machine boots to multi-user mode and I'm able to
> login:
> 
> FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1955  
> r293419M/293420:1100093: Fri Jan  8 06:26:38 PST 2016 
> r...@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64
> 
> I'll try that for my laptop, as well.

That seems to have worked for thhe laptop, as well:

FreeBSD g1-252.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #299  
r293419M/293420:1100093: Fri Jan  8 06:44:01 PST 2016 
r...@g1-252.catwhisker.org:/common/S4/obj/usr/src/sys/CANARY  amd64

Peace,
david
-- 
David H. Wolfskill  da...@catwhisker.org
Those who would murder in the name of God or prophet are blasphemous cowards.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.


signature.asc
Description: PGP signature

Re: panic: sbappendstream 1 [head/amd64 @r293419]

2016-01-08 Thread Gleb Smirnoff 
  David,

  problem confirmed. Will either fix today, or revert the change
by the end of the day.

On Fri, Jan 08, 2016 at 06:05:18AM -0800, David Wolfskill wrote:
D> After the first panic, I rebuilt the kernel without -DNO_CLEAN; the
D> crash dump & other diagnostic info is from the clean build.
D> 
D> January  8, 2016 at 05:57:27 AM PST
D> 
D> FreeBSD freebeast.catwhisker.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1954  
r293419M/293420:1100093: Fri Jan  8 05:09:57 PST 2016 
r...@freebeast.catwhisker.org:/common/S4/obj/usr/src/sys/GENERIC  amd64
D> 
D> panic: sbappendstream 1
D> 
D> ...
D> Unread portion of the kernel message buffer:
D> panic: sbappendstream 1
D> cpuid = 7
D> KDB: stack backtrace:
D> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
0xfe085e0595b0
D> vpanic() at vpanic+0x182/frame 0xfe085e059630
D> kassert_panic() at kassert_panic+0x126/frame 0xfe085e0596a0
D> sbappendstream_locked() at sbappendstream_locked+0xa5/frame 
0xfe085e0596d0
D> uipc_send() at uipc_send+0x942/frame 0xfe085e059780
D> sosend_generic() at sosend_generic+0x42f/frame 0xfe085e059840
D> kern_sendit() at kern_sendit+0x21b/frame 0xfe085e0598f0
D> sendit() at sendit+0x126/frame 0xfe085e059940
D> sys_sendmsg() at sys_sendmsg+0x61/frame 0xfe085e0599a0
D> amd64_syscall() at amd64_syscall+0x2db/frame 0xfe085e059ab0
D> Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfe085e059ab0
D> --- syscall (28, FreeBSD ELF64, sys_sendmsg), rip = 0x801270dfa, rsp = 
0x7fffa098, rbp = 0x7fffa0d0 ---
D> KDB: enter: panic
D> ...
D> Loaded symbols for /boot/kernel/autofs.ko
D> #0  doadump (textdump=0) at pcpu.h:221
D> 221 pcpu.h: No such file or directory.
D> in pcpu.h
D> (kgdb) #0  doadump (textdump=0) at pcpu.h:221
D> #1  0x8038205b in db_dump (dummy=, 
dummy2=false, 
D> dummy3=0, dummy4=0x0) at /usr/src/sys/ddb/db_command.c:533
D> #2  0x80381e4e in db_command (cmd_table=0x0)
D> at /usr/src/sys/ddb/db_command.c:440
D> #3  0x80381be4 in db_command_loop ()
D> at /usr/src/sys/ddb/db_command.c:493
D> #4  0x8038467b in db_trap (type=, code=0)
D> at /usr/src/sys/ddb/db_main.c:251
D> #5  0x80a5cfe3 in kdb_trap (type=3, code=0, tf=)
D> at /usr/src/sys/kern/subr_kdb.c:654
D> #6  0x80e6a2a8 in trap (frame=0xfe085e0594e0)
D> at /usr/src/sys/amd64/amd64/trap.c:549
D> #7  0x80e4a317 in calltrap ()
D> at /usr/src/sys/amd64/amd64/exception.S:234
D> #8  0x80a5c6cb in kdb_enter (why=0x8137af3c "panic", 
D> msg=0x80 ) at cpufunc.h:63
D> #9  0x80a1fb8f in vpanic (fmt=, 
D> ap=) at /usr/src/sys/kern/kern_shutdown.c:750
D> #10 0x80a1f9e6 in kassert_panic (fmt=)
D> at /usr/src/sys/kern/kern_shutdown.c:647
D> #11 0x80aa3375 in sbappendstream_locked (sb=0xf80044212378, 
D> m=0xf800108c7200, flags=0) at /usr/src/sys/kern/uipc_sockbuf.c:642
D> #12 0x80ab1a42 in uipc_send (so=0xf80044212000, flags=0, 
D> m=, nam=0x0, control=, 
D> td=0xf8001078e9a0) at /usr/src/sys/kern/uipc_usrreq.c:984
D> #13 0x80aa5f5f in sosend_generic (so=0xf80044212000, addr=0x0, 
D> uio=0xfe085e059890, top=, 
D> control=, flags=, 
D> td=0xfe085e059880) at /usr/src/sys/kern/uipc_socket.c:1349
D> #14 0x80aac36b in kern_sendit (td=0xf8001078e9a0, s=6, 
D> mp=, flags=0, control=0x0, segflg=UIO_USERSPACE)
D> at /usr/src/sys/kern/uipc_syscalls.c:906
D> #15 0x80aac666 in sendit (td=0xf8001078e9a0, 
D> s=, mp=0xfe085e059958, flags=0)
D> at /usr/src/sys/kern/uipc_syscalls.c:833
D> #16 0x80aac6f1 in sys_sendmsg (td=0xf8001078e9a0, 
D> uap=0xfe085e059a40) at /usr/src/sys/kern/uipc_syscalls.c:1035
D> #17 0x80e6b13b in amd64_syscall (td=0xf8001078e9a0, traced=0)
D> at subr_syscall.c:135
D> #18 0x80e4a5fb in Xfast_syscall ()
D> at /usr/src/sys/amd64/amd64/exception.S:394
D> #19 0x000801270dfa in ?? ()
D> Previous frame inner to this frame (corrupt stack?)
D> Current language:  auto; currently minimal
D> (kgdb) 
D> .
D> 
D> As indicated above, this is with a GENERIC kernel.  My laptop (running
D> a kernel built with the same sources, but a slightly customized kernel
D> config) gets to the point of allowing me to login (via xdm), but when I
D> fire off a command that creates xterms & tries to run tmux(1) in them,
D> locks up (as far as I can tell), and a power-cycle is needed to recover.
D> 
D> I can poke at the crash dump (given hints), make the dump and core.txt file
D> available.
D> 
D> Peace,
D> david
D> -- 
D> David H. Wolfskill   da...@catwhisker.org
D> Those who would murder in the name of God or prophet are blasphemous cowards.
D> 
D> See http://www.catwhisker.org/~david/publickey.gpg for my public key.



-- 
Totus tuus, Glebius.
___
freebsd-current@freebsd.org