panic: general protection fault
I just got this one: 0210415 17:01:45 all (458/755): callout_reset_on.sh kernel trap 9 with interrupts disabled Fatal trap 9: general protection fault while in kernel mode cpuid = 10; apic id = 0a instruction pointer = 0x20:0x80c7c286 stack pointer = 0x0:0xfe00e49b0730 frame pointer = 0x0:0xfe00e49b0770 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = resume, IOPL = 0 current process = 12 (swi1: netisr 0) trap number = 9 panic: general protection fault cpuid = 10 time = 1618498958 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe00e49b0440 vpanic() at vpanic+0x181/frame 0xfe00e49b0490 panic() at panic+0x43/frame 0xfe00e49b04f0 trap_fatal() at trap_fatal+0x387/frame 0xfe00e49b0550 trap() at trap+0xa4/frame 0xfe00e49b0660 calltrap() at calltrap+0x8/frame 0xfe00e49b0660 --- trap 0x9, rip = 0x80c7c286, rsp = 0xfe00e49b0730, rbp = 0xfe00e49b0770 --- turnstile_wait() at turnstile_wait+0x46/frame 0xfe00e49b0770 __rw_wlock_hard() at __rw_wlock_hard+0x464/frame 0xfe00e49b0820 _rw_wlock_cookie() at _rw_wlock_cookie+0xb7/frame 0xfe00e49b0860 in_pcblookup_hash() at in_pcblookup_hash+0x76/frame 0xfe00e49b0890 in_pcblookup_mbuf() at in_pcblookup_mbuf+0x24/frame 0xfe00e49b08b0 tcp_input() at tcp_input+0x6e8/frame 0xfe00e49b0a10 ip_input() at ip_input+0x194/frame 0xfe00e49b0aa0 swi_net() at swi_net+0x1a1/frame 0xfe00e49b0b20 ithread_loop() at ithread_loop+0x279/frame 0xfe00e49b0bb0 fork_exit() at fork_exit+0x80/frame 0xfe00e49b0bf0 https://people.freebsd.org/~pho/stress/log/log0092.txt - Peter ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
On 12/8/20, Mark Johnston wrote: > On Tue, Dec 08, 2020 at 04:40:16PM +0100, Mateusz Guzik wrote: >> I think this is a long standing bug against exiting processes. >> >> filedesc_out only increments *hold* count, but that does not prevent >> fdescfree_fds from progressing and freeing everything without any >> locks held. > > I think it is fallout from r36: before that, fdescfree() acquired > and released the exclusive fd table lock between decrementing > fdp->fd_refcount and calling fdescfree_fds(). This would serialize with > the loop in kern_proc_fildesc_out(), which checks fdp->fd_refcount > 0 > at the beginning of each iteration. Now there is no serialization and > they can race. > Oh I forgot consumers keep checking for fd_refcount. In that case probably would be best to add sx_wait_unlocked. >> A hotfix (for mfc) would add locking around it, but a long term fix >> should wait for hold count to drain. By that point there can't be any >> new arrivals due to: >> >> PROC_LOCK(p); >> p->p_fd = NULL; >> PROC_UNLOCK(p); >> >> I'll code both later today. > -- Mateusz Guzik ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
On Tue, Dec 08, 2020 at 04:40:16PM +0100, Mateusz Guzik wrote: > I think this is a long standing bug against exiting processes. > > filedesc_out only increments *hold* count, but that does not prevent > fdescfree_fds from progressing and freeing everything without any > locks held. I think it is fallout from r36: before that, fdescfree() acquired and released the exclusive fd table lock between decrementing fdp->fd_refcount and calling fdescfree_fds(). This would serialize with the loop in kern_proc_fildesc_out(), which checks fdp->fd_refcount > 0 at the beginning of each iteration. Now there is no serialization and they can race. > A hotfix (for mfc) would add locking around it, but a long term fix > should wait for hold count to drain. By that point there can't be any > new arrivals due to: > > PROC_LOCK(p); > p->p_fd = NULL; > PROC_UNLOCK(p); > > I'll code both later today. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
On Tue, Dec 08, 2020 at 10:30:41AM -0500, Mark Johnston wrote:
> On Tue, Dec 08, 2020 at 12:47:18PM +0100, Peter Holm wrote:
> > I just got this panic:
> >
> > Fatal trap 9: general protection fault while in kernel mode
> > cpuid = 9; apic id = 09
> > instruction pointer = 0x20:0x80bc6e22
> > stack pointer = 0x28:0xfe0698887630
> > frame pointer = 0x28:0xfe06988876b0
> > code segment = base 0x0, limit 0xf, type 0x1b
> >= DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags = interrupt enabled, resume, IOPL = 0
> > current process = 45966 (fstat)
> > trap number = 9
> > panic: general protection fault
> > cpuid = 9
> > time = 1607416693
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> > 0xfe0698887340
> > vpanic() at vpanic+0x181/frame 0xfe0698887390
> > panic() at panic+0x43/frame 0xfe06988873f0
> > trap_fatal() at trap_fatal+0x387/frame 0xfe0698887450
> > trap() at trap+0xa4/frame 0xfe0698887560
> > calltrap() at calltrap+0x8/frame 0xfe0698887560
> > --- trap 0x9, rip = 0x80bc6e22, rsp = 0xfe0698887630, rbp =
> > 0xfe06988876b0 ---
> > __mtx_lock_sleep() at __mtx_lock_sleep+0xd2/frame 0xfe06988876b0
> > __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfe0698887700
> > uipc_sockaddr() at uipc_sockaddr+0x4c/frame 0xfe0698887730
> > soo_fill_kinfo() at soo_fill_kinfo+0x11e/frame 0xfe0698887770
> > kern_proc_filedesc_out() at kern_proc_filedesc_out+0xb57/frame
> > 0xfe0698887810
> > sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame
> > 0xfe0698887890
> > sysctl_root_handler_locked() at sysctl_root_handler_locked+0x9c/frame
> > 0xfe06988878e0
> > sysctl_root() at sysctl_root+0x20d/frame 0xfe0698887960
> > userland_sysctl() at userland_sysctl+0x180/frame 0xfe0698887a10
> > sys___sysctl() at sys___sysctl+0x5f/frame 0xfe0698887ac0
> > amd64_syscall() at amd64_syscall+0x147/frame 0xfe0698887bf0
> > fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfe0698887bf0
> > --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8003948ea, rsp =
> > 0x7fffc138, rbp = 0x7fffc170 ---
> >
> > https://people.freebsd.org/~pho/stress/log/log0004.txt
>
> So here the unpcb is freed, and indeed the file itself has been closed:
>
> $3 = {f_flag = 0x3, f_count = 0x0, f_data = 0x0, f_ops = 0x81901f50
> ,
> f_vnode = 0x0, f_cred = 0xf80248beb600, f_type = 0x2,
> f_vnread_flags = 0x0,
> {f_seqcount = {0x0, 0x0}, f_pipegen = 0x0}, f_nextoff = {0x0, 0x0},
> f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset = 0x0}
>
> However, it must have happened very recently because soo_fill_kinfo()
> dereferences fp->f_data and yet we did not panic due to a null
> dereference.
>
> kern_proc_filedesc_out() holds the fdtable shared lock thoughout all of
> this, which is supposed to prevent the table entry from being freed
> since that requires the exclusive lock.
>
> Could you show fdp->fd_ofiles[3] and fdp->fd_map[0] from frame 26?
Sure:
(kgdb) p fdp->fd_files->fdt_ofiles[3]
$1 = {fde_file = 0xf807306fd0f0, fde_caps = {fc_rights = {cr_rights = {0x0,
0x0}}, fc_ioctls = 0x0, fc_nioctls = 0x0, fc_fcntls = 0x0}, fde_flags = 0x0,
fde_seqc = 0x2}
(kgdb) p fdp->fd_map[0]
$2 = 0x1f
(kgdb)
- Peter
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
On Tue, Dec 8, 2020 at 9:48 AM Mark Johnston wrote: > > On Tue, Dec 08, 2020 at 09:39:05AM -0600, Kyle Evans wrote: > > On Tue, Dec 8, 2020 at 9:30 AM Mark Johnston wrote: > > > kern_proc_filedesc_out() holds the fdtable shared lock thoughout all of > > > this, which is supposed to prevent the table entry from being freed > > > since that requires the exclusive lock. > > > > > > > export_file_to_sb drops the lock without it or kern_proc_filedesc_out > > holding the file it's about to look at, though. > > Yes, but that's after it calls fo_fill_kinfo(). At that point it has > already collected the to-be-exported info in an sbuf. Whoops, indeed- sorry. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
On Tue, Dec 08, 2020 at 09:39:05AM -0600, Kyle Evans wrote: > On Tue, Dec 8, 2020 at 9:30 AM Mark Johnston wrote: > > kern_proc_filedesc_out() holds the fdtable shared lock thoughout all of > > this, which is supposed to prevent the table entry from being freed > > since that requires the exclusive lock. > > > > export_file_to_sb drops the lock without it or kern_proc_filedesc_out > holding the file it's about to look at, though. Yes, but that's after it calls fo_fill_kinfo(). At that point it has already collected the to-be-exported info in an sbuf. ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
I think this is a long standing bug against exiting processes.
filedesc_out only increments *hold* count, but that does not prevent
fdescfree_fds from progressing and freeing everything without any
locks held.
A hotfix (for mfc) would add locking around it, but a long term fix
should wait for hold count to drain. By that point there can't be any
new arrivals due to:
PROC_LOCK(p);
p->p_fd = NULL;
PROC_UNLOCK(p);
I'll code both later today.
On 12/8/20, Mark Johnston wrote:
> On Tue, Dec 08, 2020 at 12:47:18PM +0100, Peter Holm wrote:
>> I just got this panic:
>>
>> Fatal trap 9: general protection fault while in kernel mode
>> cpuid = 9; apic id = 09
>> instruction pointer = 0x20:0x80bc6e22
>> stack pointer = 0x28:0xfe0698887630
>> frame pointer = 0x28:0xfe06988876b0
>> code segment = base 0x0, limit 0xf, type 0x1b
>>= DPL 0, pres 1, long 1, def32 0, gran 1
>> processor eflags = interrupt enabled, resume, IOPL = 0
>> current process = 45966 (fstat)
>> trap number = 9
>> panic: general protection fault
>> cpuid = 9
>> time = 1607416693
>> KDB: stack backtrace:
>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
>> 0xfe0698887340
>> vpanic() at vpanic+0x181/frame 0xfe0698887390
>> panic() at panic+0x43/frame 0xfe06988873f0
>> trap_fatal() at trap_fatal+0x387/frame 0xfe0698887450
>> trap() at trap+0xa4/frame 0xfe0698887560
>> calltrap() at calltrap+0x8/frame 0xfe0698887560
>> --- trap 0x9, rip = 0x80bc6e22, rsp = 0xfe0698887630, rbp =
>> 0xfe06988876b0 ---
>> __mtx_lock_sleep() at __mtx_lock_sleep+0xd2/frame 0xfe06988876b0
>> __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfe0698887700
>> uipc_sockaddr() at uipc_sockaddr+0x4c/frame 0xfe0698887730
>> soo_fill_kinfo() at soo_fill_kinfo+0x11e/frame 0xfe0698887770
>> kern_proc_filedesc_out() at kern_proc_filedesc_out+0xb57/frame
>> 0xfe0698887810
>> sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame
>> 0xfe0698887890
>> sysctl_root_handler_locked() at sysctl_root_handler_locked+0x9c/frame
>> 0xfe06988878e0
>> sysctl_root() at sysctl_root+0x20d/frame 0xfe0698887960
>> userland_sysctl() at userland_sysctl+0x180/frame 0xfe0698887a10
>> sys___sysctl() at sys___sysctl+0x5f/frame 0xfe0698887ac0
>> amd64_syscall() at amd64_syscall+0x147/frame 0xfe0698887bf0
>> fast_syscall_common() at fast_syscall_common+0xf8/frame
>> 0xfe0698887bf0
>> --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8003948ea, rsp =
>> 0x7fffc138, rbp = 0x7fffc170 ---
>>
>> https://people.freebsd.org/~pho/stress/log/log0004.txt
>
> So here the unpcb is freed, and indeed the file itself has been closed:
>
> $3 = {f_flag = 0x3, f_count = 0x0, f_data = 0x0, f_ops = 0x81901f50
> ,
> f_vnode = 0x0, f_cred = 0xf80248beb600, f_type = 0x2,
> f_vnread_flags = 0x0,
> {f_seqcount = {0x0, 0x0}, f_pipegen = 0x0}, f_nextoff = {0x0, 0x0},
> f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset = 0x0}
>
> However, it must have happened very recently because soo_fill_kinfo()
> dereferences fp->f_data and yet we did not panic due to a null
> dereference.
>
> kern_proc_filedesc_out() holds the fdtable shared lock thoughout all of
> this, which is supposed to prevent the table entry from being freed
> since that requires the exclusive lock.
>
> Could you show fdp->fd_ofiles[3] and fdp->fd_map[0] from frame 26?
> ___
> freebsd-current@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
>
--
Mateusz Guzik
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
On Tue, Dec 8, 2020 at 9:30 AM Mark Johnston wrote:
>
> On Tue, Dec 08, 2020 at 12:47:18PM +0100, Peter Holm wrote:
> > I just got this panic:
> >
> > Fatal trap 9: general protection fault while in kernel mode
> > cpuid = 9; apic id = 09
> > instruction pointer = 0x20:0x80bc6e22
> > stack pointer = 0x28:0xfe0698887630
> > frame pointer = 0x28:0xfe06988876b0
> > code segment = base 0x0, limit 0xf, type 0x1b
> >= DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags = interrupt enabled, resume, IOPL = 0
> > current process = 45966 (fstat)
> > trap number = 9
> > panic: general protection fault
> > cpuid = 9
> > time = 1607416693
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> > 0xfe0698887340
> > vpanic() at vpanic+0x181/frame 0xfe0698887390
> > panic() at panic+0x43/frame 0xfe06988873f0
> > trap_fatal() at trap_fatal+0x387/frame 0xfe0698887450
> > trap() at trap+0xa4/frame 0xfe0698887560
> > calltrap() at calltrap+0x8/frame 0xfe0698887560
> > --- trap 0x9, rip = 0x80bc6e22, rsp = 0xfe0698887630, rbp =
> > 0xfe06988876b0 ---
> > __mtx_lock_sleep() at __mtx_lock_sleep+0xd2/frame 0xfe06988876b0
> > __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfe0698887700
> > uipc_sockaddr() at uipc_sockaddr+0x4c/frame 0xfe0698887730
> > soo_fill_kinfo() at soo_fill_kinfo+0x11e/frame 0xfe0698887770
> > kern_proc_filedesc_out() at kern_proc_filedesc_out+0xb57/frame
> > 0xfe0698887810
> > sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame
> > 0xfe0698887890
> > sysctl_root_handler_locked() at sysctl_root_handler_locked+0x9c/frame
> > 0xfe06988878e0
> > sysctl_root() at sysctl_root+0x20d/frame 0xfe0698887960
> > userland_sysctl() at userland_sysctl+0x180/frame 0xfe0698887a10
> > sys___sysctl() at sys___sysctl+0x5f/frame 0xfe0698887ac0
> > amd64_syscall() at amd64_syscall+0x147/frame 0xfe0698887bf0
> > fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfe0698887bf0
> > --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8003948ea, rsp =
> > 0x7fffc138, rbp = 0x7fffc170 ---
> >
> > https://people.freebsd.org/~pho/stress/log/log0004.txt
>
> So here the unpcb is freed, and indeed the file itself has been closed:
>
> $3 = {f_flag = 0x3, f_count = 0x0, f_data = 0x0, f_ops = 0x81901f50
> ,
> f_vnode = 0x0, f_cred = 0xf80248beb600, f_type = 0x2,
> f_vnread_flags = 0x0,
> {f_seqcount = {0x0, 0x0}, f_pipegen = 0x0}, f_nextoff = {0x0, 0x0},
> f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset = 0x0}
>
> However, it must have happened very recently because soo_fill_kinfo()
> dereferences fp->f_data and yet we did not panic due to a null
> dereference.
>
> kern_proc_filedesc_out() holds the fdtable shared lock thoughout all of
> this, which is supposed to prevent the table entry from being freed
> since that requires the exclusive lock.
>
export_file_to_sb drops the lock without it or kern_proc_filedesc_out
holding the file it's about to look at, though.
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: general protection fault from uipc_sockaddr+0x4c
On Tue, Dec 08, 2020 at 12:47:18PM +0100, Peter Holm wrote:
> I just got this panic:
>
> Fatal trap 9: general protection fault while in kernel mode
> cpuid = 9; apic id = 09
> instruction pointer = 0x20:0x80bc6e22
> stack pointer = 0x28:0xfe0698887630
> frame pointer = 0x28:0xfe06988876b0
> code segment = base 0x0, limit 0xf, type 0x1b
>= DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags = interrupt enabled, resume, IOPL = 0
> current process = 45966 (fstat)
> trap number = 9
> panic: general protection fault
> cpuid = 9
> time = 1607416693
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe0698887340
> vpanic() at vpanic+0x181/frame 0xfe0698887390
> panic() at panic+0x43/frame 0xfe06988873f0
> trap_fatal() at trap_fatal+0x387/frame 0xfe0698887450
> trap() at trap+0xa4/frame 0xfe0698887560
> calltrap() at calltrap+0x8/frame 0xfe0698887560
> --- trap 0x9, rip = 0x80bc6e22, rsp = 0xfe0698887630, rbp =
> 0xfe06988876b0 ---
> __mtx_lock_sleep() at __mtx_lock_sleep+0xd2/frame 0xfe06988876b0
> __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfe0698887700
> uipc_sockaddr() at uipc_sockaddr+0x4c/frame 0xfe0698887730
> soo_fill_kinfo() at soo_fill_kinfo+0x11e/frame 0xfe0698887770
> kern_proc_filedesc_out() at kern_proc_filedesc_out+0xb57/frame
> 0xfe0698887810
> sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame
> 0xfe0698887890
> sysctl_root_handler_locked() at sysctl_root_handler_locked+0x9c/frame
> 0xfe06988878e0
> sysctl_root() at sysctl_root+0x20d/frame 0xfe0698887960
> userland_sysctl() at userland_sysctl+0x180/frame 0xfe0698887a10
> sys___sysctl() at sys___sysctl+0x5f/frame 0xfe0698887ac0
> amd64_syscall() at amd64_syscall+0x147/frame 0xfe0698887bf0
> fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfe0698887bf0
> --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8003948ea, rsp =
> 0x7fffc138, rbp = 0x7fffc170 ---
>
> https://people.freebsd.org/~pho/stress/log/log0004.txt
So here the unpcb is freed, and indeed the file itself has been closed:
$3 = {f_flag = 0x3, f_count = 0x0, f_data = 0x0, f_ops = 0x81901f50
,
f_vnode = 0x0, f_cred = 0xf80248beb600, f_type = 0x2, f_vnread_flags
= 0x0,
{f_seqcount = {0x0, 0x0}, f_pipegen = 0x0}, f_nextoff = {0x0, 0x0},
f_vnun = {fvn_cdevpriv = 0x0, fvn_advice = 0x0}, f_offset = 0x0}
However, it must have happened very recently because soo_fill_kinfo()
dereferences fp->f_data and yet we did not panic due to a null
dereference.
kern_proc_filedesc_out() holds the fdtable shared lock thoughout all of
this, which is supposed to prevent the table entry from being freed
since that requires the exclusive lock.
Could you show fdp->fd_ofiles[3] and fdp->fd_map[0] from frame 26?
___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
panic: general protection fault from uipc_sockaddr+0x4c
I just got this panic: Fatal trap 9: general protection fault while in kernel mode cpuid = 9; apic id = 09 instruction pointer = 0x20:0x80bc6e22 stack pointer = 0x28:0xfe0698887630 frame pointer = 0x28:0xfe06988876b0 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 45966 (fstat) trap number = 9 panic: general protection fault cpuid = 9 time = 1607416693 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe0698887340 vpanic() at vpanic+0x181/frame 0xfe0698887390 panic() at panic+0x43/frame 0xfe06988873f0 trap_fatal() at trap_fatal+0x387/frame 0xfe0698887450 trap() at trap+0xa4/frame 0xfe0698887560 calltrap() at calltrap+0x8/frame 0xfe0698887560 --- trap 0x9, rip = 0x80bc6e22, rsp = 0xfe0698887630, rbp = 0xfe06988876b0 --- __mtx_lock_sleep() at __mtx_lock_sleep+0xd2/frame 0xfe06988876b0 __mtx_lock_flags() at __mtx_lock_flags+0xe5/frame 0xfe0698887700 uipc_sockaddr() at uipc_sockaddr+0x4c/frame 0xfe0698887730 soo_fill_kinfo() at soo_fill_kinfo+0x11e/frame 0xfe0698887770 kern_proc_filedesc_out() at kern_proc_filedesc_out+0xb57/frame 0xfe0698887810 sysctl_kern_proc_filedesc() at sysctl_kern_proc_filedesc+0x7d/frame 0xfe0698887890 sysctl_root_handler_locked() at sysctl_root_handler_locked+0x9c/frame 0xfe06988878e0 sysctl_root() at sysctl_root+0x20d/frame 0xfe0698887960 userland_sysctl() at userland_sysctl+0x180/frame 0xfe0698887a10 sys___sysctl() at sys___sysctl+0x5f/frame 0xfe0698887ac0 amd64_syscall() at amd64_syscall+0x147/frame 0xfe0698887bf0 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfe0698887bf0 --- syscall (202, FreeBSD ELF64, sys___sysctl), rip = 0x8003948ea, rsp = 0x7fffc138, rbp = 0x7fffc170 --- https://people.freebsd.org/~pho/stress/log/log0004.txt - Peter ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: [CAM] panic / general protection fault while in kernel mode on head r256931
On 10/23/2013 8:14 AM, Alexander Motin wrote: On 23.10.2013 14:22, Bryan Drewery wrote: On 10/23/2013 1:59 AM, Alexander Motin wrote: Unfortunately I have never seen that, but I'll try to look harder. Could you please tell a bit about your controllers/devices? What triggered the issue? (Sorry for confusing pastes in email, bad client) Not sure what was done, I was just editing files in vim. Please try updating to r256960. The fixed problem looked very alike, and was reliably reproducible under PowerPC QEMU just on boot due to some timing magic. Thanks, updating to r257007 has been stable for me again. -- Regards, Bryan Drewery signature.asc Description: OpenPGP digital signature
Re: [CAM] panic / general protection fault while in kernel mode on head r256931
Hi. Unfortunately I have never seen that, but I'll try to look harder. Could you please tell a bit about your controllers/devices? What triggered the issue? If you still have the full dumps, could you check values of ccb_h, ccb_h-cbfcnp, ccb_h-path and ccb_h-path-periph for the deadcode somewhere or something? On 23.10.2013 03:25, Bryan Drewery wrote: Encountered twice: Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0x802e612a stack pointer = 0x28:0xfe118af62b20 frame pointer = 0x28:0xfe118af62b60 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 2 (doneq0) trap number = 9 panic: general protection fault cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe118af62690 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe118af62740 vpanic() at vpanic+0x126/frame 0xfe118af62780 panic() at panic+0x43/frame 0xfe118af627e0 trap_fatal() at trap_fatal+0x3a2/frame 0xfe118af62840 trap() at trap+0x863/frame 0xfe118af62a60 calltrap() at calltrap+0x8/frame 0xfe118af62a60 --- trap 0x9, rip = 0x802e612a, rsp = 0xfe118af62b20, rbp = 0xfe118af62b60 --- xpt_done_process() at xpt_done_process+0x36a/frame 0xfe118af62b60 xpt_done_td() at xpt_done_td+0x13d/frame 0xfe118af62bb0 fork_exit() at fork_exit+0x84/frame 0xfe118af62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe118af62bf0 --- trap 0, rip = 0, rsp = 0xfe118af62cb0, rbp = 0 --- 2nd: Fatal trap 9: general protection fault while in kernel mode cpuid = 5; apic id = 05 instruction pointer = 0x20:0x802e612a stack pointer = 0x28:0xfe118af62b20 frame pointer = 0x28:0xfe118af62b60 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 2 (doneq0) trap number = 9 panic: general protection fault cpuid = 5 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe118af62690 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe118af62740 vpanic() at vpanic+0x126/frame 0xfe118af62780 panic() at panic+0x43/frame 0xfe118af627e0 trap_fatal() at trap_fatal+0x3a2/frame 0xfe118af62840 trap() at trap+0x863/frame 0xfe118af62a60 calltrap() at calltrap+0x8/frame 0xfe118af62a60 --- trap 0x9, rip = 0x802e612a, rsp = 0xfe118af62b20, rbp = 0xfe118af62b60 --- xpt_done_process() at xpt_done_process+0x36a/frame 0xfe118af62b60 xpt_done_td() at xpt_done_td+0x13d/frame 0xfe118af62bb0 fork_exit() at fork_exit+0x84/frame 0xfe118af62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe118af62bf0 --- trap 0, rip = 0, rsp = 0xfe118af62cb0, rbp = 0 --- (kgdb) #0 doadump (textdump=1) at pcpu.h:219 #1 0x808bbb27 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:447 #2 0x808bc035 in vpanic (fmt=value optimized out, ap=value optimized out) at /usr/src/sys/kern/kern_shutdown.c:754 #3 0x808bc083 in panic (fmt=value optimized out) at /usr/src/sys/kern/kern_shutdown.c:683 #4 0x80cb62d2 in trap_fatal (frame=value optimized out, eva=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:882 #5 0x80cb5f13 in trap (frame=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:224 #6 0x80c9b3a2 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0x802e612a in xpt_done_process (ccb_h=0xf80145911800) at /usr/src/sys/cam/cam_xpt.c:5242 #8 0x802e905d in xpt_done_td (arg=0x8144db80) at /usr/src/sys/cam/cam_xpt.c:5274 #9 0x8088b064 in fork_exit ( callout=0x802e8f20 xpt_done_td, arg=0x8144db80, frame=0xfe118af62c00) at /usr/src/sys/kern/kern_fork.c:995 #10 0x80c9b8de in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606 #11 0x in ?? () Current language: auto; currently minimal (kgdb) -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [CAM] panic / general protection fault while in kernel mode on head r256931
= 0xdeadc0dedeadc0de}}, periph_links = {le = {le_next =
0xdeadc0dedeadc0de, le_prev = 0xdeadc0dedeadc0de}, sle = {sle_next =
0xdeadc0dedeadc0de}, tqe = {
tqe_next = 0xdeadc0dedeadc0de, tqe_prev = 0xdeadc0dedeadc0de}, stqe =
{stqe_next = 0xdeadc0dedeadc0de}}, retry_count = 3735929054, cbfcnp =
0xdeadc0dedeadc0de,
func_code = 3735929054, status = 3735929054, path = 0xdeadc0dedeadc0de,
path_id = 3735929054, target_id = 3735929054, target_lun = 3735929054,
ext_lun = {lun64 = 16045693110842147038,
lun = ��}, flags = 3735929054, xflags = 3735929054, periph_priv =
{entries = {{ptr = 0xdeadc0dedeadc0de, field = 16045693110842147038, bytes =
��}, {
ptr = 0xdeadc0dedeadc0de, field = 16045693110842147038, bytes =
��}}, bytes = }, sim_priv = {entries = {{ptr = 0xdeadc0dedeadc0de,
field = 16045693110842147038, bytes = ��}, {ptr =
0xdeadc0dedeadc0de, field = 16045693110842147038, bytes = ��}}, bytes =
}, qos = {
etime = 0xdeadc0dedeadc0de, sim_data = 16045693110842147038, periph_data
= 16045693110842147038}, timeout = 3735929054, softtimeout = {tv_sec =
-2401050962867404578,
tv_usec = -2401050962867404578}}
(kgdb) p *ccb_h-cbfcnp
Cannot access memory at address 0xdeadc0dedeadc0de
(kgdb) p ccb_h-cbfcnp
$5 = (void (*)(struct cam_periph *, union ccb *)) 0xdeadc0dedeadc0de
(kgdb) p ccb_h-path
$6 = (struct cam_path *) 0xdeadc0dedeadc0de
(kgdb) up
#8 0x802e905d in xpt_done_td (arg=0x8144db80) at
/usr/src/sys/cam/cam_xpt.c:5274
Line number 5274 out of range; /usr/src/sys/cam/cam_xpt.c has 5056 lines.
(kgdb) print *ccb_h
$7 = {pinfo = {priority = 2161940088, generation = 4294967295, index =
16973824}, xpt_links = {le = {le_next = 0xfe6d3400, le_prev =
0xf8000f3e4000}, sle = {
sle_next = 0xfe6d3400}, tqe = {tqe_next = 0xfe6d3400,
tqe_prev = 0xf8000f3e4000}, stqe = {stqe_next = 0xfe6d3400}},
sim_links = {le = {le_next = 0x0,
le_prev = 0x0}, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev
= 0x0}, stqe = {stqe_next = 0x0}}, periph_links = {le = {le_next =
0x802e64a0,
le_prev = 0xf80014197800}, sle = {sle_next = 0x802e64a0},
tqe = {tqe_next = 0x802e64a0, tqe_prev = 0xf80014197800}, stqe =
{stqe_next = 0x802e64a0}},
retry_count = 0, cbfcnp = 0, func_code = XPT_NOOP, status = 0, path = 0x0,
path_id = 0, target_id = 0, target_lun = 0, ext_lun = {lun64 = 0, lun =
\000\000\000\000\000\000\000},
flags = 0, xflags = 0, periph_priv = {entries = {{ptr = 0x0, field = 0,
bytes = \000\000\000\000\000\000\000}, {ptr = 0x0, field = 0, bytes =
\000\000\000\000\000\000\000}},
bytes = '\0' repeats 15 times}, sim_priv = {entries = {{ptr = 0x0,
field = 0, bytes = \000\000\000\000\000\000\000}, {ptr = 0x0, field = 0,
bytes = \000\000\000\000\000\000\000}}, bytes = '\0' repeats 15
times}, qos = {etime = 0x0, sim_data = 0, periph_data = 0}, timeout = 0,
softtimeout = {tv_sec = 0,
tv_usec = 0}}
(kgdb) print ccb_h-cbfcnp
$8 = (void (*)(struct cam_periph *, union ccb *)) 0
(kgdb) print ccb_h-path
$9 = (struct cam_path *) 0x0
.
On 23.10.2013 03:25, Bryan Drewery wrote:
Encountered twice:
Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer = 0x20:0x802e612a
stack pointer = 0x28:0xfe118af62b20
frame pointer = 0x28:0xfe118af62b60
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags= interrupt enabled, resume, IOPL = 0
current process = 2 (doneq0)
trap number = 9
panic: general protection fault
cpuid = 0
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
0xfe118af62690
kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe118af62740
vpanic() at vpanic+0x126/frame 0xfe118af62780
panic() at panic+0x43/frame 0xfe118af627e0
trap_fatal() at trap_fatal+0x3a2/frame 0xfe118af62840
trap() at trap+0x863/frame 0xfe118af62a60
calltrap() at calltrap+0x8/frame 0xfe118af62a60
--- trap 0x9, rip = 0x802e612a, rsp = 0xfe118af62b20, rbp
= 0xfe118af62b60 ---
xpt_done_process() at xpt_done_process+0x36a/frame 0xfe118af62b60
xpt_done_td() at xpt_done_td+0x13d/frame 0xfe118af62bb0
fork_exit() at fork_exit+0x84/frame 0xfe118af62bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfe118af62bf0
--- trap 0, rip = 0, rsp = 0xfe118af62cb0, rbp = 0 ---
2nd:
Fatal trap 9: general protection fault while in kernel mode
cpuid = 5; apic id = 05
instruction pointer = 0x20:0x802e612a
stack pointer = 0x28:0xfe118af62b20
frame pointer = 0x28:0xfe118af62b60
code segment= base 0x0, limit 0xf, type 0x1b
= DPL 0, pres 1
Re: [CAM] panic / general protection fault while in kernel mode on head r256931
On 23.10.2013 14:22, Bryan Drewery wrote: On 10/23/2013 1:59 AM, Alexander Motin wrote: Unfortunately I have never seen that, but I'll try to look harder. Could you please tell a bit about your controllers/devices? What triggered the issue? (Sorry for confusing pastes in email, bad client) Not sure what was done, I was just editing files in vim. Please try updating to r256960. The fixed problem looked very alike, and was reliably reproducible under PowerPC QEMU just on boot due to some timing magic. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
[CAM] panic / general protection fault while in kernel mode on head r256931
Encountered twice: Fatal trap 9: general protection fault while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0x802e612a stack pointer = 0x28:0xfe118af62b20 frame pointer = 0x28:0xfe118af62b60 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 2 (doneq0) trap number = 9 panic: general protection fault cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe118af62690 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe118af62740 vpanic() at vpanic+0x126/frame 0xfe118af62780 panic() at panic+0x43/frame 0xfe118af627e0 trap_fatal() at trap_fatal+0x3a2/frame 0xfe118af62840 trap() at trap+0x863/frame 0xfe118af62a60 calltrap() at calltrap+0x8/frame 0xfe118af62a60 --- trap 0x9, rip = 0x802e612a, rsp = 0xfe118af62b20, rbp = 0xfe118af62b60 --- xpt_done_process() at xpt_done_process+0x36a/frame 0xfe118af62b60 xpt_done_td() at xpt_done_td+0x13d/frame 0xfe118af62bb0 fork_exit() at fork_exit+0x84/frame 0xfe118af62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe118af62bf0 --- trap 0, rip = 0, rsp = 0xfe118af62cb0, rbp = 0 --- 2nd: Fatal trap 9: general protection fault while in kernel mode cpuid = 5; apic id = 05 instruction pointer = 0x20:0x802e612a stack pointer = 0x28:0xfe118af62b20 frame pointer = 0x28:0xfe118af62b60 code segment= base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags= interrupt enabled, resume, IOPL = 0 current process = 2 (doneq0) trap number = 9 panic: general protection fault cpuid = 5 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe118af62690 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfe118af62740 vpanic() at vpanic+0x126/frame 0xfe118af62780 panic() at panic+0x43/frame 0xfe118af627e0 trap_fatal() at trap_fatal+0x3a2/frame 0xfe118af62840 trap() at trap+0x863/frame 0xfe118af62a60 calltrap() at calltrap+0x8/frame 0xfe118af62a60 --- trap 0x9, rip = 0x802e612a, rsp = 0xfe118af62b20, rbp = 0xfe118af62b60 --- xpt_done_process() at xpt_done_process+0x36a/frame 0xfe118af62b60 xpt_done_td() at xpt_done_td+0x13d/frame 0xfe118af62bb0 fork_exit() at fork_exit+0x84/frame 0xfe118af62bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe118af62bf0 --- trap 0, rip = 0, rsp = 0xfe118af62cb0, rbp = 0 --- (kgdb) #0 doadump (textdump=1) at pcpu.h:219 #1 0x808bbb27 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:447 #2 0x808bc035 in vpanic (fmt=value optimized out, ap=value optimized out) at /usr/src/sys/kern/kern_shutdown.c:754 #3 0x808bc083 in panic (fmt=value optimized out) at /usr/src/sys/kern/kern_shutdown.c:683 #4 0x80cb62d2 in trap_fatal (frame=value optimized out, eva=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:882 #5 0x80cb5f13 in trap (frame=value optimized out) at /usr/src/sys/amd64/amd64/trap.c:224 #6 0x80c9b3a2 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:232 #7 0x802e612a in xpt_done_process (ccb_h=0xf80145911800) at /usr/src/sys/cam/cam_xpt.c:5242 #8 0x802e905d in xpt_done_td (arg=0x8144db80) at /usr/src/sys/cam/cam_xpt.c:5274 #9 0x8088b064 in fork_exit ( callout=0x802e8f20 xpt_done_td, arg=0x8144db80, frame=0xfe118af62c00) at /usr/src/sys/kern/kern_fork.c:995 #10 0x80c9b8de in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:606 #11 0x in ?? () Current language: auto; currently minimal (kgdb) -- Regards, Bryan Drewery signature.asc Description: OpenPGP digital signature
