Re: options WITNESS and locks
Yony Yossef wrote: Problem solved :-) Thank you very much for your patience Kris.. Glad to help! Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: options WITNESS and locks
Problem solved :-) Thank you very much for your patience Kris.. On Tue, Jul 15, 2008 at 5:14 PM, Kris Kennaway <[EMAIL PROTECTED]> wrote: > Yony Yossef wrote: > >> >>Something is still unclear. All my locks are MTX_DEF type, which >>means >>sleepable, including the one specified in the crash report >>(MTNIC state >>semaphore). >>This crash happens when I try to call bus_resource_alloc_any for >>a SYS_REQ >>which is trying to obtain the second lock (ACPI root bus). How >>come my >>MTX_DEF lock is being treated as as non-sleepable? >> >> >>MTX_DEF locks *are* non-sleepable :-) >> >>Kris >> >> Good news :-) >> Yet the documentation is confusing me: >> man pages say: >> MTX_DEFDefault mutexes will always allow the current thread to be >>suspended to avoid deadlock conditions against >> interrupt >>threads. The implementation of this lock type may spin >>for a while before suspending the current thread. >> and in mutex.h: >> /* >> * Mutex types and options passed to mtx_init(). MTX_QUIET and MTX_DUPOK >> * can also be passed in. >> */ >> #define MTX_DEF 0x /* DEFAULT (sleep) lock */ >> #define MTX_SPIN 0x0001 /* Spin lock (disables interrupts) */ >> #define MTX_RECURSE 0x0004 /* Option: lock allowed to recurse */ >> #define MTX_NOWITNESS 0x0008 /* Don't do any witness checking. */ >> #define MTX_NOPROFILE 0x0020 /* Don't profile this lock */ >> What does the "DEFAULT (sleep) lock" comment near MTX_DEF means? >> And which ones are sleepable?.. I guess I'm missing something here. >> > > "sleep mutex" means that the mutex holder may be put to sleep by the kernel > while the mutex is held by another process (if the lock holder is running > then it will spin, though). It is in contrast to "spin" mutexes that always > spin forever while the lock is held, even when it might be inefficient for > them to do so because the lock holder is no longer running after the CPU > context switched to another process before it released the lock. > > This describes the behaviour of the mutex while it is waiting to acquire a > lock. > > "sleepable" vs "non-sleepable" locks means what it is legal for your code > to do while holding a mutex and doesn't relate to the internal > implementation of the lock. > > This describes what it is legal for your code to do once it has acquired > the lock. > > Kris > > ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: options WITNESS and locks
Yony Yossef wrote: Something is still unclear. All my locks are MTX_DEF type, which means sleepable, including the one specified in the crash report (MTNIC state semaphore). This crash happens when I try to call bus_resource_alloc_any for a SYS_REQ which is trying to obtain the second lock (ACPI root bus). How come my MTX_DEF lock is being treated as as non-sleepable? MTX_DEF locks *are* non-sleepable :-) Kris Good news :-) Yet the documentation is confusing me: man pages say: MTX_DEFDefault mutexes will always allow the current thread to be suspended to avoid deadlock conditions against interrupt threads. The implementation of this lock type may spin for a while before suspending the current thread. and in mutex.h: /* * Mutex types and options passed to mtx_init(). MTX_QUIET and MTX_DUPOK * can also be passed in. */ #define MTX_DEF 0x /* DEFAULT (sleep) lock */ #define MTX_SPIN 0x0001 /* Spin lock (disables interrupts) */ #define MTX_RECURSE 0x0004 /* Option: lock allowed to recurse */ #define MTX_NOWITNESS 0x0008 /* Don't do any witness checking. */ #define MTX_NOPROFILE 0x0020 /* Don't profile this lock */ What does the "DEFAULT (sleep) lock" comment near MTX_DEF means? And which ones are sleepable?.. I guess I'm missing something here. "sleep mutex" means that the mutex holder may be put to sleep by the kernel while the mutex is held by another process (if the lock holder is running then it will spin, though). It is in contrast to "spin" mutexes that always spin forever while the lock is held, even when it might be inefficient for them to do so because the lock holder is no longer running after the CPU context switched to another process before it released the lock. This describes the behaviour of the mutex while it is waiting to acquire a lock. "sleepable" vs "non-sleepable" locks means what it is legal for your code to do while holding a mutex and doesn't relate to the internal implementation of the lock. This describes what it is legal for your code to do once it has acquired the lock. Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: options WITNESS and locks
> > > Something is still unclear. All my locks are MTX_DEF type, which means >> sleepable, including the one specified in the crash report (MTNIC state >> semaphore). >> This crash happens when I try to call bus_resource_alloc_any for a SYS_REQ >> which is trying to obtain the second lock (ACPI root bus). How come my >> MTX_DEF lock is being treated as as non-sleepable? >> > > MTX_DEF locks *are* non-sleepable :-) > > Kris > Good news :-) Yet the documentation is confusing me: man pages say: MTX_DEFDefault mutexes will always allow the current thread to be suspended to avoid deadlock conditions against interrupt threads. The implementation of this lock type may spin for a while before suspending the current thread. and in mutex.h: /* * Mutex types and options passed to mtx_init(). MTX_QUIET and MTX_DUPOK * can also be passed in. */ #define MTX_DEF 0x /* DEFAULT (sleep) lock */ #define MTX_SPIN 0x0001 /* Spin lock (disables interrupts) */ #define MTX_RECURSE 0x0004 /* Option: lock allowed to recurse */ #define MTX_NOWITNESS 0x0008 /* Don't do any witness checking. */ #define MTX_NOPROFILE 0x0020 /* Don't profile this lock */ What does the "DEFAULT (sleep) lock" comment near MTX_DEF means? And which ones are sleepable?.. I guess I'm missing something here. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: options WITNESS and locks
Yony Yossef wrote: Something is still unclear. All my locks are MTX_DEF type, which means sleepable, including the one specified in the crash report (MTNIC state semaphore). This crash happens when I try to call bus_resource_alloc_any for a SYS_REQ which is trying to obtain the second lock (ACPI root bus). How come my MTX_DEF lock is being treated as as non-sleepable? MTX_DEF locks *are* non-sleepable :-) Kris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: options WITNESS and locks
Hi Kris, Please see below > Hi all. >> >> I'm trying to debug a "spinlock held too long" error. >> Therefore I thought compiling my kernel with "options WITNESS" would be a >> good idea. >> >> Using the WITNESS kernel I cannot load my driver with any MTX_SPIN mutex. >> I had to change it all to MTX_DEF since every MTX_SPIN got me this error: >> >> panic: lock (network driver) spin mutex does not match earlier (sleep >> mutex) >> lock >> > > It means that somewhere you are treating a mutex with that name as a sleep > mutex and in other places as a spin mutex. WITNESS works on the lock name > so this may or may not be a bug. > > However, default mutexes should be used in most cases anyway (bear in mind > that default mutexes will also spin when it makes sense for them to do so). > > So I changed it all to MTX_DEF, just to see if things will work. >> Now I can load my driver, but calling ifconfig shows a new crash: >> >> mtnic0: Activating port:1 >> mtnic0: Ethernet address: 00:00:00:00:08:88 >> mtnic0: Activating port:2 >> mtnic1: Ethernet address: 00:00:00:00:08:89 >> lock order reversal: (sleepable after non-sleepable) >> 1st 0x81379010 MTNIC state semaphore (MTNIC state semaphore) @ >> mtnic_netdev.c:1855 >> 2nd 0x809eee00 ACPI root bus (ACPI root bus) @ >> /usr/src/sys/dev/acpica/acpi.c:1022 >> > > You are acquiring a lock that is "sleepable" (i.e. legal for consumers to > sleep while holding it), after first acquiring another lock that is not > sleepable. The danger is that if some code sleeps while holding both the > first and second lock, then other code that tries to acquire the first lock > will deadlock indefinitely. > > This is often due to a programming or design error. > > Kris Something is still unclear. All my locks are MTX_DEF type, which means sleepable, including the one specified in the crash report (MTNIC state semaphore). This crash happens when I try to call bus_resource_alloc_any for a SYS_REQ which is trying to obtain the second lock (ACPI root bus). How come my MTX_DEF lock is being treated as as non-sleepable? > > > KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a >> witness_checkorder() at witness_checkorder+0x559 >> _sx_xlock() at _sx_xlock+0x32 >> acpi_alloc_resource() at acpi_alloc_resource+0x9a >> pci_alloc_resource() at pci_alloc_resource+0x81 >> resource_list_alloc() at resource_list_alloc+0x17a >> pci_alloc_resource() at pci_alloc_resource+0x81 >> bus_alloc_resource() at bus_alloc_resource+0x89 >> mtnic_start_port() at mtnic_start_port+0x4f1 >> mtnic_open() at mtnic_open+0xb2 >> ether_ioctl() at ether_ioctl+0xb5 >> mtnic_ioctl() at mtnic_ioctl+0x3e >> in_ifinit() at in_ifinit+0x8d >> in_control() at in_control+0xc66 >> ifioctl() at ifioctl+0xea >> kern_ioctl() at kern_ioctl+0xa3 >> ioctl() at ioctl+0xf9 >> syscall() at syscall+0x1b5 >> Xfast_syscall() at Xfast_syscall+0xab >> --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x800824cfc, rsp = >> 0x7fffe3b8, rbp = 0x7fffee10 --- >> KDB: enter: witness_checkorder >> [thread pid 1051 tid 100069 ] >> Stopped at kdb_enter+0x31: leave >> db> >> >> >> Can anybody please tell me what is going on here? >> >> -Yony >> ___ >> freebsd-questions@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to " >> [EMAIL PROTECTED]" >> >> >> > ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: options WITNESS and locks
Yony Yossef wrote: Hi all. I'm trying to debug a "spinlock held too long" error. Therefore I thought compiling my kernel with "options WITNESS" would be a good idea. Using the WITNESS kernel I cannot load my driver with any MTX_SPIN mutex. I had to change it all to MTX_DEF since every MTX_SPIN got me this error: panic: lock (network driver) spin mutex does not match earlier (sleep mutex) lock It means that somewhere you are treating a mutex with that name as a sleep mutex and in other places as a spin mutex. WITNESS works on the lock name so this may or may not be a bug. However, default mutexes should be used in most cases anyway (bear in mind that default mutexes will also spin when it makes sense for them to do so). So I changed it all to MTX_DEF, just to see if things will work. Now I can load my driver, but calling ifconfig shows a new crash: mtnic0: Activating port:1 mtnic0: Ethernet address: 00:00:00:00:08:88 mtnic0: Activating port:2 mtnic1: Ethernet address: 00:00:00:00:08:89 lock order reversal: (sleepable after non-sleepable) 1st 0x81379010 MTNIC state semaphore (MTNIC state semaphore) @ mtnic_netdev.c:1855 2nd 0x809eee00 ACPI root bus (ACPI root bus) @ /usr/src/sys/dev/acpica/acpi.c:1022 You are acquiring a lock that is "sleepable" (i.e. legal for consumers to sleep while holding it), after first acquiring another lock that is not sleepable. The danger is that if some code sleeps while holding both the first and second lock, then other code that tries to acquire the first lock will deadlock indefinitely. This is often due to a programming or design error. Kris KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a witness_checkorder() at witness_checkorder+0x559 _sx_xlock() at _sx_xlock+0x32 acpi_alloc_resource() at acpi_alloc_resource+0x9a pci_alloc_resource() at pci_alloc_resource+0x81 resource_list_alloc() at resource_list_alloc+0x17a pci_alloc_resource() at pci_alloc_resource+0x81 bus_alloc_resource() at bus_alloc_resource+0x89 mtnic_start_port() at mtnic_start_port+0x4f1 mtnic_open() at mtnic_open+0xb2 ether_ioctl() at ether_ioctl+0xb5 mtnic_ioctl() at mtnic_ioctl+0x3e in_ifinit() at in_ifinit+0x8d in_control() at in_control+0xc66 ifioctl() at ifioctl+0xea kern_ioctl() at kern_ioctl+0xa3 ioctl() at ioctl+0xf9 syscall() at syscall+0x1b5 Xfast_syscall() at Xfast_syscall+0xab --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x800824cfc, rsp = 0x7fffe3b8, rbp = 0x7fffee10 --- KDB: enter: witness_checkorder [thread pid 1051 tid 100069 ] Stopped at kdb_enter+0x31: leave db> Can anybody please tell me what is going on here? -Yony ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
options WITNESS and locks
Hi all. I'm trying to debug a "spinlock held too long" error. Therefore I thought compiling my kernel with "options WITNESS" would be a good idea. Using the WITNESS kernel I cannot load my driver with any MTX_SPIN mutex. I had to change it all to MTX_DEF since every MTX_SPIN got me this error: panic: lock (network driver) spin mutex does not match earlier (sleep mutex) lock So I changed it all to MTX_DEF, just to see if things will work. Now I can load my driver, but calling ifconfig shows a new crash: mtnic0: Activating port:1 mtnic0: Ethernet address: 00:00:00:00:08:88 mtnic0: Activating port:2 mtnic1: Ethernet address: 00:00:00:00:08:89 lock order reversal: (sleepable after non-sleepable) 1st 0x81379010 MTNIC state semaphore (MTNIC state semaphore) @ mtnic_netdev.c:1855 2nd 0x809eee00 ACPI root bus (ACPI root bus) @ /usr/src/sys/dev/acpica/acpi.c:1022 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a witness_checkorder() at witness_checkorder+0x559 _sx_xlock() at _sx_xlock+0x32 acpi_alloc_resource() at acpi_alloc_resource+0x9a pci_alloc_resource() at pci_alloc_resource+0x81 resource_list_alloc() at resource_list_alloc+0x17a pci_alloc_resource() at pci_alloc_resource+0x81 bus_alloc_resource() at bus_alloc_resource+0x89 mtnic_start_port() at mtnic_start_port+0x4f1 mtnic_open() at mtnic_open+0xb2 ether_ioctl() at ether_ioctl+0xb5 mtnic_ioctl() at mtnic_ioctl+0x3e in_ifinit() at in_ifinit+0x8d in_control() at in_control+0xc66 ifioctl() at ifioctl+0xea kern_ioctl() at kern_ioctl+0xa3 ioctl() at ioctl+0xf9 syscall() at syscall+0x1b5 Xfast_syscall() at Xfast_syscall+0xab --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x800824cfc, rsp = 0x7fffe3b8, rbp = 0x7fffee10 --- KDB: enter: witness_checkorder [thread pid 1051 tid 100069 ] Stopped at kdb_enter+0x31: leave db> Can anybody please tell me what is going on here? -Yony ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
