Re: panic: in_pcblookup_local (?)

2013-05-04 Thread Ian FREISLICH
Peter Wemm wrote: offending revision. I've started a binary search. I'll let you know what that turns up. Thanks, and sorry for getting my Ian's mixed up. :-/ -- John Baldwin There's been two separate machines, at least twice each on this exact panic / trace. Always with

Re: panic: in_pcblookup_local (?)

2013-05-03 Thread Peter Wemm
On Thu, May 2, 2013 at 11:32 AM, John Baldwin j...@freebsd.org wrote: On Thursday, May 02, 2013 1:53:47 pm Ian FREISLICH wrote: John Baldwin wrote: On Thursday, May 02, 2013 7:25:08 am Robert N. M. Watson wrote: On 2 May 2013, at 11:42, Glen Barber wrote: Hmm. Perhaps it would be

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread Robert N. M. Watson
On 2 May 2013, at 01:57, Glen Barber wrote: So, I am admittedly not too familiar with DDB. In fact, I just now realize the kernel is built without DDB... DDB is a very powerful tool in that it's been custom-developed to help debug common kernel panics. It lacks some of the flexibility, and

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread Glen Barber
On Thu, May 02, 2013 at 10:27:39AM +0100, Robert N. M. Watson wrote: On 2 May 2013, at 01:57, Glen Barber wrote: So, I am admittedly not too familiar with DDB. In fact, I just now realize the kernel is built without DDB... DDB is a very powerful tool in that it's been custom-developed

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread Robert N. M. Watson
On 2 May 2013, at 11:42, Glen Barber wrote: Hmm. Perhaps it would be worthwhile for me to rebuild the current kernel with DDB support. It looks like the machine has panicked a few times over the last two weeks or so, but based on the timestamps of the crash dumps and nagios complaints,

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread Glen Barber
On Thu, May 02, 2013 at 12:25:08PM +0100, Robert N. M. Watson wrote: On 2 May 2013, at 11:42, Glen Barber wrote: Hmm. Perhaps it would be worthwhile for me to rebuild the current kernel with DDB support. It looks like the machine has panicked a few times over the last two weeks or so,

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread John Baldwin
On Thursday, May 02, 2013 5:27:39 am Robert N. M. Watson wrote: On 2 May 2013, at 01:57, Glen Barber wrote: So, I am admittedly not too familiar with DDB. In fact, I just now realize the kernel is built without DDB... DDB is a very powerful tool in that it's been custom-developed to

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread John Baldwin
On Thursday, May 02, 2013 7:25:08 am Robert N. M. Watson wrote: On 2 May 2013, at 11:42, Glen Barber wrote: Hmm. Perhaps it would be worthwhile for me to rebuild the current kernel with DDB support. It looks like the machine has panicked a few times over the last two weeks or so, but

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread Ian FREISLICH
John Baldwin wrote: On Thursday, May 02, 2013 7:25:08 am Robert N. M. Watson wrote: On 2 May 2013, at 11:42, Glen Barber wrote: Hmm. Perhaps it would be worthwhile for me to rebuild the current kernel with DDB support. It looks like the machine has panicked a few times over the

Re: panic: in_pcblookup_local (?)

2013-05-02 Thread John Baldwin
On Thursday, May 02, 2013 1:53:47 pm Ian FREISLICH wrote: John Baldwin wrote: On Thursday, May 02, 2013 7:25:08 am Robert N. M. Watson wrote: On 2 May 2013, at 11:42, Glen Barber wrote: Hmm. Perhaps it would be worthwhile for me to rebuild the current kernel with DDB

Re: panic: in_pcblookup_local (?)

2013-05-01 Thread John Baldwin
On Tuesday, April 30, 2013 5:19:08 pm Glen Barber wrote: On Tue, Apr 30, 2013 at 04:53:13PM -0400, John Baldwin wrote: Try 'p phd' to start. INP_PCBPORTHASH is a macro, so you will have to do it by hand: 'p pcbinfo-ipi_porthashbase[lport pcbinfo-ipi_porthashmask]' (That should be

Re: panic: in_pcblookup_local (?)

2013-05-01 Thread Robert N. M. Watson
On 1 May 2013, at 16:56, John Baldwin wrote: It looks like the ipi_hash_lock is locked (and udp_connect() locks it), so I think the offending code is somewhere else. Also, I can't find anything that removes an inp without hold the correct pcbinfo lock. Only thing I can think of is if the

Re: panic: in_pcblookup_local (?)

2013-05-01 Thread Glen Barber
On Wed, May 01, 2013 at 06:45:53PM +0100, Robert N. M. Watson wrote: On 1 May 2013, at 16:56, John Baldwin wrote: It looks like the ipi_hash_lock is locked (and udp_connect() locks it), so I think the offending code is somewhere else. Also, I can't find anything that removes an

Re: panic: in_pcblookup_local (?)

2013-05-01 Thread Robert N. M. Watson
On 1 May 2013, at 19:03, Glen Barber wrote: I'll need to catch up on this thread later, but a few questions: Do we know if the application in question is multithreaded, and if so, might it be attempting concurrent operations on this socket? I do not know if zabbix-agent is multithreaded,

Re: panic: in_pcblookup_local (?)

2013-05-01 Thread John Baldwin
On Wednesday, May 01, 2013 2:08:57 pm Robert N. M. Watson wrote: On 1 May 2013, at 19:03, Glen Barber wrote: I'll need to catch up on this thread later, but a few questions: Do we know if the application in question is multithreaded, and if so, might it be attempting concurrent

Re: panic: in_pcblookup_local (?)

2013-05-01 Thread Glen Barber
On Wed, May 01, 2013 at 02:30:36PM -0400, John Baldwin wrote: On Wednesday, May 01, 2013 2:08:57 pm Robert N. M. Watson wrote: If in DDB, it would be useful to do a ps so we can identify threads in the process, and in particular, whether they might be in the kernel around the moment of

Re: panic: in_pcblookup_local (?)

2013-04-30 Thread John Baldwin
On Monday, April 29, 2013 8:35:52 pm Glen Barber wrote: On Mon, Apr 29, 2013 at 12:24:06PM -0400, John Baldwin wrote: On Sunday, April 28, 2013 12:02:56 am Glen Barber wrote: On Sat, Apr 27, 2013 at 10:17:32AM +0200, Ian FREISLICH wrote: Hi I've been getting the following panic

Re: panic: in_pcblookup_local (?)

2013-04-30 Thread Glen Barber
On Tue, Apr 30, 2013 at 04:53:13PM -0400, John Baldwin wrote: Try 'p phd' to start. INP_PCBPORTHASH is a macro, so you will have to do it by hand: 'p pcbinfo-ipi_porthashbase[lport pcbinfo-ipi_porthashmask]' (That should be what 'porthash' is.) Thanks for the pointers. (Hah!)

Re: panic: in_pcblookup_local (?)

2013-04-29 Thread John Baldwin
On Sunday, April 28, 2013 12:02:56 am Glen Barber wrote: On Sat, Apr 27, 2013 at 10:17:32AM +0200, Ian FREISLICH wrote: Hi I've been getting the following panic on recent current r249717. Sadly the crashdump is useless. I just saw similar panic on 10-CURRENT r249588. Fatal trap

Re: panic: in_pcblookup_local (?)

2013-04-29 Thread Glen Barber
On Mon, Apr 29, 2013 at 12:24:06PM -0400, John Baldwin wrote: On Sunday, April 28, 2013 12:02:56 am Glen Barber wrote: On Sat, Apr 27, 2013 at 10:17:32AM +0200, Ian FREISLICH wrote: Hi I've been getting the following panic on recent current r249717. Sadly the crashdump is useless.

panic: in_pcblookup_local (?)

2013-04-27 Thread Ian FREISLICH
Hi I've been getting the following panic on recent current r249717. Sadly the crashdump is useless. Fatal trap 9: general protection fault while in kernel mode cpuid = 15; apic id = 0f instruction pointer = 0x20:0x80546fbc stack pointer = 0x28:0xff846b60 frame

Re: panic: in_pcblookup_local (?)

2013-04-27 Thread Glen Barber
On Sat, Apr 27, 2013 at 10:17:32AM +0200, Ian FREISLICH wrote: Hi I've been getting the following panic on recent current r249717. Sadly the crashdump is useless. I just saw similar panic on 10-CURRENT r249588. Fatal trap 9: general protection fault while in kernel mode cpuid = 15; apic