Re: RELENG_6: I/O deadlock under load

2006-11-03 Thread Ulrich Spoerlein
On 10/28/06, Christian S.J. Peron [EMAIL PROTECTED] wrote: It almost looks as if a user frequently runs gmirror(8) to query the status of their array. Under a high load situation, the worker is busy, so at one un-lucky momment, gmirror(8) is run: (1) gmirror(8) waits for sc-sc_lock owned

Re: RELENG_6: I/O deadlock under load

2006-10-28 Thread Ulrich Spoerlein
Ulrich Spoerlein wrote: Our fileserver deadlocked, again. It is running RELENG_6 checked out yesterday. I have enabled DDB, WITNESS and INVARIANTS and have it hooked up via serial console. Happend again, now I have DEBUG_LOCKS and DEBUG_VFS_LOCK included. There are hundreds of cron processes

Re: RELENG_6: I/O deadlock under load

2006-10-28 Thread Christian S.J. Peron
It almost looks as if a user frequently runs gmirror(8) to query the status of their array. Under a high load situation, the worker is busy, so at one un-lucky momment, gmirror(8) is run: (1) gmirror(8) waits for sc-sc_lock owned by the worker (2) The worker then drops the lock (3)

Re: RELENG_6: I/O deadlock under load

2006-10-28 Thread Christian S.J. Peron
Sorry, I forgot to include the chunk of code from the gmirror worker thread which made me suspect this could be the problem: [..] /* Get first request from the queue. */ mtx_lock(sc-sc_queue_mtx); bp = bioq_first(sc-sc_queue); if

RELENG_6: I/O deadlock under load

2006-10-27 Thread Ulrich Spoerlein
Hi all, Our fileserver deadlocked, again. It is running RELENG_6 checked out yesterday. I have enabled DDB, WITNESS and INVARIANTS and have it hooked up via serial console. I can not give out shell access, but I can run any command you might consider useful, here's more details: The system has