Yaniv, I'm adding dtrace-discuss to this email for reasons that will be obvious immediately :-) - see below
Yaniv Aknin wrote: > When volumes approach 90% usage, and under medium/light load (zpool > iostat reports 50mb/s and 750iops reads), some creat64 system calls take > over 50 seconds to complete (observed with 'truss -D touch'). When doing > manual tests, I've seen similar times on unlink() calls (truss -D rm). > > I'd like to stress this happens on /some/ of the calls, maybe every > 100th manual call (I scripted the test), which (along with normal system > operations) would probably be every 10,000th or 100,000th call. I'd suggest you do something like this (not tested, so syntax errors etc may be lurking; I'd also suggest you get the DTrace guide off of opensolaris.org and read the chapter about speculations): #!/usr/sbin/drace -Fs int limit ONE_SECOND /* you need to replace this with 10^9, I think) syscall::creat64:entry { self->spec = speculation(); speculate(self->spec); self->ts=timestamp(); self->duration = 0; } fbt:::entry, fbt:::return /self->spec/ { speculate(self->spec); } syscall::creat64:return /self->spec/ { speculate(self->spec); self->duration = timestamp() - self->ts; } syscall::creat64:return /self->duration > limit/ { commit(self->spec); self->spec = 0; } syscall::creat64:return /self->spec/ { discard(self->spec); self->spec = 0; } you may need to use a different timestamp (walltimestamp?); and perhaps you'll want to somehow reduce the number of fbt probes, but that's up to you. I hope you can take it from here. cheers Michael -- Michael Schuster Sun Microsystems, Inc. recursion, n: see 'recursion' _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss