I see nothing unusual in the lockstat data. I think you're barking up the wrong tree. -- richard
On Mar 25, 2012, at 10:51 PM, Aubrey Li wrote: > On Mon, Mar 26, 2012 at 1:19 PM, Richard Elling > <richard.ell...@richardelling.com> wrote: >> Apologies to the ZFSers, this thread really belongs elsewhere. >> >> Let me explain below: >> >> Root documentation path of apache is in zfs, you see >> it at No.3 at the above dtrace report. >> >> >> The sort is in reverse order. The large number you see below the >> stack trace is the number of times that stack was seen. By far the >> most frequently seen is tmpfs`tmp_readdir > > It's true, but I didn't see any potential issues there. > >> >> >> tmpfs(/tmp) is the place where PHP place the temporary >> folders and files. >> >> >> bingo >> > > I have to paste the lock investigation here again, > > Firstly, you can see which lock is spinning: > ======================================================= > # lockstat -D 10 sleep 2 > > Adaptive mutex spin: 1862701 events in 3.678 seconds (506499 events/sec) > > Count indv cuml rcnt nsec Lock Caller > ------------------------------------------------------------------------------- > 829064 45% 45% 0.00 33280 0xffffff117624a5d0 rrw_enter_read+0x1b > 705001 38% 82% 0.00 30983 0xffffff117624a5d0 rrw_exit+0x1d > 140678 8% 90% 0.00 10546 0xffffff117624a6e0 zfs_zget+0x46 > 37208 2% 92% 0.00 5403 0xffffff114b136840 vn_rele+0x1e > 33926 2% 94% 0.00 5417 0xffffff114b136840 lookuppnatcred+0xc5 > 27188 1% 95% 0.00 1155 vn_vfslocks_buckets+0xd980 > vn_vfslocks_getlock+0x3b > 11073 1% 96% 0.00 1639 vn_vfslocks_buckets+0x4600 > vn_vfslocks_getlock+0x3b > 9321 1% 96% 0.00 1961 0xffffff114b82a680 dnlc_lookup+0x83 > 6929 0% 97% 0.00 1590 0xffffff11573b8f28 > zfs_fastaccesschk_execute+0x6a > 5746 0% 97% 0.00 5935 0xffffff114b136840 lookuppnvp+0x566 > ------------------------------------------------------------------------------- > > Then if you look at the caller of lock(0xffffff117624a5d0), you'll see > it's ZFS, not tmpfs. > ============================================================ > Count indv cuml rcnt nsec Lock Caller > 48494 6% 17% 0.00 145263 0xffffff117624a5d0 rrw_enter_read+0x1b > > nsec ------ Time Distribution ------ count Stack > 256 | 17 rrw_enter+0x2c > 512 | 1120 zfs_root+0x3b > 1024 |@ 1718 fsop_root+0x2d > 2048 |@@ 4834 traverse+0x65 > 4096 |@@@@@@@@@@@ 18569 lookuppnvp+0x446 > 8192 |@@@@ 6620 lookuppnatcred+0x119 > 16384 |@ 2929 lookupnameatcred+0x97 > 32768 |@ 1635 lookupnameat+0x6b > 65536 | 894 cstatat_getvp+0x11e > 131072 | 1249 cstatat64_32+0x5d > 262144 |@ 1620 fstatat64_32+0x4c > 524288 |@ 2474 > _sys_sysenter_post_swapgs+0x149 > > > That's why I post this subject here. Hope it's clear this time. > > Thanks, > -Aubrey -- DTrace Conference, April 3, 2012, http://wiki.smartos.org/display/DOC/dtrace.conf ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
