On Sun, Jul 4, 2010 at 10:08 AM, Ian D <rewar...@hotmail.com> wrote: > What I don't understand is why, when I run a single query I get <100 IOPS > and <3MB/sec. The setup can obviously do better, so where is the > bottleneck? I don't see any CPU core on any side being maxed out so it > can't be it...
In what way is CPU contention being monitored? "prstat" without options is nearly useless for a multithreaded app on a multi-CPU (or multi-core/multi-thread) system. mpstat is only useful if threads never migrate between CPU's. "prstat -mL" gives a nice picture of how busy each LWP (thread) is. When viewed with "prstat -mL", A thread that has usr+sys at 100% cannot go any faster, unless you can get the CPU to go faster, as I suggest below. From my understanding (perhaps not 100% correct on the rest of this paragraph): The time spent in TRP may be reclaimed by running the application in a processor set with interrupts disabled on all of its processors. If TFL or DFL are high, optimizing the use of cache may be beneficial. Examples of how you can optimize the use of cache include using the FX scheduler with a priority that gives relatively long time slices, using processor sets to keep other processes off of the same caches (which are often shared by multiple cores), or perhaps disabling CPU's (threads) to ensure that only a single core is using each cache. With current generation Intel CPU's, this can allow the CPU clock rate to increase, thereby allowing more work to get done. > The database is MySQL, it runs on a Linux box that connects to the Nexenta Oh, since the database runs on Linux I guess you need to dig up top's equivalent of "prstat -mL". Unfortunately, I don't think that Linux has microstate accounting and as such you may not have visibility into time spent on traps, text faults, and data faults on a per-process basis. > server through 10GbE using iSCSI. Have you done any TCP tuning? Based on the numbers you cite above, it looks like you are doing about 32 KB I/O's. I think you can perform a test that involves mainly the network if you use netperf with options like: netperf -H $host -t TCP_RR -r 32768 -l 30 That is speculation based on reading http://www.netperf.org/netperf/training/Netperf.html. Someone else (perhaps on networking or performance lists) may have better tests to run. -- Mike Gerdts http://mgerdts.blogspot.com/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
