On 04/10/10 04:21 AM, Craig Ringer wrote: > Phil Stracchino wrote: >> On 04/09/10 02:35, Henrik Johansen wrote: >>> I have spend some time profiling the bacula-sd under production load on >>> Solaris. Besides checking for signs of lock contention and memory / CPU >>> usage issues I also tracked the I/O statistics for each file descriptor >>> opened within the bacula-sd process while running. >>> >>> According to my notes there was a noticeable change in throughput when >>> testing 100 concurrent jobs against 100 separate devices (100 separate >>> volumes) compared against writing the same jobs against 20 concurrent >>> volumes. >>> >>> I/O throughput increased with more concurrency. >> >> That is an extremely interesting, and somewhat counter-intuitive, result. > > It's exactly what I expect to see on any decent hardware. Experience on > the PostgreSQL mailing list suggests that for database workloads at > least, optimal write throughput tends to be achieved at > nthreads=nspindles+ncpus. > > Of course database workloads tend to be a bit more random in their write > patterns and tend to hit the CPU more than backula-sd's write loads are > likely to. For bacula-sd I'd start at nthreads=nspindles and explore > around there to find where my hardware performed best. I've certainly > found that pure sequential write throughput also tends to increase with > number of writing threads/processes - up to a point that varies with > raid level, disk array layout, spindle count, controller/swraid type, etc.
It's just a matter of sending enough sequential I/O threads through the same pipe before it appears random to your storage subsystem. > Some benefit may even be gained from multiple writing threads on > *single* *disk* setups if TCQ/NCQ is enabled and none of the programs > are using fsync() to force write ordering. Probably not for pure > sequential writes, though I'd have to test that. > > ( I almost wonder if it's worth adapting that tool to produce a quick > and dirty optimizer that searches for near-optimal writing thread count > on a given storage device. Hmm. Might be handy if bacula-sd ever gets a > "max concurrent writers" param or the like. ) > > I need to put together a decent benchmark platform before I can do much > hard testing for bacula or with that test tool, as the main decent > arrays I have access to are always busy with backups, production loads, etc. > > -- > Craig Ringer > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > Bacula-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/bacula-devel -- Med venlig hilsen / Best Regards Henrik Johansen [email protected] Tlf. 75 53 35 00 ScanNet Group A/S ScanNet ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel
