BTW, I think I noticed something as well (before and after the below optimization).
sh-2.05# qmail-qstat messages in queue: 17957 messages in queue but not yet preprocessed: 1229 With everything queue running off one hard disk (disk 1), I never noticed such a few emails not even being able to be preprocessed. It seems the system's ability to preprocess the messages has declined since putting the queue on disk 2. I don't see any reason why... but anyway, facts are facts :-/ Sincerely, Jason ----- Original Message ----- From: "Jason Lim" <[EMAIL PROTECTED]> To: <[email protected]> Cc: "Russell Coker" <[EMAIL PROTECTED]> Sent: Friday, June 08, 2001 10:14 PM Subject: Re: Finding the Bottleneck > I agree with you that splitting the mail queue to another server wouldn't > help, especially since you've seen the top results, and know that it isn't > very heavily loaded with other jobs in the first place. So I think you are > very correct in saying that the hard disk is the limit here. > > Today I played around with hdparm to see if I could tweak some additional > performance out of the existing drives, and it helped by about 10% (not a > huge jump, but anything helps!). > > Specifically, I set /sbin/hdparm -a4 -c3 -d1 -m16 -u1 /dev/hdc: > > -a Get/set sector count for filesystem read-ahead. > This is used to improve performance in sequential > reads of large files, by prefetching additional > blocks in anticipation of them being needed by the > running task. In the current kernel version > (2.0.10) this has a default setting of 8 sectors > (4KB). This value seems good for most purposes, > but in a system where most file accesses are random > seeks, a smaller setting might provide better per� > formance. Also, many IDE drives also have a sepa� > rate built-in read-ahead function, which alleviates > the need for a filesystem read-ahead in many situa� > tions. > (Since most the emails are small and randomly placed around, I thought > maybe 2KB read-ahead might make more sense. Tell me if I'm wrong... > because the performance jump may not be due to this setting) > > -c Query/enable (E)IDE 32-bit I/O support. A numeric > parameter can be used to enable/disable 32-bit I/O > support: Currently supported values include 0 to > disable 32-bit I/O support, 1 to enable 32-bit data > transfers, and 3 to enable 32-bit data transfers > with a special sync sequence required by many > chipsets. The value 3 works with nearly all 32-bit > IDE chipsets, but incurs slightly more overhead. > Note that "32-bit" refers to data transfers across > a PCI or VLB bus to the interface card only; all > (E)IDE drives still have only a 16-bit connection > over the ribbon cable from the interface card. > > (Couldn't hurt to have it going 32 bit rather than 16 bit) > > -d Disable/enable the "using_dma" flag for this drive. > This option only works with a few combinations of > drives and interfaces which support DMA and which > are known to the IDE driver (and with all supported > XT interfaces). In particular, the Intel Triton > chipset is supported for bus-mastered DMA operation > with many drives (experimental). It is also a good > idea to use the -X34 option in combination with -d1 > to ensure that the drive itself is programmed for > multiword DMA mode2. Using DMA does not necessar� > ily provide any improvement in throughput or system > performance, but many folks swear by it. Your > mileage may vary. > (this is a dma100 7200 drive so setting this couldn't hurt either. Didn't > see much performance increase with this though) > > -m Get/set sector count for multiple sector I/O on the > drive. A setting of 0 disables this feature. Mul� > tiple sector mode (aka IDE Block Mode), is a fea� > ture of most modern IDE hard drives, permitting the > transfer of multiple sectors per I/O interrupt, > rather than the usual one sector per interrupt. > When this feature is enabled, it typically reduces > operating system overhead for disk I/O by 30-50%. > On many systems, it also provides increased data > throughput of anywhere from 5% to 50%. Some > drives, however (most notably the WD Caviar > > series), seem to run slower with multiple mode > enabled. Your mileage may vary. Most drives sup� > port the minimum settings of 2, 4, 8, or 16 (sec� > tors). Larger settings may also be possible, > depending on the drive. A setting of 16 or 32 > seems optimal on many systems. Western Digital > recommends lower settings of 4 to 8 on many of > their drives, due tiny (32kB) drive buffers and > non-optimized buffering algorithms. The -i flag > can be used to find the maximum setting supported > by an installed drive (look for MaxMultSect in the > output). Some drives claim to support multiple > mode, but lose data at some settings. Under rare > circumstances, such failures can result in massive > filesystem corruption. > (I set it to 16... do you think 32 would make more sense?) > > -u Get/set interrupt-unmask flag for the drive. A > setting of 1 permits the driver to unmask other > interrupts during processing of a disk interrupt, > which greatly improves Linux's responsiveness and > eliminates "serial port overrun" errors. Use this > feature with caution: some drive/controller combi� > nations do not tolerate the increased I/O latencies > possible when this feature is enabled, resulting in > massive filesystem corruption. In particular, > CMD-640B and RZ1000 (E)IDE interfaces can be unre� > liable (due to a hardware flaw) when this option is > used with kernel versions earlier than 2.0.13. > Disabling the IDE prefetch feature of these inter� > faces (usually a BIOS/CMOS setting) provides a safe > fix for the problem for use with earlier kernels. > (this seem to have the largest performance boost) > > Anyway... there it is. Maybe someone else could use these results to get a > free 10% increase as well. I stupidly set write_cache on as well, which > ended up trashing a bunch of stuff. Thank goodness at that time the server > was not being used, and I immediately rebuilt the mail queue. > > Does anyone have any better configs than above, or some utility that could > further boost performance? > > Sincerely, > Jason > > ----- Original Message ----- > From: "Russell Coker" <[EMAIL PROTECTED]> > To: "Jason Lim" <[EMAIL PROTECTED]>; "Brian May" > <[EMAIL PROTECTED]> > Cc: <[email protected]> > Sent: Friday, June 08, 2001 7:17 PM > Subject: Re: Finding the Bottleneck > > > On Friday 08 June 2001 12:25, Jason Lim wrote: > > The network is connected via 100Mb to a switch, so server to server > > connections would be at that limit. Even 10Mb wouldn't be a problem as > > I don't think that much data would be crossing the cable.. would it? > > 10Mb shouldn't be a problem for DNS. Of course there's the issue of what > else is on the same cable. > > There will of course be a few extra milli-seconds latency, but you are > correct that it shouldn't make a difference. > > > As for the "single machine" issue, that would depend. If you're talking > > about either getting a couple of SCSI disks, putting them on a hardware > > raid, or getting an additional small server just for the queue, then I > > think the cost would end up approximately the same. This client doesn't > > have the cash for a huge upgrade, but something moderate would be okay. > > However getting an extra server will not make things faster, in fact it > will probably make things slower (maybe a lot slower). Faster hard > drives is what you need! > > > BTW, just to clarify for people who are not familar with qmail, qmail > > stores outgoing email in a special queue, not in Maildir. Only incoming > > mail is stored in Maildir. The Maildirs are actually stored on Disk 1 > > (along with the operating system and everything else except the queue). > > I know Maildir can be put in a NFS disk... BUT i've never heard of > > anyone putting the mail queue on NFS, so I'm not sure if the file > > locking issues you mention would pertain to that as well. > > For the queue, Qmail creates file names that match Inode numbers (NFS > doesn't have Inodes). Qmail also relies on certain link operations being > atomic and reliable, while on NFS they aren't guaranteed to be atomic, > and packet loss can cause big reliability problems. > > Consider "ln file file2", when an NFS packet is sent to the server the > server will create the link and return success, if the return packet is > lost due to packet corruption then the client will re-send the request. > The server will notice that file2 exists and return an error message. > The result is that the operation succeeded but the client thinks it > failed! > > There are many other issues with NFS for this type of thing. NFS is only > good for data that has simple access patterns (read-only files and simple > operations like mounting a home directory and editing a file with "vi"), > and for applications which have carefully been written to work with NFS > (Maildir programs). > > -- > http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark > http://www.coker.com.au/postal/ Postal SMTP/POP benchmark > http://www.coker.com.au/projects.html Projects I am working on > http://www.coker.com.au/~russell/ My home page > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact > [EMAIL PROTECTED] > > > > > -- > To UNSUBSCRIBE, email to [EMAIL PROTECTED] > with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] > >
