On 8/13/2011 10:57 AM, Paul J Stevens wrote: > I'll take your word for it.
== Dovecot 2.0.13 == MDBOX (no SiS): real 6m14.485s user 2m26.425s sys 0m20.389s 578548 /mail/home == mdbox w/attachment SiS (sis posix mode, SHA1) == real 6m34.305s user 2m28.025s sys 0m16.030s 363724 /mail/attachments 157316 /mail/home/usertest 521040 total MailDir (ext4 + dir_index, no SiS attachment scheme): real 6m46.217s user 2m26.521s sys 0m16.109s 612992 /mail/home > How many concurrent IMAP connections are you using? 1, single-threaded. I start testing very "simple", and work up to multi-threaded/multi-user slams once I establish a functional baseline. I "preload" all of the imported mailboxes into RAM via cat * > /dev/null to reduce the "read cost" to near zero. > A load below 1 indeed sounds suspicious. I should have obtained mysql process status, but I only saw one mysqld worker process during the operation. Should I configure innodb differently? Reindl Harald wrote: > if it is a option for you to upgrade mysql to 5.5 > i would strongly recommend this, our dbmail.machines > became a hughe perofrmance boost against 5.1.x I'll try this on the bare metal system if I don't see much improvement, thanks Reindl. I'll give your my.cnf settings a try as well, much appreciated. > That number is about retrieval. And then only on POP3. And a very old > number (dbmail-1.0?). > > I'd be curious about more up-to-date numbers. I'd be more than happy to provide some once I've gotten things working properly. Do you have a performance testing framework or set of scripts? I have the ImapTest suite, but that's more for validating compliance with the imap4r1 spec than performance. I know about MSTONE http://mstone.sourceforge.net/ but a tool that targets areas that developers suspect/worry are weak would seen sensible. > Also, I'm not at all familiar with IO performance on VMWare, but on Xen > IO really lags behind bare-iron databases. And thats no joke. I have the pvscsi, vmxnet3, and other "driver helpers", and support for a paravirtualised kernel/timers, etc running and actually see very good I/O performance. I know hdparm -t -T isn't a real disk I/O performance tool, but: /dev/sda: <-- /tmp lives here Timing cached reads: 7902 MB in 1.97 seconds = 4005.51 MB/sec Timing buffered disk reads: 182 MB in 3.03 seconds = 60.02 MB/sec /dev/sdb: <-- this is the MySQL spindle Timing cached reads: 7702 MB in 1.97 seconds = 3901.96 MB/sec Timing buffered disk reads: 236 MB in 3.00 seconds = 78.63 MB/sec These are 3-4 year old Samsung 500GB drives, and this performance is about right when compared to what the host experiences natively. They're all formatted with ext4 (implying has_journal, extent, huge_file, flex_bg, uninit_bg, dir_nlink, extra_isize, sparse_super, filetype, resize_inode, dir_index, ext_attr) > I need to keep thread-concurrency at 1 (read: > one) to keep throughput at acceptable levels with IMAP concurrencies > around 50-100 connections, POP3 around 10-20, with an added 5 LMTP > connections for insertions. Yuck, are you serious? Maybe splitting the table across multiple spindles would help. I can try a "bare iron" test machine today. What is your "blessed" or preferred Linux Distro where you do your development and regression testing? I have no distro religious issues, though being a control freak, I will usually run "home" to the minimalistic comfort of Slackware. > That said: dbmail will never be able to achieve the kind of numbers you > mention for dovecot or panda. The single-instance storage used for both > mime-blobs and header values, and fully indexed headers prevent it. > DBMail insertion is not about creating simple files in directories. I appreciate this difference. I realise that with Dovecot's scheme, they defer indexing until a user does an IMAP SEARCH TEXT/BODY, making the user wait a bit for that initial search before updating the index. I suppose to be fair, I should fire off a SEARCH for a known-to-be-non-existent string before closing the mailbox. It'd be trivial to update the script to do this since I am worried I'm not comparing "apples to apples" here. I appreciate all of the replies I've received so far, especially from one of the developers. =R= _______________________________________________ DBmail mailing list [email protected] http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
