As it happens, I'm currently involved with a project doing some performance analysis for this... but it is currently a WIP. Comments below.
Robert Milkowski wrote: > Hello Adam, > > Tuesday, October 21, 2008, 2:00:46 PM, you wrote: > > ANC> We're using a rather large (3.8TB) ZFS volume for our mailstores on a > ANC> JMS setup. Does anybody have any tips for tuning ZFS for JMS? I'm > ANC> looking for even the most obvious tips, as I am a bit of a novice. > Thanks, > > Well, it's kind of broad topic and it depends on a specific > environment. Then do not tune for the sake of tuning - try to > understand your problem first. Nevertheless you should consider things like > (random order): > > 1. RAID level - you probably will end-up with relatively small random > IOs - generally avoid RAID-Z > Of course it could be that RAID-Z in your environment is perfectly > fine. > There are some write latency-sensitive areas that will begin to cause consternation for large loads. Storage tuning is very important in this space. In our case, we're using a ST6540 array which has a decent write cache and fast back-end. > 2. Depending on your workload and disk subsystem ZFS's slog on SSD > could help to improve performance > My experiments show that this is not the main performance issue for large message volumes. > 3. Disable atime updates on zfs file system > Agree. JMS doesn't use it, so it just means extra work. > 4. Enabling compression like lzjb in theory could help - depends on > how weel you data would compress and how much CPU you have left and if > you are mostly IO bond > We have not experimented with this yet, but know that some of the latency-sensitive writes are files with a small number of bytes, which will not compress to be less than one disk block. [opportunities for cleverness are here :-)] There may be a benefit for the message body, but in my tests we are not concentrating on that at this time. > 5. ZFS recordsize - probably not as in most cases when you read > anything from email you will probably read entire mail anyway. > Nevertheless could be easily checked with dtrace. > This does not seem to be an issue. > 6. IIRC JMS keeps an index/db file per mailbox - so just maybe L2ARC > on large SSD would help assuming it would nicely cache these files - > would need to be simulated/tested > This does not seem to be an issue, but in our testing the message stores have plenty of memory, and hence, ARC size is on the order of tens of GBytes. > 7. Disabling vdev pre-fetching in ZFS could help - see ZFS Evile tuning > guide > My experiments showed no benefit by disabling pre-fetch. However, there are multiple layers of pre-fetching at play when you are using an array, and we haven't done a complete analysis on this yet. It is clear that we are not bandwidth limited, so prefetching may not hurt. > > Except for #3 and maybe #7 first identify what is your problem and > what are you trying to fix. > > Yep. -- richard _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss