Cassandra and -XX:+UseLargePages
Hello. Can anyone comment on the performance impact (positive or negative) of running Cassandra configured to use large pages under Linux? Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc. Thanks! David
Re: Cassandra and -XX:+UseLargePages
I've seen about a 13% improvement in practice. -Chris On Jan 16, 2011, at 4:01 PM, David Dabbs wrote: Hello. Can anyone comment on the performance impact (positive or negative) of running Cassandra configured to use large pages under Linux? Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc. Thanks! David
Re: Cassandra and -XX:+UseLargePages
Chris, could you provide some more info on you experience? Were you using mmapped files? Using row or key caches? What were the memtabe thresholds? Using mlockall() ? There are a couple of issues listed in the first paragraphs here that at first glance may cause issues http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.html cheers Aaron On 17/01/2011, at 1:45 PM, Chris Goffinet c...@chrisgoffinet.com wrote: I've seen about a 13% improvement in practice. -Chris On Jan 16, 2011, at 4:01 PM, David Dabbs wrote: Hello. Can anyone comment on the performance impact (positive or negative) of running Cassandra configured to use large pages under Linux? Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc. Thanks! David
RE: Cassandra and -XX:+UseLargePages
-Original Message- From: Aaron Morton [mailto:aa...@thelastpickle.com] Sent: Sunday, January 16, 2011 7:07 PM To: user@cassandra.apache.org Subject: Re: Cassandra and -XX:+UseLargePages Chris, could you provide some more info on you experience? Were you using mmapped files? Using row or key caches? What were the memtabe thresholds? Using mlockall() ? There are a couple of issues listed in the first paragraphs here that at first glance may cause issues http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.ht ml cheers Aaron Aaron, are you referring to: However please note sometimes using large page memory can negatively affect system performance. For example, when a large amount of memory is pinned by an application, it may create a shortage of regular memory and cause excessive paging in other applications and slow down the entire system. Also please note for a system that has been up for a long time, excessive fragmentation can make it impossible to reserve enough large page memory. When it happens, either the OS or JVM will revert to using regular pages. As I read this (and various posts about UseLargePages), you may need to reboot the box and start your app quickly so memory is not fragmented and that reserving lots of memory may starve others. Not really issues if you're dedicating a box/VM to Cassandra, which I'm guessing most folks are doing. Or did you see something else that says there be dragons for Cassandra? I didn't inquire before about experiences with the recent transparent hugepages kernel patch (we're not on that fresh a kernel) but anyone's experiences +/- would be of interest. David Dabbs On 17/01/2011, at 1:45 PM, Chris Goffinet c...@chrisgoffinet.com wrote: I've seen about a 13% improvement in practice. -Chris On Jan 16, 2011, at 4:01 PM, David Dabbs wrote: Hello. Can anyone comment on the performance impact (positive or negative) of running Cassandra configured to use large pages under Linux? Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc. Thanks! David
Re: RE: Cassandra and -XX:+UseLargePages
I pretty much had the same thoughts as you. I think setting xms to the same as xmx helps the JVM allocate all the memory up front.See the comments in conf/cassandra-env.sh.I was confident in defeating the imaginary bats, but was a little concerned that they should turn to dragons if I ignored them long enough.AOn 17 Jan, 2011,at 02:37 PM, David Dabbs dmda...@gmail.com wrote: -Original Message- From: Aaron Morton [mailto:aa...@thelastpickle.com] Sent: Sunday, January 16, 2011 7:07 PM To: user@cassandra.apache.org Subject: Re: Cassandra and -XX:+UseLargePages Chris, could you provide some more info on you experience? Were you using mmapped files? Using row or key caches? What were the memtabe thresholds? Using mlockall() ? There are a couple of issues listed in the first paragraphs here that at first glance may cause issues http://www.oracle.com/technetwork/java/javase/tech/largememory-jsp-137182.ht ml cheers Aaron Aaron, are you referring to: However please note sometimes using large page memory can negatively affect system performance. For example, when a large amount of memory is pinned by an application, it may create a shortage of regular memory and cause excessive paging in other applications and slow down the entire system. Also please note for a system that has been up for a long time, excessive fragmentation can make it impossible to reserve enough large page memory. When it happens, either the OS or JVM will revert to using regular pages. As I read this (and various posts about UseLargePages), you may need to reboot the box and start your app quickly so memory is not fragmented and that reserving lots of memory may starve others. Not really issues if you're dedicating a box/VM to Cassandra, which I'm guessing most folks are doing. Or did you see something else that says "there be dragons" for Cassandra? I didn't inquire before about experiences with the recent transparent hugepages kernel patch (we're not on that fresh a kernel) but anyone's experiences +/- would be of interest. David Dabbs On 17/01/2011, at 1:45 PM, Chris Goffinet c...@chrisgoffinet.com wrote: I've seen about a 13% improvement in practice. -Chris On Jan 16, 2011, at 4:01 PM, David Dabbs wrote: Hello. Can anyone comment on the performance impact (positive or negative) of running Cassandra configured to use large pages under Linux? Yes, YMMV applies, but I thought I'd ask before enlisting sysadmin Fu, etc. Thanks! David
