Also, For our data nodes we follow best practices with 50% of memory for java heap, while for our master and query nodes we allocate a higher percentage with the thought that they really do not need big disk caching. Could that be our problem?
In addition, the systems actually are not swapping - no swap in use, just the kswapd process runs away at 100% cpu. We are on: java version "1.7.0_17" Java(TM) SE Runtime Environment (build 1.7.0_17-b02) Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode) elastic search 1.3.2. Thanks in an advance for any pointers, hopefully somebody has seen this before and knows the quick fix. - Mike On Oct 9, 2014, at 10:23 AM, Michael deMan (ES) <[email protected]> wrote: > Hi All, > > This is a bit off topic, but we only see this on some of our elastic search > hosts, and it is also the only place where we enable mlockall for java which > is our understanding is a strongly recommended best practice. > > Basically we from time to time see kswapd run away at 100% on a single core. > > It seems to hit our master nodes more frequently, and they also have the > least amount of memory. > masters are: > CentOS 6.4 > 4GB RAM > 4GB swap > ES_HEAP_SIZE=2908m > > > > Does anybody know much about this and how to prevent it? > We have hunted google groups, but have not really found the magic bullet. > > We have considered turning off swap and seeing what happens in the lab but > prefer not to do that unless it is well known as the correct solution. > > Thanks, > - Mike > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/57F0E223-ACBE-4819-A37C-4D935E2D71AF%40deman.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/F3633E59-60F7-49E2-965E-FFA495A60EE7%40deman.com. For more options, visit https://groups.google.com/d/optout.
