It seems you run into trouble because you changed some of the default settings, worsening your situation.
Increase ping_timout from 9s to 60s as first band aid - you have GCs with 35secs running. You should reduce the bulk thread pool of 100 to 50, this reduces high memory pressure on the 20% memory you allow. Give more heap space to indexing, use 50% instead of 20%. Better help would be to diagnose the nodes if you exceed the capacity for search and index operations. If so, think about adding nodes. More finetuning after adding nodes could include G1 GC with Java 8, which is targeted to minimize GC stalls. This would not solve node capacity problems though. Jörg On Thu, Mar 27, 2014 at 4:46 PM, Binh Ly <[email protected]> wrote: > I would probably not master enable any node that can potentially gc for a > couple seconds. You want your master-eligible nodes to make decisions as > quick as possible. > > About your GC situation, I'd find out what the underlying cause is: > > 1) Do you have bootstrap.mlockall set to true? > > 2) Does it usually triggered while running queries? Or is there a pattern > on when it usually triggers? > > 3) Is there anything else running on these nodes that would overload and > affect normal ES operations? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/cd594a91-00c4-43ae-97d8-bbda35618d8e%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/cd594a91-00c4-43ae-97d8-bbda35618d8e%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFGS59bM%2Bk-%2BcPeMTwhc5DahP5N63ZDRrgUWMZPzZGRcA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
