Sounds like your cluster is reaching capacity. I'd try adding more data nodes, if you cannot upgrade java and ES.
Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 8 April 2014 15:18, Ranjith <[email protected]> wrote: > I was searching for some solution, i came across this one. > > https://github.com/netty/netty/issues/1873 > > Whether my issue is related to this one ? > > > On Mon, Apr 7, 2014 at 12:44 PM, Ranjith Venkatesan < > [email protected]> wrote: > >> Hi, >> >> We are using Elasticsearch-0.90.5 in our search application. We have 3 >> master nodes and 3 data nodes each with heap size of 4gb running in JDK >> 1.6. We have our Node Client running in our APP_SERVER. Elasticsearch >> master gets HIGH_CPU frequently at least for every 20 days and it is not >> coming down until that node is restarted. When i see thread dump, nothing >> peculiar except GC. Number of GC threads is almost 20. *GC MarkSweep >> Frequency* and *GC ParNew Frequency* is very very less when compared to >> other nodes. >> >> When i restart the master, system load becomes normal. But our APP_SERVER >> gets HIGH_CPU. With our monitoring tool, we could detect some packets were >> dropped due to network buffer overflow. When we restart APP_SERVER, all >> things become normal. >> >> For every 20 days, we have to restart our master and then our >> APP_SERVERS. This has become our routine. I have attached Elasticsearch >> Masters thread dump here <https://gist.github.com/anonymous/10015420> >> >> We couldn't analyze why we are getting HIGH_CPU repeatedly. Any help in >> resolving this issue will be appreciated. >> >> Thanks in advance >> >> Ranjith Venkatesan >> >> >> PS : Elasticsearch version and JDK version cant be updated immediately in >> our case. >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/aglssMPhYQk/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/aaa14b57-3253-42dc-9a67-3378bb892a3f%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/aaa14b57-3253-42dc-9a67-3378bb892a3f%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAM2Z_7DKMSLVAaeuTVy0OhZ6Z%3DYQ%2Bs1LMdnV2_yJty2aH51Ydw%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAM2Z_7DKMSLVAaeuTVy0OhZ6Z%3DYQ%2Bs1LMdnV2_yJty2aH51Ydw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZEbNOqFLw0BwJfX_%3Dy_17HLQ_VLfcShw3GLQZnc4f%3DGw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
