Check out ElasticHQ instead of head, it gives you more insight into cluster and node performance.
How much is your heap? What ES and java versions are you on? It's probable you are running into the limits of your node, but a bit more info would clarify this. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 23 June 2014 18:20, Maxim Terletsky <[email protected]> wrote: > Hi guys, > I have one node with 4 different indices on it. Each index has 2 shard and > one replica (which is just defined without actual replicating since only > one node is currently applicable). Sizes of indexing is: > ads1 > size: 18.5G (18.5G) > docs: 4,414,787 (4,414,787) > > ads2 > size: 19.1G (19.1G) > docs: 4,259,828 (4,259,828) > > ads3 > size: 18.9G (18.9G) > docs: 4,221,637 (4,221,637) > > ads4 > size: 20.3G (20.3G) > docs: 4,263,329 (4,263,329). > > From some point the ES stopped responding to rest request, even as simple > as count > http://somemachine:9200/ads1/pagedocuments/_count > The Head plugin, however, seems to be working - it show the info about > indices, etc.. > Latest logs from ES console are: > [2014-06-22 17:13:58,922][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] stop throttling indexing: numMergesInFlight=4, > maxNumMerges=5 > [2014-06-22 17:13:58,922][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] now throttling indexing: numMergesInFlight=6, > maxNumMerges=5 > [2014-06-22 19:16:13,384][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] stop throttling indexing: numMergesInFlight=4, > maxNumMerges=5 > [2014-06-22 19:16:13,535][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] now throttling indexing: numMergesInFlight=6, > maxNumMerges=5 > [2014-06-22 20:54:46,650][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] stop throttling indexing: numMergesInFlight=4, > maxNumMerges=5 > [2014-06-22 20:54:46,655][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] now throttling indexing: numMergesInFlight=6, > maxNumMerges=5 > [2014-06-22 23:19:08,747][INFO ][monitor.jvm ] > [adds_node_one] [gc][young][36246][12421] duration [886ms], collections > [1]/[1.6s], total [886ms]/[5.7m], memory [3.2gb]->[2.9gb]/[4.9gb], all_po > ols {[young] [449.6mb]->[107.5mb]/[532.5mb]}{[survivor] > [40.9mb]->[63.2mb]/[66.5mb]}{[old] [2.8gb]->[2.8gb]/[4.3gb]} > [2014-06-22 23:19:50,901][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] stop throttling indexing: numMergesInFlight=4, > maxNumMerges=5 > [2014-06-22 23:19:50,901][INFO ][index.engine.internal ] > [adds_node_one] [ads2][0] now throttling indexing: numMergesInFlight=6, > maxNumMerges=5 > [2014-06-22 23:24:24,878][WARN ][monitor.jvm ] > [adds_node_one] [gc][young][36494][12533] duration [1.5s], collections > [1]/[3.1s], total [1.5s]/[5.8m], memory [3.4gb]->[3.3gb]/[4.9gb], all_pool > s {[young] [234.6mb]->[62.1mb]/[532.5mb]}{[survivor] > [65.4mb]->[50.5mb]/[66.5mb]}{[old] [3.1gb]->[3.2gb]/[4.3gb]} > [2014-06-22 23:47:07,611][WARN ][monitor.jvm ] > [adds_node_one] [gc][young][37569][13012] duration [1.7s], collections > [1]/[2.6s], total [1.7s]/[6.1m], memory [3.6gb]->[3.3gb]/[4.9gb], all_pool > s {[young] [341.7mb]->[13.8mb]/[532.5mb]}{[survivor] > [66.5mb]->[66.5mb]/[66.5mb]}{[old] [3.2gb]->[3.2gb]/[4.3gb]} > [2014-06-22 23:57:15,573][WARN ][monitor.jvm ] > [adds_node_one] [gc][young][38039][13231] duration [1s], collections > [1]/[2.6s], total [1s]/[6.3m], memory [3.2gb]->[3.1gb]/[4.9gb], all_pools > {[ > young] [301.7mb]->[215.7mb]/[532.5mb]}{[survivor] > [48.3mb]->[37.9mb]/[66.5mb]}{[old] [2.8gb]->[2.8gb]/[4.3gb]} > [2014-06-22 23:57:25,552][WARN ][monitor.jvm ] > [adds_node_one] [gc][young][38046][13235] duration [1s], collections > [1]/[2.5s], total [1s]/[6.3m], memory [3.2gb]->[3gb]/[4.9gb], all_pools > {[yo > ung] [293.9mb]->[78.1mb]/[532.5mb]}{[survivor] > [62.9mb]->[49.4mb]/[66.5mb]}{[old] [2.9gb]->[2.9gb]/[4.3gb]} > > Does anybody have any ideas? Perhaps we came close to some limit of ES on > this machine? > > Any advice will be appreciated.. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/1b78fe25-23e0-4b2a-beb3-54691cd3f468%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/1b78fe25-23e0-4b2a-beb3-54691cd3f468%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZXiFRDma0MYkrAnb72x3sEJaVRtBkGFfZLaCujFvt7%2Bw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
