It'd depend on your config I'd guess, in particular how many workers/threads you have and what ES output you are using in LS.
Why are you cleaning an index like this anyway? It seems horribly inefficient. Basically the error is "OutOfMemoryError", which means you've run out of heap for the operation to complete. What are the specs for your node, how much heap does ES have? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 16 July 2014 00:43, Bastien Chong <[email protected]> wrote: > I have a basic setup with a logstash shipper, an indexer and an > elasticsearch cluster. > Elasticsearch listen on the standart 9200/9300 and logstash indexer > 9301/9302. > > When I do a netstat | wc -l for the ES process: 184 found > (sample) > >> tcp 0 0 ::ffff:172.17.7.87:9300 ::ffff:172.17.8.39:59573 >> ESTABLISHED 23224/java >> tcp 0 0 ::ffff:172.17.7.87:9300 ::ffff:172.17.7.87:47609 >> ESTABLISHED 23224/java >> tcp 0 0 ::ffff:172.17.7.87:53493 ::ffff:172.17.7.87:9302 >> ESTABLISHED 23224/java >> tcp 0 0 ::ffff:172.17.7.87:9300 ::ffff:172.17.8.39:59564 >> ESTABLISHED 23224/java >> tcp 0 0 ::ffff:172.17.7.87:9300 ::ffff:172.17.7.87:47657 >> ESTABLISHED 23224/java >> > > Same thing for the logstash indexer : 160 found > (sample) > >> tcp 0 0 ::ffff:172.17.7.87:50132 ::ffff:172.17.8.39:9300 >> ESTABLISHED 1516/java >> tcp 0 0 ::ffff:172.17.7.87:9301 ::ffff:172.17.7.87:60153 >> ESTABLISHED 1516/java >> tcp 0 0 ::ffff:172.17.7.87:9301 ::ffff:172.17.7.87:60145 >> ESTABLISHED 1516/java >> tcp 0 0 ::ffff:172.17.7.87:50129 ::ffff:172.17.8.39:9300 >> ESTABLISHED 1516/java >> tcp 0 0 ::ffff:172.17.7.87:9302 ::ffff:172.17.7.87:53501 >> ESTABLISHED 1516/java >> > > Also, not sure if related, when I try to delete some documents by query ( > curl -XDELETE 'http://localhost:9200/check/_query?pretty=1' -d > '{"query":{"range":{"@timestamp":{"from":"2014-07-10T00:00:00","to":"2014-07-14T05:00:00"}}}}' > ) > > "RemoteTransportException[[Stonewall][inet[/172.17.8.39:9300]][deleteByQuery/shard]]; > nested: OutOfMemoryError[unable to create new native thread]; " > > I have a script that run this kind of query every 30 seconds to clean up > this particular index. > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/1167d4a6-6b87-45a9-835a-eba0ba696825%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/1167d4a6-6b87-45a9-835a-eba0ba696825%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624Z%2BoDy0a7teHvD-TsSYYhHPs1xZRn9gSM4Qf0C7KsHBTg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
