Hello ES group,
I have had recurring heap problems ("java.lang.OutOfMemoryError: Java heap
space”) on my 2-nodes ES cluster (16GB RAM/node, 8GB allocated to ES) the
last month and I really don’t know how to tackle them.
It started at a time where I was doing aggregations on a “milliseconds
since EPOCH” field, and I was given to understand that it was probably the
cause of my problems since it created a very large number of buckets before
aggregating them. So I stopped doing aggregations on this field (I did not
delete it though).
Recently I was told that my index had too few shards respective to its size
(2 primary shards, 1 replica each, 100-150 Mdocs). So I decided to try
reindexing into a new index with more shards (I am using es-reindex.rb,
which itself uses the bulk API). But now I am having OutOfMemoryError happen
during reindexing. Needless to say, once an OutOfMemoryError happens, my
cluster seems to never recover until I reboot each node.
It should be noted that I use ES almost exclusively with search_type=count,
since I am only trying to do analytics on website data.
I am not sure how to proceed from this point, I don’t know the right tool
to pinpoint my memory problems and there doesn’t seem to be a way to ask ES
for heap usage by index/query/task type
I’d be very grateful for any advice you can offer.
Thanks in advance,
Vincent Bernardi
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4687cf5b-34c8-4f5d-88be-f134367a888b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
