Hi Jaguar, This is indeed the case. The OS has it's own file system caches that will consume this additional memory. ES can be IO intensive when indexing. You should not let the heap exceed 32gb. Garbage collection will be very slow and cause pauses to your application. Which version of Java are you using? 1.8 seems to be stable now and I have some success with using G1GC instead of the default GC mechanism. This wasn't really stable in the old 1.7 releases of Java, so be a little careful with this.
I have gone through a similar process to what you're going through now. We had 10 nodes each with 128gb of memory. I had huge heaps, things crashed and timed out. I then reduced the heap to 32gb and put two ES nodes on each server. Still had some issues with timeouts. I then reduced to one node per server. It is now stable. There were a few other changes that I made, but these made some big differences. One thing to note is that every node in the cluster means that every other node needs to be aware of all it's actions. when you start having 20 nodes in a cluster you get a very talkitive network and the nodes are busy just trying to talk to each other. In the configuration that I have just mentioned to you we bulk index 10k+ docs every second and store over 20TB of searchable data. It's a log processing system so it's designed for high index volumes. I don't think it's 100% tuned yet, but it's getting very close. Let me know if you have any further questions. On Tuesday, November 18, 2014 7:26:07 PM UTC+1, Jaguar wrote: > > Hi Robert, > For the comment of 32g heap, I do notice the same recommendation in > official guide. But it don’t seems to make senses to give only 25% ram to > ES as there is total 128g. One alternative could be 2 ES processes per > node, but it would be a bit difficult to manage. Does anyone have similar > setup and would like to share your indexing performance numbers? > > Thanks! > Jaguar > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/47453cc3-7862-41dd-97c5-ec299d465a2d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
