I have similar problems with Java Garbage Collection. In my case, gc throughput is less than generated garbage from In my opinion, best way is to generate less garbage. This is the largest elasticsearch allocation problems:
1. using "_source" field to store document source inside elastic - lucene generates a lot of garbage when allocating heap byte buffers for read/write document sources. Try not to use it, and get data from primary storage like sql database 2. http/tcp buffers - try increase transport settings, like: http.max_chunk_size: 128kb, max_composite_buffer_components: 65536 (this is a netty related settings for better reuse allocated buffers and prevent copying composite buffers to simple heap byte buffers ) 3. field data an filter caches - try set less memory size for it среда, 19 ноября 2014 г., 9:52:19 UTC+3 пользователь tetlika написал: > > hi, > > we have 6 servers and 14 shards in cluster, the index size 26GB, we have 1 > replica so total size is 52GB, and ES v1.4.0, java version "1.7.0_65" > > we use servers with RAM of 14GB (m3.xlarge), and heap is set to 7GB > > around week ago we started facing next issue: > > random cluster servers around once per day/two are hitting the heap size > limit (java.lang.OutOfMemoryError: Java heap space) in log, and cluster is > failing - becomes red or yellow > > we tried adding more servers to cluster - even 8, but than it's a matter > of time when we'll hit the problem, so looks no matter how many servers are > in cluster - it will still hit the limit after some time > > before we started facing the problem we were running smoothly with 3 > servers > also we set indices.fielddata.cache.size: 40% but it didnt helped > > also, there are possible workarounds to decrease heap usage: > > 1) reboot some server - than heap becomes under 70% and for some time > cluster is ok > > or > > 2) decrease number of replicas to 0, and than back to 1 > > but I dont like to use those workarounds > > how it can happen while all index can fit into RAM it can run out of it? > > thanks much for possible help > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8d1a860e-4e19-4895-9fbc-e575da38889f%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
