So you believe in "the more heap the better"? This assumption you have just proved wrong. Note, in your test, at 16GB heap, GC took half a second, while at 30GB heap, GC took around a second (and maybe more overhead in stop-the-world pauses). This is a hint about current JVM GC scalability of Java 7.
Jörg On Wed, Feb 18, 2015 at 7:45 PM, Srikanth Valiveti < [email protected]> wrote: > We have 5 shards(give each size) totaling to 76.6GB in a single host > c3.8xlarge <http://aws.amazon.com/ec2/instance-types/> system (60gb ram, > 32 core, 2*320 SSD ) > > We have multiple fields in our record, but single field is ngram analyzed > on which we search results for. This search need to performed on all 5 > shards of the host to get results, as there is no routing in our case. > > We observed huge variations in search TPS with *decrease* in elastic > search heap memory size. Attached bigdesk images for both of the below > cases! > > CASE1) > When ess_heap size = 16gb > Search tps observed is 50 > > CASE2) > When ess_heap_size=30gb > Search tps observed is 18 > > Surprising thing is as we decrease ess_heap_size the search tps got > increased. All the resources(cpu,memory etc) are not fully utilized, OS > heap memory is not changed much, observed lot of zig zag ess_heap > usage(increase and decrease of ess heap usage, may be because of high index > size that need to be brought to RAM) and reads I/Os followed same zig zag > manner in both the cases. > > Please note that we have run this experiment *multiple* times and > observed the same pattern. Can you please guide on what is going wrong? > what dec ess_heap increasing the tps, should we further decrease to achieve > better tps or we doing something wrong? > > -Thanks > Srikanth V. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/f1d1bf51-ab4d-4242-94e5-4b5c3adf466c%40googlegroups.com > <https://groups.google.com/d/msgid/elasticsearch/f1d1bf51-ab4d-4242-94e5-4b5c3adf466c%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEpT8rNM5%2BosX%3D2%2B70RF-A12ujT2bw3ak4LcH5aQhCA%3DQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
