Thanks for the explanation, that really helps. Does that mean that on a virtual host with 64GB memory it might make sense to make two virtual servers each running a node? I had expected that multiple nodes on a single host would not help, but I guess if the VM is the limitation it might?
I have a read-heavy workload, with good use of facets/aggregations and also some really complex queries (>1000 terms), but most of them limited to subsets of <10k or 100k documents (out of 50M). Any recommendations would be much appreciated! -- Wouter On Tue, Apr 15, 2014 at 4:40 PM, [email protected] < [email protected]> wrote: > The advisory of "a lot of heap" means, give as much heap as the JVM is > able to process efficiently. There is an upper limit due to JVM engineering > state of today. You will not find JVMs that can efficiently manage heaps > >32G (except rare expensive commercial JVM products). By efficient I mean > GC stalls under a second. There is heavy engineering going on, known as the > Shenandoah project, to tackle heaps over 100G by millisecond GC: > http://openjdk.java.net/jeps/189 > > The mere index size is not related to heap size choice. You need large > heap if you want filter caching and aggregations/facets cached. > > Example: I have 350G on index files. On my 3x64G RAM nodes I have assigned > 3x16G heap and I do not cache filters, due to the nature of my queries. The > other ~48G I left to OS, for file system buffers (direct I/O is the key to > fast systems). If I assigned 32G to heap, GC would be inacceptable high, > and system would go sluggish after some days, as you had described. It is > not a matter of heap size, but of balancing things carefully out between > JVM management abilities and operating system I/O power. The challenge is > that many ES workload patterns require different balancings. > > Jörg > > > > On Tue, Apr 15, 2014 at 3:42 PM, Wouter van Atteveldt < > [email protected]> wrote: > >> >> >> >> On Tue, Apr 15, 2014 at 2:00 PM, [email protected] < >> [email protected]> wrote: >> >>> This is not Elasticsearch related. If you use a 40g heap of such extreme >>> size, you must expect that garbage collection must run for minutes, on >>> every JVM I know. >>> >>> >> Right, but it is actually advised to give elastic a lot of heap, right? >> The whole index is around 140G, so I would have thought that all frequently >> used parts should get loaded in memory, but it still starts running slow >> after a while. >> >> Any ideas? >> >> -- Wouter >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CACXi6Xe_Q3xy8xZNo5QDqPpsJot8AknCYXfo5N9Vw2AyOfbVZA%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CACXi6Xe_Q3xy8xZNo5QDqPpsJot8AknCYXfo5N9Vw2AyOfbVZA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/yfQv5sDuF40/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERmpqOqp%2Bzq3jMaCZ7e2OrXuboVP9W6BFU%3DDS51RL%3DPw%40mail.gmail.com<https://groups.google.com/d/msgid/elasticsearch/CAKdsXoERmpqOqp%2Bzq3jMaCZ7e2OrXuboVP9W6BFU%3DDS51RL%3DPw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACXi6XcTY4P4AG7UarPfrBjx24%3DhnbcmouEGGfHz1WLHXT5b3w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
