Is this consistent with other people's experience? Would some charts -- heap usage, disk usage, etc. -- make this more approachable On Jul 11, 2014 6:36 PM, "Adam Georgiou" <[email protected]> wrote:
> To clarify, the "documents" I'm referring to as being stored in "the index > I'm percolating against" are my .percolator indexed queries, and there are > no other documents stored in said index. > On Jul 11, 2014 6:21 PM, "Adam Georgiou" <[email protected]> wrote: > >> *Going to try and keep this concise.* >> >> *Issue (Potential bug?)* >> >> - My cluster has been running into memory issues; garbage collection >> loops, stopping the world, etc. >> - In a test cluster I ran a few experiments. After a `jmap` i've >> determined that the >> `org.elasticsearch.index.percolator.PercolatorQueriesRegistry` is taking >> up >> nearly 40% of my heap, even though my percolator queries are a fraction of >> the size of the *regular *documents I'm storing. >> - I understand that percolate queries are all always kept in memory >> >> <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html#_how_it_works_under_the_hood>, >> and I'm trying to plan accordingly, but to put things in perspective the >> index I'm percolating on contains *documents that are ~**317M on disk >> and taking up ~3Gb in memory*. I've determined this ratio through >> jmap output and by just watching the heap size before and after opening >> the >> index with the queries. >> - My test cluster consists of a single node (v1.0.1) and the index >> I'm storing percolator queries in has 5 shards and *0 replicas*. >> >> *Question* >> >> A nearly 10-1 ratio of memory usage to disk usage seems wrong to me. Is >> there something specific about the way percolator documents are stored >> under the hood that makes them take up so much memory compared to the way >> their JSON representations are stored on disk? >> >> -Adam >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/NRKENFOwmmE/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/4ee474c6-9aa8-4a50-b140-d30860ff98fa%40googlegroups.com >> <https://groups.google.com/d/msgid/elasticsearch/4ee474c6-9aa8-4a50-b140-d30860ff98fa%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAF4J1auG3dG%2Bm6aZs0TaqSf%2B4pf7PLct_oLOePpOhv%2B2P-hG%2Bg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
