Evictions are 0 for all cache types. Your server max heap space with 12G is pretty huge. Which is good I think. The CPU on my server is a 8-Core Intel i7 965.
Commit frequency is low, because shards are added and old shards exist for historical reasons. Old shards will be then cleaned after couple of months. I will try to add maximum 15mio per shard and see what will happen here. This thing is, that I will add more shards over time, so that I can handle maybe 500-800mio documents. Maybe more. It depends. On Tue, Jan 17, 2012 at 2:14 PM, Dmitry Kan <dmitry....@gmail.com> wrote: > Hi Daniel, > > My index is 6,5G. I'm sure it can be bigger. facet.limit we ask for is > beyond 100 thousand. It is sub-second speed. I run it with -Xms1024m > -Xmx12000m under tomcat, it currently takes 5,4G of RAM. Amount of docs is > over 6,5 million. > > Do you see any evictions in your caches? What kind of server is it, in > terms of CPU and OS? How often do you commit to the index? > > Dmitry > > On Tue, Jan 17, 2012 at 3:01 PM, Daniel Bruegge < > daniel.brue...@googlemail.com> wrote: > > > Hi Dmitry, > > > > I had everything on one Solr Instance before, but this got to heavy and I > > had the same issue here, that the 1st facet.query was really slow. > > > > When querying the facet: > > - facet.limit = 100 > > > > Cache settings are like this: > > > > <filterCache class="solr.FastLRUCache" > > size="16384" > > initialSize="4096" > > autowarmCount="4096"/> > > > > <queryResultCache class="solr.LRUCache" > > size="512" > > initialSize="512" > > autowarmCount="0"/> > > > > <documentCache class="solr.LRUCache" > > size="512" > > initialSize="512" > > autowarmCount="0"/> > > > > How big was your index? Did it fit into the RAM which you gave the Solr > > instance? > > > > Thanks > > > > > > On Tue, Jan 17, 2012 at 1:56 PM, Dmitry Kan <dmitry....@gmail.com> > wrote: > > > > > I had a similar problem for a similar task. And in my case merging the > > > results from two shards turned out to be a culprit. If you can > logically > > > store your data just in one shard, your faceting should become faster. > > Size > > > wise it should not be a problem for SOLR. > > > > > > Also, you didn't say anything about the facet.limit value, cache > > > parameters, usage of filter queries. Some of these can be > interconnected. > > > > > > Dmitry > > > > > > On Tue, Jan 17, 2012 at 2:49 PM, Daniel Bruegge < > > > daniel.brue...@googlemail.com> wrote: > > > > > > > Hi, > > > > > > > > I have 2 Solr-shards. One is filled with approx. 25mio documents > (local > > > > index 6GB), the other with 10mio documents (2.7GB size). > > > > I am trying to create some kind of 'word cloud' to see the frequency > of > > > > words for a *text_general *field. > > > > For this I am currently using a facet over this field and I am also > > > > restricting the documents by using some other filters in the query. > > > > > > > > The performance is really bad for the first call and then pretty fast > > for > > > > the following calls. > > > > > > > > The maximum Java heap size is 3G for each shard. Both shards are > > running > > > on > > > > the same physical server which has 12G RAM. > > > > > > > > Question: Should I reduce the documents in one shard, so that the > index > > > is > > > > equal or less the Java Heap size for this shard? Or is > > > > there another method to avoid this slow calls? > > > > > > > > Thank you > > > > > > > > Daniel > > > > > > > > > > > > > > > > -- > > > Regards, > > > > > > Dmitry Kan > > > > > > > > > -- > Regards, > > Dmitry Kan >
