DocSet getting cached in filterCache for facet request with {!cache=false}
Hello,
It seems Solr is caching when facting even with fq={!cache=false}*:* specified.
This is what I am doing on Solr 4.10.0 on jre 1.7.0_51.
Query 1) No cache in filterCache as expected
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache
confirms this.
Query 2) Query result docset cached in filterCache unexpectedly ?
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*facet=truefacet.field=foobarfacet.method=enum
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache shows
entry of item_*:*: org.apache.solr.search.BitDocSet@66afbbf cached.
Suggestions why or how this may be avoided since I don't want to cache anything
other than facet(ed) terms in the filterCache (for predictable heap usage).
The culprit seems to be line 1431 @
http://svn.apache.org/viewvc/lucene/dev/tags/lucene_solr_4_10_2/solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java?view=markup
Thanks.
-M
Re: DocSet getting cached in filterCache for facet request with {!cache=false}
On 11/11/2014 1:22 AM, Mohsin Beg Beg wrote:
It seems Solr is caching when facting even with fq={!cache=false}*:*
specified. This is what I am doing on Solr 4.10.0 on jre 1.7.0_51.
Query 1) No cache in filterCache as expected
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache
confirms this.
Query 2) Query result docset cached in filterCache unexpectedly ?
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*facet=truefacet.field=foobarfacet.method=enum
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache
shows entry of item_*:*: org.apache.solr.search.BitDocSet@66afbbf cached.
Suggestions why or how this may be avoided since I don't want to cache
anything other than facet(ed) terms in the filterCache (for predictable heap
usage).
I hope this is just for testing, because fq=*:* is completely
unnecessary, and will cause Solr to do extra work that it doesn't need
to do.
Try changing that second query so q and fq are not the same, so you can
see for sure which one is producing the filterCache entry. With the
same query for both, you cannot know which one is populating the
filterCache. If it's coming from the q parameter, then it's probably
working as designed. If it comes from the fq, then we probably actually
do have a problem that needs investigation.
Thanks,
Shawn
Re: DocSet getting cached in filterCache for facet request with {!cache=false}
Well, the difference that you're faceting with method=enum, which uses
the filterCache (I think, it's been a while).
I admit I'm a little surprised that when I tried faceting with the
inStock field in the standard distro I got 3 entries when there are
only two values but I'm willing to let that go ;)
i.e. this produces 3 entries in the filterCache:
http://localhost:8983/solr/techproducts/select?q=*:*rows=0facet=truefacet.field=inStockfacet.method=enum
not an fq clause in sight..
Best,
Erick
On Tue, Nov 11, 2014 at 9:31 AM, Shawn Heisey apa...@elyograg.org wrote:
On 11/11/2014 1:22 AM, Mohsin Beg Beg wrote:
It seems Solr is caching when facting even with fq={!cache=false}*:*
specified. This is what I am doing on Solr 4.10.0 on jre 1.7.0_51.
Query 1) No cache in filterCache as expected
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache
confirms this.
Query 2) Query result docset cached in filterCache unexpectedly ?
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*facet=truefacet.field=foobarfacet.method=enum
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache
shows entry of item_*:*: org.apache.solr.search.BitDocSet@66afbbf cached.
Suggestions why or how this may be avoided since I don't want to cache
anything other than facet(ed) terms in the filterCache (for predictable heap
usage).
I hope this is just for testing, because fq=*:* is completely
unnecessary, and will cause Solr to do extra work that it doesn't need
to do.
Try changing that second query so q and fq are not the same, so you can
see for sure which one is producing the filterCache entry. With the
same query for both, you cannot know which one is populating the
filterCache. If it's coming from the q parameter, then it's probably
working as designed. If it comes from the fq, then we probably actually
do have a problem that needs investigation.
Thanks,
Shawn
Re: DocSet getting cached in filterCache for facet request with {!cache=false}
Shawn, then how to skip filterCache for facet.method=enum ?
Wiki says fq={!cache=false}*:* is ok, no?
https://wiki.apache.org/solr/SolrCaching#filterCache
-Mohsin
- Original Message -
From: erickerick...@gmail.com
To: solr-user@lucene.apache.org
Sent: Tuesday, November 11, 2014 8:40:54 AM GMT -08:00 US/Canada Pacific
Subject: Re: DocSet getting cached in filterCache for facet request with
{!cache=false}
Well, the difference that you're faceting with method=enum, which uses
the filterCache (I think, it's been a while).
I admit I'm a little surprised that when I tried faceting with the
inStock field in the standard distro I got 3 entries when there are
only two values but I'm willing to let that go ;)
i.e. this produces 3 entries in the filterCache:
http://localhost:8983/solr/techproducts/select?q=*:*rows=0facet=truefacet.field=inStockfacet.method=enum
not an fq clause in sight..
Best,
Erick
On Tue, Nov 11, 2014 at 9:31 AM, Shawn Heisey apa...@elyograg.org wrote:
On 11/11/2014 1:22 AM, Mohsin Beg Beg wrote:
It seems Solr is caching when facting even with fq={!cache=false}*:*
specified. This is what I am doing on Solr 4.10.0 on jre 1.7.0_51.
Query 1) No cache in filterCache as expected
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache
confirms this.
Query 2) Query result docset cached in filterCache unexpectedly ?
http://localhost:8983/solr/collection1/select?q=*:*rows=0fq={!cache=false}*:*facet=truefacet.field=foobarfacet.method=enum
http://localhost:8983/solr/#/collection1/plugins/cache?entry=filterCache
shows entry of item_*:*: org.apache.solr.search.BitDocSet@66afbbf cached.
Suggestions why or how this may be avoided since I don't want to cache
anything other than facet(ed) terms in the filterCache (for predictable heap
usage).
I hope this is just for testing, because fq=*:* is completely
unnecessary, and will cause Solr to do extra work that it doesn't need
to do.
Try changing that second query so q and fq are not the same, so you can
see for sure which one is producing the filterCache entry. With the
same query for both, you cannot know which one is populating the
filterCache. If it's coming from the q parameter, then it's probably
working as designed. If it comes from the fq, then we probably actually
do have a problem that needs investigation.
Thanks,
Shawn
Re: DocSet getting cached in filterCache for facet request with {!cache=false}
On Tue, Nov 11, 2014 at 1:25 PM, Mohsin Beg Beg mohsin@oracle.com wrote:
Wiki says fq={!cache=false}*:* is ok, no?
That's for the filtering... not for the faceting.
then how to skip filterCache for facet.method=enum ?
Specify a high minDF (the min docfreq or number of documents that
need to match a term before the filter cache will be used).
facet.enum.cache.minDf=1000
-Yonik
http://heliosearch.org - native code faceting, facet functions,
sub-facets, off-heap data
Re: DocSet getting cached in filterCache for facet request with {!cache=false}
The first thing I'd try is to stop explicitly _telling_ solr to use
the enum method by
omitting the facet.method=enum from your URL ;)...
I'm guessing that the field in question has very few unique values, so
you probably
need to do what Yonik suggests
Erick
On Tue, Nov 11, 2014 at 1:30 PM, Yonik Seeley yo...@heliosearch.com wrote:
On Tue, Nov 11, 2014 at 1:25 PM, Mohsin Beg Beg mohsin@oracle.com wrote:
Wiki says fq={!cache=false}*:* is ok, no?
That's for the filtering... not for the faceting.
then how to skip filterCache for facet.method=enum ?
Specify a high minDF (the min docfreq or number of documents that
need to match a term before the filter cache will be used).
facet.enum.cache.minDf=1000
-Yonik
http://heliosearch.org - native code faceting, facet functions,
sub-facets, off-heap data
