Thanks Erick. Even 1 second commit interval is fine for us. But in that case also filter cache will be flushed after 1 sec. The end user will still feel slowness due to this as the query will take around 1 sec if we use filter query.
-----Original Message----- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: 20 August 2015 00:44 To: solr-user@lucene.apache.org Subject: Re: Performance issue with FILTER QUERY If you're committing that rapidly then you're correct, filter caching may not be a good fit. The entire _point_ of filter caching is to increase performance of subsequent executions of the exact same fq clause. But if you're throwing them away every second there's little/no benefit. You really have two choices here 1> lengthen out the commit interval. Frankly, 1 second commit intervals are rarely necessary despite what your product manager says. Really, check this requirement out. 2> disable caches. Autowarming is potentially useful here, but if your filter queries are taking on the order of a second and you're committing every second then autowarming takes too long to help. Best, Erick On Wed, Aug 19, 2015 at 12:26 AM, Mikhail Khludnev <mkhlud...@griddynamics.com> wrote: > Maulin, > Did you check performance with segmented filters which I advised recently? > > On Wed, Aug 19, 2015 at 10:24 AM, Maulin Rathod <mrat...@asite.com> wrote: > >> As per my understanding caches are flushed every time when add new >> document to collection (we do soft commit at every 1 sec to make >> newly added document available for search). Due to which it is not >> effectively uses cache and hence it slow every time in our case. >> >> -----Original Message----- >> From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] >> Sent: 19 August 2015 12:16 >> To: solr-user@lucene.apache.org >> Subject: Re: Performance issue with FILTER QUERY >> >> On Wed, 2015-08-19 at 05:55 +0000, Maulin Rathod wrote: >> > SLOW WITH FILTER QUERY (takes more than 1 second) >> > ============================================ >> > >> > q=+recipient_id:(4042) AND project_id:(332) AND >> > resource_id:(13332247 >> > 13332245 13332243 13332241 13332239) AND entity_type:(2) AND >> > -action_id:(20 32) ==> This returns 5 records >> > fq=+action_status:(0) AND is_active:(true) ==> This Filter Query >> > returns 9432252 records >> >> The fq is evaluated independently of the q: For the fq a bitset is >> allocated, filled and stored in cache. Then the q is evaluated and >> the two bitsets are merged. >> >> Next time you use the same fq, it should be cached (if you have >> caching >> enabled) and be a lot faster. >> >> >> Also, if you ran your two tests right after each other, the second >> one benefits from disk caching. If you had executed them in reverse >> order, the >> q+fq might have been the fastest one. >> >> - Toke Eskildsen, State and University Library, Denmark >> >> >> > > > -- > Sincerely yours > Mikhail Khludnev > Principal Engineer, > Grid Dynamics > > <http://www.griddynamics.com> > <mkhlud...@griddynamics.com>
