Muhzin, The PostFilter does indeed have some slow operations in it. You're getting the BinaryDocValues for the id field each collect and reading them into the "page" BytesRef, then not using it.
Then you pull the id field from stored field in the document, which is slow. So unless I'm missing something you already have the value you need read into the "page" BytesRef. So you don't need to pull from the stored field. The Set<String> you're using to validate will be faster if you store all the values as BytesRefs, that way you won't need to convert the BytesRef to a string on each collect() call. The BytesRef in the collect method can be reused so you can have one instance for the class and avoid creating a BytesRef for each collect. You also need to implement hashCode() and equals() on the RedisPostFilter class so caching will work properly. On Tue, Oct 22, 2013 at 5:30 AM, Muhzin <rmuh...@gmail.com> wrote: > Hi, > Thanks for the insight Uwe. I could get it up and running thanks to you. > But the query seems to be expensive in terms of time. > Please have a look at the gist > https://gist.github.com/rmuhzin/6991544 > > Is there anyway that I can fine tune the filter? Another problem is that > it seems cache the filtered result so that the result set is same after I > execute the query. Please point me in the right direction. > > > On Thu, Oct 17, 2013 at 12:00 PM, Uwe Schindler <u...@thetaphi.de> wrote: > >> Hi,**** >> >> ** ** >> >> I think the main issue is this sentence in Erik’s blog post:**** >> >> **· **This implementation can only be used as a filter query (fq) >> parameter, not a q parameter.**** >> >> ** ** >> >> So you can only use the postfilter in a fq= parameter, not – as you do – >> in q=. If you use it in fq it seems that Solr does not execute the query >> part. Your filter might be executed after you change your query URL, but >> its still has the problem that it incorrectly assumes the doc id given in >> the collect method is global and can be used with indexSearcher. The >> collector works per index segment, so you have to lookup the doc id using >> the AtomicReaderContext given in setNextReader() [e.g. >> context.reader().document(docid).get(…)]. See also Eriks code, where he >> uses the FieldCache, also per segment.**** >> >> ** ** >> >> Uwe**** >> >> ** ** >> >> -----**** >> >> Uwe Schindler**** >> >> H.-H.-Meier-Allee 63, D-28213 Bremen**** >> >> http://www.thetaphi.de**** >> >> eMail: u...@thetaphi.de**** >> >> ** ** >> >> *From:* Muhzin [mailto:rmuh...@gmail.com] >> *Sent:* Thursday, October 17, 2013 7:21 AM >> *To:* dev@lucene.apache.org >> *Subject:* Re: solr Post Filter**** >> >> ** ** >> >> Hi Uwe, **** >> >> Thank you very much for the insight . I was following the article >> by Erik Hatcher**** >> >> http://searchhub.org/2012/02/22/custom-security-filtering-in-solr/ **** >> >> ** ** >> >> and this presentation **** >> >> >> http://www.lucenerevolution.org/sites/default/files/Using%20LuceneSolr%20to%20Build%20Advertising%20Systems.pdf >> **** >> >> ** ** >> >> ** ** >> >> Is there some other way implement the post filter. Can you please point >> me in a good direction.**** >> >> ** ** >> >> On Tue, Oct 15, 2013 at 7:58 PM, Uwe Schindler <u...@thetaphi.de> wrote:** >> ** >> >> Hi,**** >> >> **** >> >> ExtendedQueryBase extends the abstract Lucene query class, but you don’t >> implement any query-specific method. So Solr does not know which query you >> want to execute? The postfilter alone is no valid query.**** >> >> **** >> >> Also there may be some bugs in your code:**** >> >> - Where does indexSearcher come from?**** >> >> - The collect method uses the atomic docid to query the >> top-level indexsearcher. It should either use the atomicReaderContext and >> request the document from there or it should use docBase to transform the >> docid to be global**** >> >> But this code is not yet executed, I just wanted to mention this.**** >> >> **** >> >> Uwe**** >> >> **** >> >> -----**** >> >> Uwe Schindler**** >> >> H.-H.-Meier-Allee 63, D-28213 Bremen**** >> >> http://www.thetaphi.de**** >> >> eMail: u...@thetaphi.de**** >> >> **** >> >> *From:* Muhzin [mailto:rmuh...@gmail.com] >> *Sent:* Tuesday, October 15, 2013 4:11 PM >> *To:* dev@lucene.apache.org >> *Subject:* solr Post Filter**** >> >> **** >> >> Hi, I am developing a post filter implementation in solr 4.2.1 . the >> following is a gist to my implementation.**** >> >> **** >> >> https://gist.github.com/rmuhzin/6991544**** >> >> **** >> >> I deployed my plugin in the library folder and updated the solrconfig.xml >> **** >> >> **** >> >> <queryParser name="lastLoginParser" >> class="com.m4marry.PostQParserPlugin" />**** >> >> **** >> >> But when I apply the plugin using **** >> >> **** >> >> >> http:/localhost/solr/profiles/select?q={!lastLoginParser}*%3A*&wt=xml&indent=true >> **** >> >> **** >> >> I get the following error **** >> >> **** >> >> java.lang.UnsupportedOperationException: Query {!cache=false cost=100} >> does not implement createWeight**** >> >> at org.apache.lucene.search.Query.createWeight(Query.java:80)**** >> >> at **** >> >> **** >> >> Full stack trace at : https://gist.github.com/rmuhzin/6992029**** >> >> **** >> >> **** >> >> Could someone please shed some light into it ?**** >> >> ** ** >> > > > > -- > BR > Muhsin > -- Joel Bernstein Professional Services LucidWorks
