Re: Can the export handler be used with the edismax or dismax query handler
Thanks Mikhail and Erick, I don't need ranks or score. I just need the full set of results. Will the export handler work with a fq that uses edismax? (I'm not at work today, but I can try it out tomorrow.) I compared a simple (not edismax) query and the export handler with cursormark with rows = 50K to 200K. The export handler took about 8 ms to export all 1.9 million results and had minimal impact on server CPU and memory. With the cursormark it took about 1 minute 20 seconds, CPU use increased by about 25% and there were many more garbage collections although the time for GC totaled only a few seconds. Tom On Sat, Jul 28, 2018 at 4:25 AM, Mikhail Khludnev wrote: > Tom, > Do you say you don't need rank results or you don't need to export score? > If the former is true, you can just put edismax to fq. > Just a note: using cursor mark with the score may cause some kind of hit > dupes and probably missing some ones. > > On Sat, Jul 28, 2018 at 5:20 AM Erick Erickson > wrote: > > > What about cursorMark? That's designed to handle repeated calls with > > increasing "start" parameters without bogging down. > > > > https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html > > > > Best, > > Erick > > > > On Fri, Jul 27, 2018 at 9:47 AM, Tom Burton-West > > wrote: > > > Thanks Joel, > > > > > > My use case is that I have a complex edismax query (example below) and > > the > > > user wants to download the set of *all* search results (ids and some > > small > > > metadata fields). So they don't need the relevance ranking. > However, I > > > need to somehow get the exact set that the complex edismax query > matched. > > > > > > Should I try to write some code to rewrite the logic of the edismax > > query > > > with a complex boolean query or would it make sense for me to look at > > > possibly modifying the export handler for my use case? > > > > > > Tom > > > > > > "q= _query_:"{!edismax > > > > > qf='ocr^5+allfieldsProper^2+allfields^1+titleProper^50+ > title_topProper^30+title_restProper^15+title^10+title_ > top^5+title_rest^2+series^5+series2^5+author^80+author2^ > 50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+ > rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+' > > > > > pf='title_ab^1+titleProper^1500+title_topProper^1000+title_ > restProper^800+series^100+series2^100+author^1600+ > author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+' > > > mm='100%25' tie='0.9' } European Art History" > > > > > > > > > On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein > > wrote: > > > > > >> The export handler doesn't allow sorting by score at this time. It > only > > >> supports sorting on fields. So the edismax qparser won't cxcurrently > > work > > >> with the export handler. > > >> > > >> Joel Bernstein > > >> http://joelsolr.blogspot.com/ > > >> > > >> On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West > > >> wrote: > > >> > > >> > Hello all, > > >> > > > >> > I am completely new to the export handler. > > >> > > > >> > Can the export handler be used with the edismax or dismax query > > handler? > > >> > > > >> > I tried using local params : > > >> > > > >> > q= _query_:"{!edismax qf='ocr^5+allfields^1+titleProper^50' > > >> > mm='100%25' > > >> > tie='0.9' } art" > > >> > > > >> > which does not seem to be working. > > >> > > > >> > Tom > > >> > > > >> > > > > > -- > Sincerely yours > Mikhail Khludnev >
Re: Can the export handler be used with the edismax or dismax query handler
Tom, Do you say you don't need rank results or you don't need to export score? If the former is true, you can just put edismax to fq. Just a note: using cursor mark with the score may cause some kind of hit dupes and probably missing some ones. On Sat, Jul 28, 2018 at 5:20 AM Erick Erickson wrote: > What about cursorMark? That's designed to handle repeated calls with > increasing "start" parameters without bogging down. > > https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html > > Best, > Erick > > On Fri, Jul 27, 2018 at 9:47 AM, Tom Burton-West > wrote: > > Thanks Joel, > > > > My use case is that I have a complex edismax query (example below) and > the > > user wants to download the set of *all* search results (ids and some > small > > metadata fields). So they don't need the relevance ranking. However, I > > need to somehow get the exact set that the complex edismax query matched. > > > > Should I try to write some code to rewrite the logic of the edismax > query > > with a complex boolean query or would it make sense for me to look at > > possibly modifying the export handler for my use case? > > > > Tom > > > > "q= _query_:"{!edismax > > > qf='ocr^5+allfieldsProper^2+allfields^1+titleProper^50+title_topProper^30+title_restProper^15+title^10+title_top^5+title_rest^2+series^5+series2^5+author^80+author2^50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+' > > > pf='title_ab^1+titleProper^1500+title_topProper^1000+title_restProper^800+series^100+series2^100+author^1600+author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+' > > mm='100%25' tie='0.9' } European Art History" > > > > > > On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein > wrote: > > > >> The export handler doesn't allow sorting by score at this time. It only > >> supports sorting on fields. So the edismax qparser won't cxcurrently > work > >> with the export handler. > >> > >> Joel Bernstein > >> http://joelsolr.blogspot.com/ > >> > >> On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West > >> wrote: > >> > >> > Hello all, > >> > > >> > I am completely new to the export handler. > >> > > >> > Can the export handler be used with the edismax or dismax query > handler? > >> > > >> > I tried using local params : > >> > > >> > q= _query_:"{!edismax qf='ocr^5+allfields^1+titleProper^50' > >> > mm='100%25' > >> > tie='0.9' } art" > >> > > >> > which does not seem to be working. > >> > > >> > Tom > >> > > >> > -- Sincerely yours Mikhail Khludnev
Re: Can the export handler be used with the edismax or dismax query handler
What about cursorMark? That's designed to handle repeated calls with increasing "start" parameters without bogging down. https://lucene.apache.org/solr/guide/6_6/pagination-of-results.html Best, Erick On Fri, Jul 27, 2018 at 9:47 AM, Tom Burton-West wrote: > Thanks Joel, > > My use case is that I have a complex edismax query (example below) and the > user wants to download the set of *all* search results (ids and some small > metadata fields). So they don't need the relevance ranking. However, I > need to somehow get the exact set that the complex edismax query matched. > > Should I try to write some code to rewrite the logic of the edismax query > with a complex boolean query or would it make sense for me to look at > possibly modifying the export handler for my use case? > > Tom > > "q= _query_:"{!edismax > qf='ocr^5+allfieldsProper^2+allfields^1+titleProper^50+title_topProper^30+title_restProper^15+title^10+title_top^5+title_rest^2+series^5+series2^5+author^80+author2^50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+' > pf='title_ab^1+titleProper^1500+title_topProper^1000+title_restProper^800+series^100+series2^100+author^1600+author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+' > mm='100%25' tie='0.9' } European Art History" > > > On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein wrote: > >> The export handler doesn't allow sorting by score at this time. It only >> supports sorting on fields. So the edismax qparser won't cxcurrently work >> with the export handler. >> >> Joel Bernstein >> http://joelsolr.blogspot.com/ >> >> On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West >> wrote: >> >> > Hello all, >> > >> > I am completely new to the export handler. >> > >> > Can the export handler be used with the edismax or dismax query handler? >> > >> > I tried using local params : >> > >> > q= _query_:"{!edismax qf='ocr^5+allfields^1+titleProper^50' >> > mm='100%25' >> > tie='0.9' } art" >> > >> > which does not seem to be working. >> > >> > Tom >> > >>
Re: Can the export handler be used with the edismax or dismax query handler
Thanks Joel, My use case is that I have a complex edismax query (example below) and the user wants to download the set of *all* search results (ids and some small metadata fields). So they don't need the relevance ranking. However, I need to somehow get the exact set that the complex edismax query matched. Should I try to write some code to rewrite the logic of the edismax query with a complex boolean query or would it make sense for me to look at possibly modifying the export handler for my use case? Tom "q= _query_:"{!edismax qf='ocr^5+allfieldsProper^2+allfields^1+titleProper^50+title_topProper^30+title_restProper^15+title^10+title_top^5+title_rest^2+series^5+series2^5+author^80+author2^50+issn^1+isbn^1+oclc^1+sdrnum^1+ctrlnum^1+id^1+rptnum^1+topicProper^2+topic^1+hlb3^1+fullgeographic^1+fullgenre^1+era^1+' pf='title_ab^1+titleProper^1500+title_topProper^1000+title_restProper^800+series^100+series2^100+author^1600+author2^800+topicProper^200+fullgenre^200+hlb3^200+allfieldsProper^100+' mm='100%25' tie='0.9' } European Art History" On Thu, Jul 26, 2018 at 6:02 PM, Joel Bernstein wrote: > The export handler doesn't allow sorting by score at this time. It only > supports sorting on fields. So the edismax qparser won't cxcurrently work > with the export handler. > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West > wrote: > > > Hello all, > > > > I am completely new to the export handler. > > > > Can the export handler be used with the edismax or dismax query handler? > > > > I tried using local params : > > > > q= _query_:"{!edismax qf='ocr^5+allfields^1+titleProper^50' > > mm='100%25' > > tie='0.9' } art" > > > > which does not seem to be working. > > > > Tom > > >
Re: Can the export handler be used with the edismax or dismax query handler
The export handler doesn't allow sorting by score at this time. It only supports sorting on fields. So the edismax qparser won't cxcurrently work with the export handler. Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Jul 26, 2018 at 5:52 PM, Tom Burton-West wrote: > Hello all, > > I am completely new to the export handler. > > Can the export handler be used with the edismax or dismax query handler? > > I tried using local params : > > q= _query_:"{!edismax qf='ocr^5+allfields^1+titleProper^50' > mm='100%25' > tie='0.9' } art" > > which does not seem to be working. > > Tom >
Can the export handler be used with the edismax or dismax query handler
Hello all, I am completely new to the export handler. Can the export handler be used with the edismax or dismax query handler? I tried using local params : q= _query_:"{!edismax qf='ocr^5+allfields^1+titleProper^50' mm='100%25' tie='0.9' } art" which does not seem to be working. Tom