Re: Boosting results

2008-11-11 Thread Erik Hatcher
On Nov 11, 2008, at 8:32 AM, Stefan Trcek wrote: On Tuesday 11 November 2008 02:18:39 Erik Hatcher wrote: The integration won't be too painful... the main thing is that Solr requires* some configuration files, literally on the filesystem, in order to fire up and be happy. And you'll need to

Re: Boosting results

2008-11-11 Thread Stefan Trcek
On Monday 10 November 2008 14:58:15 Mark Miller wrote: But: it's slow to load a field for the first time.  LUCENE-1231 (column-stride fields) aims to greatly speed up the load time. Test it out though. In some recent testing I was doing it was *way* faster than I thought it would be based

Re: Boosting results

2008-11-11 Thread Stefan Trcek
On Tuesday 11 November 2008 02:18:39 Erik Hatcher wrote: The integration won't be too painful... the main thing is that Solr requires* some configuration files, literally on the filesystem, in order to fire up and be happy. And you'll need to craft Solr's schema.xml to jive with how you

Re: Boosting results

2008-11-10 Thread Mark Miller
Michael McCandless wrote: But: it's slow to load a field for the first time. LUCENE-1231 (column-stride fields) aims to greatly speed up the load time. Test it out though. In some recent testing I was doing it was *way* faster than I thought it would be based on what I had been reading. Of

Re: Boosting results

2008-11-10 Thread Michael McCandless
Well .. the FieldCache API is documented here (for 2.4.0): http://lucene.apache.org/java/2_4_0/api/core/org/apache/lucene/search/FieldCache.html EG you can load ints (for example) like this: FieldCache.DEFAULT.getInts(reader, myfield); This returns an array mapping docID -- int

Re: Boosting results

2008-11-10 Thread Stefan Trcek
On Friday 07 November 2008 18:46:17 Michael McCandless wrote: Sorting populates the field cache (internal to Lucene) for that field,   meaning it loads all values for all docs and holds them in memory. This makes the first query slow, and, consumes RAM, in proportion to how large your index

Re: Boosting results

2008-11-10 Thread Stefan Trcek
On Monday 10 November 2008 13:55:31 Michael McCandless wrote: Finally, you might want to instead look at Solr, which provides facet counting out of the box, rather than roll your own... Doooh - new api, but it's facet counting sounds good. Any starting points for moving from plain lucene to

Re: Boosting results

2008-11-10 Thread Erik Hatcher
On Nov 10, 2008, at 2:42 PM, Stefan Trcek wrote: On Monday 10 November 2008 13:55:31 Michael McCandless wrote: Finally, you might want to instead look at Solr, which provides facet counting out of the box, rather than roll your own... Doooh - new api, but it's facet counting sounds good.

Re: Boosting results

2008-11-07 Thread Erick Erickson
dh, sorting. I absolutely love it when I overlook the obvious G. [EMAIL PROTECTED] On Fri, Nov 7, 2008 at 4:58 AM, Michael McCandless [EMAIL PROTECTED] wrote: Couldn't you just do a single Query that sorts first by category and second by relevance? Mike Erick Erickson wrote: It

Re: Boosting results

2008-11-07 Thread Michael McCandless
Couldn't you just do a single Query that sorts first by category and second by relevance? Mike Erick Erickson wrote: It seems to me that the easiest thing would be to fire two queries and then just concatenate the results category:A AND body:fred category:B AND body:fred If you

Re: Boosting results

2008-11-07 Thread Matthew DeLoria
This actually brings up an interesting question, and something I have been curious about. In this case, does it make more sense to do Boosting by Category, or to do sorting? From what I understand, Lucene sorting involves putting the relevant fields into memory, and then executing a sort. Is

RE: Boosting results

2008-11-07 Thread Scott Smith
fields and score (aka relevancy) is one of the pseudo fields. That'll work. Thanks. Scott -Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: Friday, November 07, 2008 5:59 AM To: java-user@lucene.apache.org Subject: Re: Boosting results dh, sorting. I absolutely

Re: Boosting results

2008-11-07 Thread Michael McCandless
This is a good point. Sorting populates the field cache (internal to Lucene) for that field, meaning it loads all values for all docs and holds them in memory. This makes the first query slow, and, consumes RAM, in proportion to how large your index is. Whereas boosting should be able

Re: Boosting results

2008-11-07 Thread Peter Keegan
To: java-user@lucene.apache.org Subject: Re: Boosting results dh, sorting. I absolutely love it when I overlook the obvious G. [EMAIL PROTECTED] On Fri, Nov 7, 2008 at 4:58 AM, Michael McCandless [EMAIL PROTECTED] wrote: Couldn't you just do a single Query that sorts first

Re: Boosting results

2008-11-06 Thread Erick Erickson
It seems to me that the easiest thing would be to fire two queries and then just concatenate the results category:A AND body:fred category:B AND body:fred If you really, really didn't want to fire two queries, you could create filters on category A and category B and make a couple of passes