On thinking more about this... I think with a few small changes we
could achieve Sort by field without materializing a full array. We
can decouple this change from LUCENE-831.
I think all that's needed is:
* Expose sub-readers (LUCENE-1475) by adding IndexReader[]
IndexReader.getSubReaders. Default impl could just return
length-1 array of itself.
* Change IndexSearcher.sort that takes a Sort, to first call
IndexReader.getSubReaders, and then do the same logic that
MultiSearcher does, with improvements from LUCENE-1471 (run
separate search per-reader, then merge-sort the top hits from
each).
The results should be functionally identical to what we have today,
but, searching after doing a reopen() should be much faster since we'd
no longer re-build the global FieldCache array.
Does this make sense? It's a small change for a big win, I think.
Does anyone want to take a crack at this patch?
Mike
Mark Miller wrote:
Michael McCandless wrote:
I'd like to decouple "upgraded to Object" vs "materialize full
array", ie, so we can access native values w/o materializing the
full array. I also think "upgrade to Object" is dangerous to even
offer since it's so costly.
I'm right with you. I didn't think the Object approach was really an
upgrade (beyond losing the merge, which is especially important for
StringIndex - it has no merge option at the moment) which is why I
left both options for now. So I def agree we need to move to
iterator, drop object, etc.
Its the doin' that aint so easy. The iterator approach seems
somewhat straightforward (though its complicated by needing to
provide a random access object as well), but I'm still working
through how we control so many iterator types (I dont see how you
can use polymorphism yet ).
- Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
