We are running into the same issue. Remember that hits just give you
doc id and getting hit details from the hit does another read. So
looping through the hits to access every document will do a read per
document. If it is a small number of hits, no big deal, but the more
hits to access, the more time. For our situation limiting the query
doesn't work, we need to know information about the hit itself (i.e. a
certain field so we can do a count based on the field). We implemented
it using HitCollector modifications in Lucene. This works but is not
ideal in terms of speed so we are looking at making modifications to the
IndexReader itself so when it gets the Hits it also gets our field.
Understand that doing something like this though changes core Lucene
functionality. I am not necessarily recommending doing it this way, we
just couldn't find another way.
Dennis
Andrzej Bialecki wrote:
steveb wrote:
I would like to use my own HitCollector when doing a search using the
NutchBean as I have a requirement to access every document in the
result set
but without incurring the cost of traversing the Hits collection.
Accessing every document will be costly no matter what interface you
will use ... you may need to rethink your requirements. If you only
need to check for presence / absence of certain fields/terms, then
perhaps using Lucene query filters is a better idea.
- Re: Plugin HitCollector Dennis Kubes
-
