[
https://issues.apache.org/jira/browse/LUCENE-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12797134#action_12797134
]
Grant Ingersoll commented on LUCENE-2127:
-----------------------------------------
bq. A fixed array of ScoreDocs should work fine? But you do need to tell
FieldComparator up front the biggest it will be
It's not fixed. I want to experiment with the idea of collecting all the
results and then sorting, based on the discussion in the link above. I'm
seeing more and more cases, esp. for heavy machine learning/post processing
apps, where the app will retrieve 1M+ docs per query.
Thus, I modified ArrayUtils to have a grow method that takes in a ScoreDoc
array, but can also pass in a preallocated array. I'm not convinced this is
going to be better, but I think it's interesting to find out. Should have a
patch today.
FWIW, I think you approximate PQ makes a lot of sense too.
> Improved large result handling
> ------------------------------
>
> Key: LUCENE-2127
> URL: https://issues.apache.org/jira/browse/LUCENE-2127
> Project: Lucene - Java
> Issue Type: New Feature
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
>
> Per
> http://search.lucidimagination.com/search/document/350c54fc90d257ed/lots_of_results#fbb84bd297d15dd5,
> it would be nice to offer some other Collectors that are better at handling
> really large number of results. This could be implemented in a variety of
> ways via Collectors. For instance, we could have a raw collector that does
> no sorting and just returns the ScoreDocs, or we could do as Mike suggests
> and have Collectors that have heuristics about memory tradeoffs and only
> heapify when appropriate.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-dev-h...@lucene.apache.org
