[
https://issues.apache.org/jira/browse/LUCENE-769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12463969
]
Artem Vasiliev commented on LUCENE-769:
---------------------------------------
Hi guys!
Thanks for value comments. What a feedback! :)
I'd like to stress the point of my fix - to avoid costly FieldCache population
with field values from the whole index.
Your point that it will be slower for cases when filtered sets be nearly as
large as the whole index is valid. But is it a practical point? Lucene shines
on big indexes and queries resulting with full index are not very useful I
guess.
I think it's good idea to hide the caching reader class and utilize
FieldSelector mechanism to make the fix more effective. However do you think
this improvement worth doing? You are strong opposition and I'm not feeling up
to an endless fight :) I'm serious, let me know what you think. This fix will
have its limitations by no means but I think the above OutOfMemory scenario
with current sorting mechanism alone makes this fix legitimate.
> [PATCH] Performance improvement for some cases of sorted search
> ---------------------------------------------------------------
>
> Key: LUCENE-769
> URL: https://issues.apache.org/jira/browse/LUCENE-769
> Project: Lucene - Java
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Artem Vasiliev
> Attachments: DocCachingSorting.patch, DocCachingSorting.patch
>
>
> It's a small addition to Lucene that significantly lowers memory consumption
> and improves performance for sorted searches with frequent index updates and
> relatively big indexes (>1mln docs) scenario. This solution supports only
> single-field sorting currently (which seem to be quite popular use case).
> Multiple fields support can be added without much trouble.
> The solution is this: documents from the sorting set (instead of given
> field's values from the whole index - current FieldCache approach) are cached
> in a WeakHashMap so the cached items are candidates for GC. Their fields
> values are then fetched from the cache and compared while sorting.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
