[
https://issues.apache.org/jira/browse/LUCENE-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537977
]
Yonik Seeley commented on LUCENE-1035:
--------------------------------------
A couple of random thoughts
- previous tests showed that vint decoding was often bottleneck, but these
tests seem to indicate otherwise (that the bottleneck is the system call to
move data from OS FS cache to userspace?)... perhaps this is due to the fact
that all queries are "AND" and match a max of 1000 docs? The recent addition
of multi-level skipping perhaps removes the vint decoding bottleneck for these
types of queries that match few documents.
- most lucene usecases store much more than just the document id... that would
really affect locality.
- It seems like a simple LRU cache could really be blown out of the water by
certain types of queries (retrieve a lot of stored fields, or do an expanding
term query) that would force out all previously cached hotspots. Most OS level
caching has protection against this (multi-level LRU or whatever). But of our
user-level LRU cache fails, we've also messed up the OS level cache since we've
been hiding page hits from it.
- I'd like to see single term queries, "OR" queries, and queries across
multiple fields (also a common usecase) that match more documents tested also.
> Optional Buffer Pool to Improve Search Performance
> --------------------------------------------------
>
> Key: LUCENE-1035
> URL: https://issues.apache.org/jira/browse/LUCENE-1035
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Store
> Reporter: Ning Li
> Attachments: LUCENE-1035.patch
>
>
> Index in RAMDirectory provides better performance over that in FSDirectory.
> But many indexes cannot fit in memory or applications cannot afford to
> spend that much memory on index. On the other hand, because of locality,
> a reasonably sized buffer pool may provide good improvement over FSDirectory.
> This issue aims at providing such an optional buffer pool layer. In cases
> where it fits, i.e. a reasonable hit ratio can be achieved, it should provide
> a good improvement over FSDirectory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
