[
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286868#comment-13286868
]
Simon Willnauer commented on LUCENE-4069:
-----------------------------------------
bq. I really dont think we should make this complex to save 2 or 3 files total
(even in a complex config with many fields). Its not worth the complexity.
I agree. I think those postings formats should only deal with encoding and not
with handling certain fields different. A user / app should handle this in the
codec. Ideally you don't have any conditions in the relevant methods like
termsConsumer etc.
bq. For me this falls into one of the many faster-if-you-know-about-it
optimisations like FieldSelectors or recycling certain objects. Basically a
useful hint to Lucene to save some extra effort but one which you dont need to
use.
why is this a speed improvement? reading from one file vs. multiple is not
really faster though.
Anyway, I think we should make this patch as simple as possible and don't
handle fields in the PF. We can still open another issue or wait until
LUCENE-4093 is in to discuss this issue?
> Segment-level Bloom filters for a 2 x speed up on rare term searches
> --------------------------------------------------------------------
>
> Key: LUCENE-4069
> URL: https://issues.apache.org/jira/browse/LUCENE-4069
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 3.6, 4.0
> Reporter: Mark Harwood
> Priority: Minor
> Fix For: 4.0, 3.6.1
>
> Attachments: BloomFilterPostings40.patch,
> MHBloomFilterOn3.6Branch.patch, PrimaryKey40PerformanceTestSrc.zip
>
>
> An addition to each segment which stores a Bloom filter for selected fields
> in order to give fast-fail to term searches, helping avoid wasted disk access.
> Best suited for low-frequency fields e.g. primary keys on big indexes with
> many segments but also speeds up general searching in my tests.
> Overview slideshow here:
> http://www.slideshare.net/MarkHarwood/lucene-bloomfilteredsegments
> Benchmarks based on Wikipedia content here: http://goo.gl/X7QqU
> Patch based on 3.6 codebase attached.
> There are no 3.6 API changes currently - to play just add a field with "_blm"
> on the end of the name to invoke special indexing/querying capability.
> Clearly a new Field or schema declaration(!) would need adding to APIs to
> configure the service properly.
> Also, a patch for Lucene4.0 codebase introducing a new PostingsFormat
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
