[
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286598#comment-13286598
]
Mark Harwood commented on LUCENE-4069:
--------------------------------------
bq. Instead i think the concrete Bloom+Lucene40 that you have in tests should
be moved into src/java and registered there
What problem would that be trying to solve? Registration (or creation) of any
BloomFilteringPostingsFormat subclasses is not necessary to decode index
contents. Offering a "Bloom40" would only buy users a pairing of
Lucene40Postings and Bloom filtering but they would still have to declare which
fields they want Bloom filtering on at write time. This isn't too hard using
the code in the existing patch:
{code:title=ThisWorks.java}
final Set<String>bloomFilteredFields=new HashSet<String>();
bloomFilteredFields.add(PRIMARY_KEY_FIELD_NAME);
iwc.setCodec(new Lucene40Codec(){
BloomFilteringPostingsFormat postingOptions=new
BloomFilteringPostingsFormat(new Lucene40PostingsFormat(), bloomFilteredFields);
@Override
public PostingsFormat getPostingsFormatForField(String field) {
return postingOptions;
}
});
{code}
No extra subclasses/registration required here to read the index built with the
above setup.
> Segment-level Bloom filters for a 2 x speed up on rare term searches
> --------------------------------------------------------------------
>
> Key: LUCENE-4069
> URL: https://issues.apache.org/jira/browse/LUCENE-4069
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 3.6, 4.0
> Reporter: Mark Harwood
> Priority: Minor
> Fix For: 4.0, 3.6.1
>
> Attachments: BloomFilterPostings40.patch,
> MHBloomFilterOn3.6Branch.patch, PrimaryKey40PerformanceTestSrc.zip
>
>
> An addition to each segment which stores a Bloom filter for selected fields
> in order to give fast-fail to term searches, helping avoid wasted disk access.
> Best suited for low-frequency fields e.g. primary keys on big indexes with
> many segments but also speeds up general searching in my tests.
> Overview slideshow here:
> http://www.slideshare.net/MarkHarwood/lucene-bloomfilteredsegments
> Benchmarks based on Wikipedia content here: http://goo.gl/X7QqU
> Patch based on 3.6 codebase attached.
> There are no 3.6 API changes currently - to play just add a field with "_blm"
> on the end of the name to invoke special indexing/querying capability.
> Clearly a new Field or schema declaration(!) would need adding to APIs to
> configure the service properly.
> Also, a patch for Lucene4.0 codebase introducing a new PostingsFormat
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
