[
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286754#comment-13286754
]
Mark Harwood commented on LUCENE-4069:
--------------------------------------
Its true to say that Bloom is a different case to Pulsing - Bloom does not
interfere in any with the normal recording of content in the wrapped delegate
whereas Pulsing does.
It may prove useful for us to mark a formal distinction between these
mutating/non mutating types so we can treat them differently and provide
optimisations?
bq. And separately, you can always contain the number of files even today by
using only unique instances yourself when writing
Contained but not optimal - roughly double the number of required files if I
want the common case of a primary key indexed with Bloom. I can't see a way of
indexing with Bloom-plus-Lucene40 on field "A" and indexing with just Lucene40
on fields B,C and D and winding up with only one Lucene40 set of files with a
common segment suffix. The way I did find of achieving this was to add a
"bloomFilteredFields" set into my single Bloom+Lucene40 instance used for all
fields. Is there any other option here currently?
Looking to the future, 4093 may have more capabilities at optimising if it
understands the distinction between mutating wrappers and non-mutating ones and
how they are composed?
> Segment-level Bloom filters for a 2 x speed up on rare term searches
> --------------------------------------------------------------------
>
> Key: LUCENE-4069
> URL: https://issues.apache.org/jira/browse/LUCENE-4069
> Project: Lucene - Java
> Issue Type: Improvement
> Components: core/index
> Affects Versions: 3.6, 4.0
> Reporter: Mark Harwood
> Priority: Minor
> Fix For: 4.0, 3.6.1
>
> Attachments: BloomFilterPostings40.patch,
> MHBloomFilterOn3.6Branch.patch, PrimaryKey40PerformanceTestSrc.zip
>
>
> An addition to each segment which stores a Bloom filter for selected fields
> in order to give fast-fail to term searches, helping avoid wasted disk access.
> Best suited for low-frequency fields e.g. primary keys on big indexes with
> many segments but also speeds up general searching in my tests.
> Overview slideshow here:
> http://www.slideshare.net/MarkHarwood/lucene-bloomfilteredsegments
> Benchmarks based on Wikipedia content here: http://goo.gl/X7QqU
> Patch based on 3.6 codebase attached.
> There are no 3.6 API changes currently - to play just add a field with "_blm"
> on the end of the name to invoke special indexing/querying capability.
> Clearly a new Field or schema declaration(!) would need adding to APIs to
> configure the service properly.
> Also, a patch for Lucene4.0 codebase introducing a new PostingsFormat
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
