[
https://issues.apache.org/jira/browse/CASSANDRA-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640907#comment-14640907
]
Paulo Motta edited comment on CASSANDRA-9830 at 7/24/15 6:48 PM:
-----------------------------------------------------------------
Initial trunk patch available.
Basic idea was to add option "skipBloomFilter" to supress bloom filter creation
on SStableWriter, and use this option during creation of top-level SSTables
when the skip_top_level_bloom_filter property is enabled for
LeveledCompactionStrategy. SSTableReader also had a minor modification to avoid
recreating bloom filters due to the missing FILTER file.
The test scenario creates a LCS table with 3 levels and checks the bloom filter
is not loaded in memory nor persisted to disk. Also, the SSTables are reloaded
from disk to check the bloom filter of persisted SSTables is skipped.
I called the compactrion suproperty to skip top level sstables:
skip_top_level_bloom_filter, don't know if that's the best name. I added CQLSH
auto-completion and documentation.
was (Author: pauloricardomg):
Initial trunk patch available.
Basic idea was to add option "skipBloomFilter" to supress bloom filter creation
on SStableWriter, and use this option during creation of top-level SSTables
when the skip_top_level_sstables property is enabled for
LeveledCompactionStrategy. SSTableReader also had a minor modification to avoid
recreating bloom filters due to the missing FILTER file.
The test scenario creates a LCS table with 3 levels and checks the bloom filter
is not loaded in memory nor persisted to disk. Also, the SSTables are reloaded
from disk to check the bloom filter of persisted SSTables is skipped.
I called the compactrion suproperty to skip top level sstables:
skip_top_level_sstables, don't know if that's the best name. I added CQLSH
auto-completion and documentation.
> Option to disable bloom filter in highest level of LCS sstables
> ---------------------------------------------------------------
>
> Key: CASSANDRA-9830
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9830
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Jonathan Ellis
> Assignee: Paulo Motta
> Priority: Minor
> Labels: performance
> Fix For: 3.x
>
>
> We expect about 90% of data to be in the highest level of LCS in a fully
> populated series. (See also CASSANDRA-9829.)
> Thus if the user is primarily asking for data (partitions) that has actually
> been inserted, the bloom filter on the highest level only helps reject
> sstables about 10% of the time.
> We should add an option that suppresses bloom filter creation on top-level
> sstables. This will dramatically reduce memory usage for LCS and may even
> improve performance as we no longer check a low-value filter.
> (This is also an idea from RocksDB.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
