[
https://issues.apache.org/jira/browse/CASSANDRA-5182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13577446#comment-13577446
]
Sylvain Lebresne commented on CASSANDRA-5182:
---------------------------------------------
bq. If our goal is to throw out the maximum possible amount of obsolete data
I kind agree with Bryan, this doesn't have to be black and white. What we want
is doing the best we can to remove obsolete rows without impacting compaction
too much. Now if you do have active bloom filters, then I think just checking
the bloom filters as we do now is the right trade-off: it maximize with a very
high probability the amount of removed data at the relatively cheap cost. Using
getPosition in that case would be a bad idea, because the reward (a tiny
fraction of additional data potentially removed) is not worth the cost (hitting
disk each time a row we compact is also in a non-compacted sstable) imo, hence
my opposition to the idea.
But if you deactivate bloom filters, you also fully destroy our bloom filter
trade-off. So using getPosition does now provide a substantial benefit as it
allows to go from 'no deletion' to 'maximize deletion'. The reward is, in that
case, likely worth the cost, especially since people shouldn't desactivate
bloom filters unless their index files fits in memory, in which case
getPosition costs won't be that big.
So overall I do like the last patch attached by Yuki. Of course, the solution
of just saying "you shouldn't disable bloom filters on workloads that perform
deletes" works too, and I wouldn't oppose it, but it doesn't have my preference
because I'm always a bit afraid of solving an issue by saying "don't do this",
as it usually end up in people getting bitten first and hearing they shouldn't
have done it second.
> Deletable rows are sometimes not removed during compaction
> ----------------------------------------------------------
>
> Key: CASSANDRA-5182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5182
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.0.7
> Reporter: Binh Van Nguyen
> Assignee: Yuki Morishita
> Fix For: 1.2.2
>
> Attachments: 5182-1.1.txt, 5182-1.2.txt, test_ttl.tar.gz
>
>
> Our use case is write heavy and read seldom. To optimize the space used,
> we've set the bloom_filter_fp_ratio=1.0 That along with the fact that each
> row is only written to one time and that there are more than 20 SSTables
> keeps the rows from ever being compacted. Here is the code:
> https://github.com/apache/cassandra/blob/cassandra-1.1/src/java/org/apache/cassandra/db/compaction/CompactionController.java#L162
> We hit this conner case and because of this C* keeps consuming more and more
> space on disk while it should not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
