Re: Bloom filters and LCS
I think my impression that Bloom Filters were off in 1.1.9 was a misinterpretation of this thread: http://www.mail-archive.com/user@cassandra.apache.org/msg27787.html and this bug: https://issues.apache.org/jira/browse/CASSANDRA-5029 I read it that Bloom filters were added to 1.2.2 for Bloomfilters, but apparently after being shutoff in an earlier version of 1.2? -Mike On Mar 7, 2013, at 4:48 PM, Edward Capriolo wrote: I read that the change was made because Cassandra does not work well when they are off. This makes sense because cassandra uses bloom filters to decide if a row can be deleted without major compaction. However since LCS does not major compact without bloom filters you can end up in cases where rows never get deleted. Edward On Thu, Mar 7, 2013 at 4:30 PM, Wei Zhu wz1...@yahoo.com wrote: Where did you read that bloom filters are off for LCS on 1.1.9? Those are the two issues I can find regarding this matter: https://issues.apache.org/jira/browse/CASSANDRA-4876 https://issues.apache.org/jira/browse/CASSANDRA-5029 Looks like in 1.2, it defaults at 0.1, not sure about 1.1.X -Wei - Original Message - From: Michael Theroux mthero...@yahoo.com To: user@cassandra.apache.org Sent: Thursday, March 7, 2013 1:18:38 PM Subject: Bloom filters and LCS Hello, (Hopefully) Quick question. We are running Cassandra 1.1.9. I recently converted some tables from Size tiered to Leveled Compaction. The amount of space for Bloom Filters on these tables went down tremendously (which is expected, LCS in 1.1.9 does not use bloom filters). However, although its far less, its still using a number of megabytes. Why is it not zero? Column Family: SSTable count: 526 Space used (live): 7251063348 Space used (total): 7251063348 Number of Keys (estimate): 23895552 Memtable Columns Count: 45719 Memtable Data Size: 21207173 Memtable Switch Count: 579 Read Count: 21773431 Read Latency: 4.155 ms. Write Count: 16183367 Write Latency: 0.029 ms. Pending Tasks: 0 Bloom Filter False Positives: 2442 Bloom Filter False Ratio: 0.00245 Bloom Filter Space Used: 44674656 Compacted row minimum size: 73 Compacted row maximum size: 105778 Compacted row mean size: 1104 Thanks, -Mike
Bloom filters and LCS
Hello, (Hopefully) Quick question. We are running Cassandra 1.1.9. I recently converted some tables from Size tiered to Leveled Compaction. The amount of space for Bloom Filters on these tables went down tremendously (which is expected, LCS in 1.1.9 does not use bloom filters). However, although its far less, its still using a number of megabytes. Why is it not zero? Column Family: SSTable count: 526 Space used (live): 7251063348 Space used (total): 7251063348 Number of Keys (estimate): 23895552 Memtable Columns Count: 45719 Memtable Data Size: 21207173 Memtable Switch Count: 579 Read Count: 21773431 Read Latency: 4.155 ms. Write Count: 16183367 Write Latency: 0.029 ms. Pending Tasks: 0 Bloom Filter False Positives: 2442 Bloom Filter False Ratio: 0.00245 Bloom Filter Space Used: 44674656 Compacted row minimum size: 73 Compacted row maximum size: 105778 Compacted row mean size: 1104 Thanks, -Mike
Re: Bloom filters and LCS
It was found out that that having no bloom filter is a bad idea because it causes issues where deleted rows are never removed from disk. Newer versions have fixed this. You should adjust your bloom filter settings to be 0 sized. On Thu, Mar 7, 2013 at 4:18 PM, Michael Theroux mthero...@yahoo.com wrote: Hello, (Hopefully) Quick question. We are running Cassandra 1.1.9. I recently converted some tables from Size tiered to Leveled Compaction. The amount of space for Bloom Filters on these tables went down tremendously (which is expected, LCS in 1.1.9 does not use bloom filters). However, although its far less, its still using a number of megabytes. Why is it not zero? Column Family: SSTable count: 526 Space used (live): 7251063348 Space used (total): 7251063348 Number of Keys (estimate): 23895552 Memtable Columns Count: 45719 Memtable Data Size: 21207173 Memtable Switch Count: 579 Read Count: 21773431 Read Latency: 4.155 ms. Write Count: 16183367 Write Latency: 0.029 ms. Pending Tasks: 0 Bloom Filter False Positives: 2442 Bloom Filter False Ratio: 0.00245 Bloom Filter Space Used: 44674656 Compacted row minimum size: 73 Compacted row maximum size: 105778 Compacted row mean size: 1104 Thanks, -Mike
Re: Bloom filters and LCS
Where did you read that bloom filters are off for LCS on 1.1.9? Those are the two issues I can find regarding this matter: https://issues.apache.org/jira/browse/CASSANDRA-4876 https://issues.apache.org/jira/browse/CASSANDRA-5029 Looks like in 1.2, it defaults at 0.1, not sure about 1.1.X -Wei - Original Message - From: Michael Theroux mthero...@yahoo.com To: user@cassandra.apache.org Sent: Thursday, March 7, 2013 1:18:38 PM Subject: Bloom filters and LCS Hello, (Hopefully) Quick question. We are running Cassandra 1.1.9. I recently converted some tables from Size tiered to Leveled Compaction. The amount of space for Bloom Filters on these tables went down tremendously (which is expected, LCS in 1.1.9 does not use bloom filters). However, although its far less, its still using a number of megabytes. Why is it not zero? Column Family: SSTable count: 526 Space used (live): 7251063348 Space used (total): 7251063348 Number of Keys (estimate): 23895552 Memtable Columns Count: 45719 Memtable Data Size: 21207173 Memtable Switch Count: 579 Read Count: 21773431 Read Latency: 4.155 ms. Write Count: 16183367 Write Latency: 0.029 ms. Pending Tasks: 0 Bloom Filter False Positives: 2442 Bloom Filter False Ratio: 0.00245 Bloom Filter Space Used: 44674656 Compacted row minimum size: 73 Compacted row maximum size: 105778 Compacted row mean size: 1104 Thanks, -Mike
Re: Bloom filters and LCS
I read that the change was made because Cassandra does not work well when they are off. This makes sense because cassandra uses bloom filters to decide if a row can be deleted without major compaction. However since LCS does not major compact without bloom filters you can end up in cases where rows never get deleted. Edward On Thu, Mar 7, 2013 at 4:30 PM, Wei Zhu wz1...@yahoo.com wrote: Where did you read that bloom filters are off for LCS on 1.1.9? Those are the two issues I can find regarding this matter: https://issues.apache.org/jira/browse/CASSANDRA-4876 https://issues.apache.org/jira/browse/CASSANDRA-5029 Looks like in 1.2, it defaults at 0.1, not sure about 1.1.X -Wei - Original Message - From: Michael Theroux mthero...@yahoo.com To: user@cassandra.apache.org Sent: Thursday, March 7, 2013 1:18:38 PM Subject: Bloom filters and LCS Hello, (Hopefully) Quick question. We are running Cassandra 1.1.9. I recently converted some tables from Size tiered to Leveled Compaction. The amount of space for Bloom Filters on these tables went down tremendously (which is expected, LCS in 1.1.9 does not use bloom filters). However, although its far less, its still using a number of megabytes. Why is it not zero? Column Family: SSTable count: 526 Space used (live): 7251063348 Space used (total): 7251063348 Number of Keys (estimate): 23895552 Memtable Columns Count: 45719 Memtable Data Size: 21207173 Memtable Switch Count: 579 Read Count: 21773431 Read Latency: 4.155 ms. Write Count: 16183367 Write Latency: 0.029 ms. Pending Tasks: 0 Bloom Filter False Positives: 2442 Bloom Filter False Ratio: 0.00245 Bloom Filter Space Used: 44674656 Compacted row minimum size: 73 Compacted row maximum size: 105778 Compacted row mean size: 1104 Thanks, -Mike
