Re: disabling bloomfilter not working? or did I do this wrong?
Hmmm, ok, that makes sense. I suspect the same is true with leveled compaction as well? Thanks, Dean On 2/25/13 6:47 AM, Edward Capriolo edlinuxg...@gmail.com wrote: Mostly but not 100%. You have a bloom filter for each sstable, so going to disk means finding the row in each sstable if you end up skipping some you are better off. Sometimes you have the data but not in sstable N. The bloom filter helps avoid checking sstable N to find nothing.
disabling bloomfilter not working? or did I do this wrong?
So in the cli, I ran update column family nreldata with bloom_filter_fp_chance=1.0; Then I ran nodetool upgradesstables databus5 nreldata; But my bloom filter size is still around 2gig(and I want to free up this heap) According to nodetool cfstats command… Column Family: nreldata SSTable count: 10 Space used (live): 96841497731 Space used (total): 96841497731 Number of Keys (estimate): 1249133696 Memtable Columns Count: 7066 Memtable Data Size: 4286174 Memtable Switch Count: 924 Read Count: 19087150 Read Latency: 0.595 ms. Write Count: 21281994 Write Latency: 0.013 ms. Pending Tasks: 0 Bloom Filter False Postives: 974393 Bloom Filter False Ratio: 0.8 Bloom Filter Space Used: 2318392048 Compacted row minimum size: 73 Compacted row maximum size: 446 Compacted row mean size: 143
Re: disabling bloomfilter not working? or did I do this wrong?
Bloom Filter Space Used: 2318392048 Just to be sane do a quick check of the -Filter.db files on disk for this CF. If they are very small try a restart on the node. Number of Keys (estimate): 1249133696 Hey a billion rows on a node, what an age we live in :) Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 23/02/2013, at 4:35 AM, Hiller, Dean dean.hil...@nrel.gov wrote: So in the cli, I ran update column family nreldata with bloom_filter_fp_chance=1.0; Then I ran nodetool upgradesstables databus5 nreldata; But my bloom filter size is still around 2gig(and I want to free up this heap) According to nodetool cfstats command… Column Family: nreldata SSTable count: 10 Space used (live): 96841497731 Space used (total): 96841497731 Number of Keys (estimate): 1249133696 Memtable Columns Count: 7066 Memtable Data Size: 4286174 Memtable Switch Count: 924 Read Count: 19087150 Read Latency: 0.595 ms. Write Count: 21281994 Write Latency: 0.013 ms. Pending Tasks: 0 Bloom Filter False Postives: 974393 Bloom Filter False Ratio: 0.8 Bloom Filter Space Used: 2318392048 Compacted row minimum size: 73 Compacted row maximum size: 446 Compacted row mean size: 143
Re: disabling bloomfilter not working? or did I do this wrong?
Thanks, but I found out it is still running. It looks like I have about a 5 hour wait left for my upgradesstables(waited 4 hours already). I will check the bloomfilter after that. Out of curiosity, if I had much wider rows (ie. 900k) per row, will compaction run faster(e…upgradesstables) at all or would it basically run at the same speed. I guess what I am wondering is 9 hours a normal compaction time for 130gb of data? Thanks, Dean From: aaron morton aa...@thelastpickle.commailto:aa...@thelastpickle.com Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Date: Friday, February 22, 2013 10:29 AM To: user@cassandra.apache.orgmailto:user@cassandra.apache.org user@cassandra.apache.orgmailto:user@cassandra.apache.org Subject: Re: disabling bloomfilter not working? or did I do this wrong? Bloom Filter Space Used: 2318392048 Just to be sane do a quick check of the -Filter.db files on disk for this CF. If they are very small try a restart on the node. Number of Keys (estimate): 1249133696 Hey a billion rows on a node, what an age we live in :) Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 23/02/2013, at 4:35 AM, Hiller, Dean dean.hil...@nrel.govmailto:dean.hil...@nrel.gov wrote: So in the cli, I ran update column family nreldata with bloom_filter_fp_chance=1.0; Then I ran nodetool upgradesstables databus5 nreldata; But my bloom filter size is still around 2gig(and I want to free up this heap) According to nodetool cfstats command… Column Family: nreldata SSTable count: 10 Space used (live): 96841497731 Space used (total): 96841497731 Number of Keys (estimate): 1249133696 Memtable Columns Count: 7066 Memtable Data Size: 4286174 Memtable Switch Count: 924 Read Count: 19087150 Read Latency: 0.595 ms. Write Count: 21281994 Write Latency: 0.013 ms. Pending Tasks: 0 Bloom Filter False Postives: 974393 Bloom Filter False Ratio: 0.8 Bloom Filter Space Used: 2318392048 Compacted row minimum size: 73 Compacted row maximum size: 446 Compacted row mean size: 143
