Re: Query regarding SSTable timestamps and counts

2012-12-10 Thread B. Todd Burruss
my two cents ... i know this thread is a bit old, but the fact that odd-sized SSTABLEs (usually large ones) will hang around for a while can be very troublesome on disk space and planning. our data is temporal in cassandra, being deleted constantly. we have seen space usage in the 1+ TB range

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread aaron morton
My understanding of the compaction process was that since data files keep continuously merging we should not have data files with very old last modified timestamps It is perfectly OK to have very old SSTables. But performing an upgradesstables did decrease the number of data files and

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread Edward Capriolo
On Tue, Nov 20, 2012 at 5:23 PM, aaron morton aa...@thelastpickle.com wrote: My understanding of the compaction process was that since data files keep continuously merging we should not have data files with very old last modified timestamps It is perfectly OK to have very old SSTables. But

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread Ananth Gundabattula
Thanks a lot Aaron and Edward. The mail thread clarifies some things for me. For letting others know on this thread, running an upgradesstables did decrease our bloom filter false positive ratios a lot. ( upgradesstables was run not to upgrade from a casasndra version to a higher cassandra

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread aaron morton
upgradetables re-writes every sstable to have the same contents in the newest format. Agree. In the world of compaction, and excluding upgrades, have older sstables is expected. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton

Re: Query regarding SSTable timestamps and counts

2012-11-19 Thread Rob Coli
On Sun, Nov 18, 2012 at 7:57 PM, Ananth Gundabattula agundabatt...@gmail.com wrote: As per the above url, After running a major compaction, automatic minor compactions are no longer triggered, frequently requiring you to manually run major compactions on a routine basis. ( Just before the

Re: Query regarding SSTable timestamps and counts

2012-11-18 Thread aaron morton
As per datastax documentation, a manual compaction forces the admin to start compaction manually and disables the automated compaction (atleast for major compactions but not minor compactions ) It does not disable compaction. it creates one big file, which will not be compacted until there

Re: Query regarding SSTable timestamps and counts

2012-11-18 Thread Ananth Gundabattula
Hello Aaron, Thanks a lot for the reply. Looks like the documentation is confusing. Here is the link I am referring to: http://www.datastax.com/docs/1.1/operations/tuning#tuning-compaction It does not disable compaction. As per the above url, After running a major compaction, automatic