Re: One time major deletion/purge vs periodic deletion

2018-03-20 Thread Carl Mueller
It's possible you'll run into compaction headaches. Likely actually. If you have time-bucketed purge/archives, I'd implement a time bucketing strategy using rotating tables dedicated to a time period so that when an entire table is ready for archiving you just snapshot its sstables and then

Re: One time major deletion/purge vs periodic deletion

2018-03-07 Thread kurt greaves
The important point to consider is whether you are deleting old data or recently written data. How old/recent depends on your write rate to the cluster and there's no real formula. Basically you want to avoid deleting a lot of old data all at once because the tombstones will end up in new SSTables

Re: One time major deletion/purge vs periodic deletion

2018-03-07 Thread Rahul Singh
t;jens.ran...@tink.se> > > > Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> > > > Date: Tuesday, March 6, 2018 at 12:34 AM > > > To: "user@cassandra.apache.org" <user@cassandra.apache.org> > > > Subject: Re:

Re: One time major deletion/purge vs periodic deletion

2018-03-07 Thread Ben Slater
r@cassandra.apache.org" <user@cassandra.apache.org> > *Date: *Tuesday, March 6, 2018 at 12:34 AM > *To: *"user@cassandra.apache.org" <user@cassandra.apache.org> > *Subject: *Re: One time major deletion/purge vs periodic deletion > > > > Sounds lik

Re: One time major deletion/purge vs periodic deletion

2018-03-06 Thread Charulata Sharma (charshar)
ra.apache.org> Date: Tuesday, March 6, 2018 at 12:34 AM To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: Re: One time major deletion/purge vs periodic deletion Sounds like you are using Cassandra as a queue. It's an antibiotic pattern. What I would do would

Re: One time major deletion/purge vs periodic deletion

2018-03-06 Thread Jens Rantil
Sounds like you are using Cassandra as a queue. It's an antibiotic pattern. What I would do would be to rely on TTL for removal of data and use the TWCS compaction strategy to handle removal and you just focus on insertion. On Tue, Mar 6, 2018, 07:39 Charulata Sharma (charshar)