So is this at least a decent candidate for a feature request ticket?
On Tue, Feb 13, 2018 at 8:09 PM, Carl Mueller <carl.muel...@smartthings.com>
> I'm particularly interested in getting the tombstones to "promote" up the
> levels of LCS more quickly. Currently they get attached at the low level
> and don't propagate up to higher levels until enough activity at a lower
> level promotes the data. Meanwhile, LCS means compactions can occur in
> parallel at each level. So row tombstones in their own sstable could be up
> promoted the LCS levels preferentially before normal processes would move
> them up.
> So if the delete-only sstables could move up more quickly, the compaction
> at the levels would happen more quickly.
> The threshold stuff is nice if I read 7019 correctly, but what is the %
> there? % of rows? % of columns? or % of the size of the sstable? Row
> tombstones are pretty compact being just the rowkey and the tombstone
> marker. So if 7019 is triggered at 10% of the sstable size, even a crapton
> of tombstones deleting practially the entire database would only be a small
> % size of the sstable.
> Since the row tombstones are so compact, that's why I think they are good
> candidates for special handling.
> On Tue, Feb 13, 2018 at 5:22 PM, J. D. Jordan <jeremiah.jor...@gmail.com>
>> Have you taken a look at the new stuff introduced by
>> https://issues.apache.org/jira/browse/CASSANDRA-7019 ? I think it may
>> go a ways to reducing the need for something complicated like this.
>> Though it is an interesting idea as special handling for bulk deletes.
>> If they were truly just sstables that only contained deletes the logic from
>> 7109 would probably go a long ways. Though if you are bulk inserting
>> deletes that is what you would end up with, so maybe it already works.
>> > On Feb 13, 2018, at 6:04 PM, Jeff Jirsa <jji...@gmail.com> wrote:
>> > On Tue, Feb 13, 2018 at 2:38 PM, Carl Mueller <
>> > wrote:
>> >> In process of doing my second major data purge from a cassandra system.
>> >> Almost all of my purging is done via row tombstones. While performing
>> >> the second time while trying to cajole compaction to occur (in 2.1.x,
>> >> LevelledCompaction) to goddamn actually compact the data, I've been
>> >> thinking as to why there isn't a separate set of sstable infrastructure
>> >> setup for row deletion tombstones.
>> >> I'm imagining that row tombstones are written to separate sstables than
>> >> mainline data updates/appends and range/column tombstones.
>> >> By writing them to separate sstables, the compaction systems can
>> >> preferentially merge / process them when compacting sstables.
>> >> This would create an additional sstable for lookup in the bloom
>> >> granted. I had visions of short circuiting the lookups to other
>> sstables if
>> >> a row tombstone was present in one of the special row tombstone
>> > All of the above sounds really interesting to me, but I suspect it's a
>> > of work to make it happen correctly.
>> > You'd almost end up with 2 sets of logs for the LSM - a tombstone
>> > log/generation, and a data log/generation, and the tombstone logs would
>> > read-only inputs to data compactions.
>> >> But that would only be possible if there was the notion of a "super row
>> >> tombstone" that permanently deleted a rowkey and all future writes
>> would be
>> >> invalidated. Kind of like how a tombstone with a mistakenly huge
>> >> becomes a sneaky permanent tombstone, but intended. There could be a
>> >> special operation / statement to undo this permanent tombstone, and
>> >> the row tombstones would be in their own dedicated sstables, they could
>> >> process and compact more quickly, with prioritization by the compactor.
>> > This part sounds way less interesting to me (other than the fact you can
>> > already do this with a timestamp in the future, but it'll gc away at
>> >> I'm thinking there must be something I am forgetting in the
>> >> read/write/compaction paths that invalidate this.
>> > There are a lot of places where we do "smart" things to make sure we
>> > accidentally resurrect data. Read path includes old sstables for
>> > for example. Those all need to be concretely identified and handled (and
>> > tested),.