I would definitely be interested in this. Clint On Mar 15, 2016 9:36 PM, "Eric Stevens" <migh...@gmail.com> wrote:
> We have been working on filtering compaction for a month or so (though we > call it deleting compaction, its implementation is as a filtering > compaction strategy). The feature is nearing completion, and we have used > it successfully in a limited production capacity against DSE 4.8 series. > > Our use case is that our records are written anywhere between a month, up > to several years before they are scheduled for deletion. Tombstones are > too expensive, as we have tables with hundreds of billions of rows. In > addition, traditional TTLs don't work for us because our customers are > permitted to change their retention policy such that already-written > records should not be deleted if they increase their retention after the > record was written (or vice versa). > > We can clean up data more cheaply and more quickly with filtered > compaction than with tombstones and traditional compaction. Our > implementation is a wrapper compaction strategy for another underlying > strategy, so that you can have the characteristics of whichever strategy > makes sense in terms of managing your SSTables, while interceding and > removing records during compaction (including cleaning up secondary > indexes) that otherwise would have survived into the new SSTable. > > We are hoping to contribute it back to the community, so if you'd be > interested in helping test it out, I'd love to hear from you. > > On Sat, Mar 12, 2016 at 5:12 AM Marcus Eriksson <krum...@gmail.com> wrote: > >> We don't have anything like that, do you have a specific use case in mind? >> >> Could you create a JIRA ticket and we can discuss there? >> >> /Marcus >> >> On Sat, Mar 12, 2016 at 7:05 AM, Dikang Gu <dikan...@gmail.com> wrote: >> >>> Hello there, >>> >>> RocksDB has the feature called "Compaction Filter" to allow application >>> to modify/delete a key-value during the background compaction. >>> https://github.com/facebook/rocksdb/blob/v4.1/include/rocksdb/options.h#L201-L226 >>> >>> I'm wondering is there a plan/value to add this into C* as well? Or is >>> there already a similar thing in C*? >>> >>> Thanks >>> >>> -- >>> Dikang >>> >>> >>
