Re: SSTable exclusion from read path based on sstable metadata marked by custom compaction strategies

2019-01-31 Thread Jonathan Haddad
In addition to what Jeff mentioned, there was an optimization in 3.4 that can significantly reduce the number of sstables accessed when a LIMIT clause was used. This can be a pretty big win with TWCS.

Re: SSTable exclusion from read path based on sstable metadata marked by custom compaction strategies

2019-01-31 Thread Jeff Jirsa
In my original TWCS talk a few years back, I suggested that people make the partitions match the time window to avoid exactly what you’re describing. I added that to the talk because my first team that used TWCS (the team for which I built TWCS) had a data model not unlike yours, and the

SSTable exclusion from read path based on sstable metadata marked by custom compaction strategies

2019-01-31 Thread Carl Mueller
Situation: We use TWCS for a task history table (partition is user, column key is timeuuid of task, TWCS is used due to tombstone TTLs that rotate out the tasks every say month. ) However, if we want to get a "slice" of tasks (say, tasks in the last two days and we are using TWCS sstable blocks