We use TWCS for a task history table (partition is user, column key is
timeuuid of task, TWCS is used due to tombstone TTLs that rotate out the
tasks every say month. )

However, if we want to get a "slice" of tasks (say, tasks in the last two
days and we are using TWCS sstable blocks of 12 hours).

The problem is, this is a frequent user and they have tasks in ALL the
sstables that are organized by the TWCS into time-bucketed sstables.

So Cassandra has to first read in, say 80 sstables to reconstruct the row,
THEN it can exclude/slice on the column key.


Or am I wrong that the read path needs to grab all relevant sstables before
applying column key slicing and this is possible? Admittedly we are in 2.1
for this table (we in the process of upgrading now that we have an
automated upgrading program that seems to work pretty well)

If my assumption is correct, then the compaction strategy knows as it
writes the sstables what it is bucketing them as (and could encode in
sstable metadata?). If my assumption about slicing is that the whole row
needs reconstruction, if we had a perfect infinite monkey coding team that
could generate whatever we wanted within some feasibility, could we provide
special hooks to do sstable exclusion based on metadata if we know that
that the metadata will indicate exclusion/inclusion of columns based on


The overall goal would be to support exclusion of sstables from a read
path, in case we had compaction strategies hand-tailored for other queries.
Essentially we would be doing a first-pass bucketsort exclusion with the
sstable metadata marking the buckets. This might aid support of superwide
rows and paging through column keys if we allowed the table creator to
specify bucketing as flushing occurs. In general it appears query
performance quickly degrades based on # sstables required for a lookup.

I still don't know the code nearly well enough to do patches, it would seem
based on my looking at custom compaction strategies and the basic read path
that this would be a useful extension for advanced users.

The fallback would be a set of tables to serve as buckets and we span the
buckets with queries when one bucket runs out. The tables rotate.

Reply via email to