Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread Jonathan Ellis
I'm down for adding JOIN support within a partition, eventually. I can see a lot of stuff I'd rather prioritize higher in the short term though. On Fri, May 1, 2015 at 8:44 AM, Jonathan Haddad j...@jonhaddad.com wrote: I think what Benedict has described feels very much like a very specialized

Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread Benedict Elliott Smith
It also doesn't solve the atomicity problem, which is its own challenge. We would probably need to merge the memtables for the entire keyspace/node, and split them out into their own sstables on flush. Or introduce mutual exclusion at the partition key level for the node. On Fri, May 1, 2015 at

Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread graham sanderson
Naively (I may be missing something) it seems much easier to flush a single memtable to more than one stable on disk (static and non static) and then allow for separate compaction of those On May 1, 2015, at 9:06 AM, Benedict Elliott Smith belliottsm...@datastax.com wrote: It also doesn't

Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread Benedict Elliott Smith
How would it be different from creating an actual real extra table instead? There's nothing that warrants making the codebase more complex to accomplish something it already does. As far as I was aware, the only point of static columns was to support the thrift ability to mutate and read

Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread graham sanderson
I 100% agree with Benedict, but just to be clear about my use case 1) We have state of lets say real estate listings 2) We get field level deltas for them 3) Previously we would store the base state all the deltas in partition and roll them up from the beginning of time (this was a prototype and

Re: DateTieredCompactionStrategy and static columns

2015-05-01 Thread Jonathan Haddad
I think what Benedict has described feels very much like a very specialized version of the following: 1. Updates to different tables in a batch become atomic if the node is a replica for the partition 2. Supporting Inner joins if the partition key is the same in both tables. I'd rather see join