Also we have 2.1.x and 2.2 clusters, so we can't use CDC since apparently
that is a 3.8 feature.
Virtual tables are very exciting so we could do some collating stuff (which
I'd LOVE to do with our scheduling application where we can split tasks
into near term/most frequent(hours to days),
new DC and then split is one way, but you have to wait for it to stream,
and then how do you know the dc coherence is good enough to switch the
targetted DC for local_quorum? And then once we split it we'd have downtime
to "change the name" and other work that would distinguish it from the
It reminds me of “shadow writes” described in [1].
During data migration the coordinator forwards a copy of any write request
regarding tokens that are being transferred to the new node.
[1] Incremental Elasticity for NoSQL Data Stores, SRDS’17,
https://ieeexplore.ieee.org/document/8069080
Trigger based has worked for us in the past to get once only output of what’s
happened - pushing this to Kafka and using Kafka Connect allowed to then direct
the stream to to other endpoints.
CDC based streaming has the issue of duplicates which are technically fine if
you don’t care that much
I might be missing something but we’ve done this operation on a few
occasions by:
1) Commission the new cluster and join it to the existing cluster as a 2nd
DC
2) Replicate just the keyspace that you want to move to the 2nd DC
3) Make app changes to read moved tables from 2nd DC
4) Change keyspace
Could be done with CDC
Could be done with triggers
(Could be done with vtables — double writes or double reads — if they were
extended to be user facing)
Would be very hard to generalize properly, especially handling failure cases
(write succeeds in one cluster/table but not the other) which
Isn't this what CDC was designed for?
https://issues.apache.org/jira/browse/CASSANDRA-8844
On Thu, Oct 18, 2018 at 10:54 AM Carl Mueller
wrote:
> tl;dr: a generic trigger on TABLES that will mirror all writes to
> facilitate data migrations between clusters or systems. What is necessary
> to
Thanks. Well, at a minimum I'll probably start writing something soon for
trigger-based write mirroring, and we will probably support kafka and
another cassandra cluster, so if those seem to work I will contribute
those.
On Thu, Oct 18, 2018 at 11:27 AM Jeff Jirsa wrote:
> The write sampling is
The write sampling is adding an extra instance with the same schema to test
things like yaml params or compaction without impacting reads or correctness -
it’s different than what you describe
--
Jeff Jirsa
> On Oct 18, 2018, at 5:57 PM, Carl Mueller
> wrote:
>
> I guess there is also
I guess there is also write-survey-mode from cass 1.1:
https://issues.apache.org/jira/browse/CASSANDRA-3452
Were triggers intended to supersede this capability? I can't find a lot of
"user level" info on it.
On Thu, Oct 18, 2018 at 10:53 AM Carl Mueller
wrote:
> tl;dr: a generic trigger on
tl;dr: a generic trigger on TABLES that will mirror all writes to
facilitate data migrations between clusters or systems. What is necessary
to ensure full write mirroring/coherency?
When cassandra clusters have several "apps" aka keyspaces serving
applications colocated on them, but the
11 matches
Mail list logo
