[
https://issues.apache.org/jira/browse/CASSANDRA-18998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17805802#comment-17805802
]
Josh McKenzie commented on CASSANDRA-18998:
-------------------------------------------
I can speak to some of the thinking "Back in the Day".
The heap and compute pressure of doing CDC into another table (ser/deser, LSM
tree maintenance, write amplification, compaction, etc) would make the solution
pretty unattractive to most use-cases. Further, leaning on coordinator-level
push of a mutation after write opens up pandora's box w/replica failure and
orphaned data in edge cases (i.e. you get write acks from 3 replicas, all 3
die, but you've acked the CDC data downstream and the source data that prompted
it is now lost w/out any meaningful way to reconcile that you now have orphaned
data in your downstream system - very similar problem to what we've run across
with materialized views...)
Having the sidecar parse local files, a separate CDC-specific file from the
CommitLog w/CDC only information, and deduping before feeding downstream would
all be pretty viable things to do from the sidecar space IMO.
> Expose CDC via CQL
> ------------------
>
> Key: CASSANDRA-18998
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18998
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Change Data Capture
> Reporter: Stefen Hillman
> Assignee: Dinesh Joshi
> Priority: Normal
>
> It looks like there was originally a plan to expose change data externally
> via CQL for any client to consume, as noted under Design Options #4 on the
> original design document:
> [https://docs.google.com/document/d/1ZxCWYkeZTquxsvf5hdPc0fiUnUHna8POvgt6TIzML4Y/edit?usp=sharing.]
>
> It appears that this was ultimately not implemented. Is there a plan to
> improve CDC in the future with this functionality? The current implementation
> is tedious to use - you have to write a local agent that lives on every
> single c* node and then implement the Apache-provided CommitLogReader in
> order to parse the files. On top of that, it makes deduplication and ordering
> basically impossible if there isn't an inherently ordered property.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
