[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565212#comment-17565212
]
Andrew Prudhomme commented on CASSANDRA-15210:
--
[~e.dimitrova] Thank you for the ping. I'll carve out some time to work on this.
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
> Fix For: 3.11.x, 4.0.x, 4.x
>
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564704#comment-17564704
]
Ekaterina Dimitrova commented on CASSANDRA-15210:
-
Added fix versions as per the comments - 3.11.x, 4.0.x and 4.x.
[~aprudhomme], I was wondering whether you are still planning to finish it or
we should un-assign it to signal people that someone can take it over and cross
the finish line?
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
> Fix For: 3.11.x, 4.0.x, 4.x
>
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380437#comment-17380437
]
Benjamin Lerer commented on CASSANDRA-15210:
Thank you [~aprudhomme]. :-) Sorry for the delay, your patch fall through the
cracks. We are working on avoiding this kind of problem in the future.
I will put your patch back in progress so that it reflects more accurately its
status.
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380142#comment-17380142
]
Andrew Prudhomme commented on CASSANDRA-15210:
--
Thank you for looking at this. It has been quite a while and I will need to
rebuild some context, but since it doesn't seem urgent I would be happy to take
a shot at finishing this.
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379783#comment-17379783
]
Benjamin Lerer commented on CASSANDRA-15210:
{quote}Despite the problem exists, it does not affect anything rather than
performance (with unnecessary writing to commit log for CDC table on non-CDC
nodes). I feel it is not so critical to make it mandatory for 4.0 release.
WDYT?{quote}
It is not mandatory for 4.0.0 as it also affect 3.11 but it should be added to
a 4.0.X patch release.
{quote}Even though the patch looks good, there are no unit tests for the
change. Moreover, currently the affected class is not covered by tests. I trier
to find a simple reference test, but looks like streaming tests are quite
complicated. Do we need to have a test developed to get these changes merged?
Even though it is 1 line patch, my belief that having a test is mandatory.
Could you please share a reference test that can be used to cover CDC
functionality in CassandraStreamReceiver?{quote}
For streaming we have several distributed tests. What I would suggest for this
patch is an in-jvm DTest (see
org.apache.cassandra.distributed.testStreamingTest for an example) that check
both scenario with CDC enabled and disabled. It should be possible to check the
effect of the configuration parameter by looking at the cdc directory.
[~aprudhomme] Are you interested in finishing your patch or should we reasign
it ?
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379460#comment-17379460
]
Aleksei Zotov commented on CASSANDRA-15210:
---
[~blerer]
I briefly read about CDC feature in Cassandra and this ticket makes sense to
me. As per my understanding, non-CDC node should not care about CDC tables and
enforce writing corresponding mutations to commit log. The patch also looks
good to me. Two questions to you:
# Despite the problem exists, it does not affect anything rather than
performance (with unnecessary writing to commit log for CDC table on non-CDC
nodes). I feel it is not so critical to make it mandatory for 4.0 release. WDYT?
# Even though the patch looks good, there are no unit tests for the change.
Moreover, currently the affected class is not covered by tests. I trier to find
a simple reference test, but looks like streaming tests are quite complicated.
Do we need to have a test developed to get these changes merged? Even though it
is 1 line patch, my belief that having a test is mandatory. Could you please
share a reference test that can be used to cover CDC functionality in
{{CassandraStreamReceiver}}?
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004427#comment-17004427
]
Andrew Prudhomme commented on CASSANDRA-15210:
--
Thanks, I'm still learning this workflow.
Since it has been a while, I did a rebase and retest. It doesn't look like much
has changed.
||Branch||Tests||Local dtest||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aprudhomme:15210-trunk]|[cci|https://circleci.com/workflow-run/11fbfb4e-fa3d-4dda-b08e-6850a539a335]|[pb|https://pastebin.com/dN6cpH9C]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...aprudhomme:15210-3.11]|[cci|https://circleci.com/workflow-run/c378d858-0ca1-4d11-aa95-50cabfef6897]|[pb|https://pastebin.com/U5Mqy4WY]|
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16994963#comment-16994963
]
Jeremiah Jordan commented on CASSANDRA-15210:
-
[~aprudhomme] if your patch is ready for review, then hit the "submit patch"
button above to let people know that.
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905662#comment-16905662
]
Andrew Prudhomme commented on CASSANDRA-15210:
--
For some more context, this is causing us problems in the case where a CDC
tracked table has large partitions. Since the streaming bootstrap is played
through the commit log, the stream will fail because of the (0.5 * commit log)
mutation size limit. This issue means that streaming will fail even when CDC is
disabled at the node level.
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled
[
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905658#comment-16905658
]
Andrew Prudhomme commented on CASSANDRA-15210:
--
||Branch||Tests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aprudhomme:15210-trunk]|[cci|https://circleci.com/workflow-run/184e2de1-8893-481a-91f1-910ed4ac246a]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...aprudhomme:15210-3.11]|[cci|https://circleci.com/workflow-run/f5fcba5a-e940-4cc7-b02d-540d8887c258]|
I did not have the circleci resources for dtests, so I ran them locally.
[trunk|https://pastebin.com/8aLbh7GF] - The 3 failed tests passed on retry. The
rebuild_test error also occurred on the base branch.
[3.11|https://pastebin.com/bE9GBCm8] - The counter_test, largecolumn_test,
offline_tools_test, and replace_address_test failures also occurred on the base
branch. All other failures passed on retry.
> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed
> through the write path to ensure they are made available through the commit
> log. However, currently only the CDC state of the table is checked. Since CDC
> is enabled at both the node and table level, a node with CDC disabled (with
> cdc_enabled: false) will unnecessarily send updates through the write path if
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>
> {code:java}
> - hasCDC = cfs.metadata.params.cdc;
> + hasCDC = cfs.metadata.params.cdc &&
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
