[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled

2019-08-12 Thread Andrew Prudhomme (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905662#comment-16905662
 ] 

Andrew Prudhomme commented on CASSANDRA-15210:
--

For some more context, this is causing us problems in the case where a CDC 
tracked table has large partitions. Since the streaming bootstrap is played 
through the commit log, the stream will fail because of the (0.5 * commit log) 
mutation size limit. This issue means that streaming will fail even when CDC is 
disabled at the node level.

> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed 
> through the write path to ensure they are made available through the commit 
> log. However, currently only the CDC state of the table is checked. Since CDC 
> is enabled at both the node and table level, a node with CDC disabled (with 
> cdc_enabled: false) will unnecessarily send updates through the write path if 
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>  
> {code:java}
> -   hasCDC = cfs.metadata.params.cdc;
> +   hasCDC = cfs.metadata.params.cdc && 
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-15210) Streaming with CDC does not honor cdc_enabled

2019-08-12 Thread Andrew Prudhomme (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-15210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905658#comment-16905658
 ] 

Andrew Prudhomme commented on CASSANDRA-15210:
--

||Branch||Tests||
|[trunk|https://github.com/apache/cassandra/compare/trunk...aprudhomme:15210-trunk]|[cci|https://circleci.com/workflow-run/184e2de1-8893-481a-91f1-910ed4ac246a]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...aprudhomme:15210-3.11]|[cci|https://circleci.com/workflow-run/f5fcba5a-e940-4cc7-b02d-540d8887c258]|

I did not have the circleci resources for dtests, so I ran them locally.

[trunk|https://pastebin.com/8aLbh7GF] - The 3 failed tests passed on retry. The 
rebuild_test error also occurred on the base branch.

[3.11|https://pastebin.com/bE9GBCm8] - The counter_test, largecolumn_test, 
offline_tools_test, and replace_address_test failures also occurred on the base 
branch. All other failures passed on retry.

 

> Streaming with CDC does not honor cdc_enabled
> -
>
> Key: CASSANDRA-15210
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15210
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Streaming, Feature/Change Data Capture
>Reporter: Andrew Prudhomme
>Assignee: Andrew Prudhomme
>Priority: Normal
>
> When SSTables are streamed for a CDC enabled table, the updates are processed 
> through the write path to ensure they are made available through the commit 
> log. However, currently only the CDC state of the table is checked. Since CDC 
> is enabled at both the node and table level, a node with CDC disabled (with 
> cdc_enabled: false) will unnecessarily send updates through the write path if 
> CDC is enabled on the table. This seems like an oversight.
> I'd imagine the fix would be something like
>  
> {code:java}
> -   hasCDC = cfs.metadata.params.cdc;
> +   hasCDC = cfs.metadata.params.cdc && 
> DatabaseDescriptor.isCDCEnabled();{code}
> in
> org.apache.cassandra.db.streaming.CassandraStreamReceiver (4)
> org.apache.cassandra.streaming.StreamReceiveTask (3.11)
>  



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org