[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17791001#comment-17791001 ] Stefan Miklosovic commented on CASSANDRA-18935: --- I built the patch for trunk as TCM has happened there (1). Even the build contains a lot of failures, this is all known and tracked and I do not see anything wrong with the patch itself. Also, for the reference, test_counter_leader_with_partial_view test is NOT flaky. I run the multiplexer and it all just passes fine. So I had to introduce the instability in my attempt to bring in the changes in located in CASSANDRA-19103. Anyway, as already communicated, we go without that. I am starting the merging process. (1) [https://app.circleci.com/pipelines/github/instaclustr/cassandra/3572/workflows/70ea8bc0-08c3-472c-a292-a1c050bc65dd] > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790733#comment-17790733 ] Stefan Miklosovic commented on CASSANDRA-18935: --- Good news is that I have a non-flaky test in CASSANDRA-19103 implemented. Maybe we could rewrite what is in dtest to in-jvm dtest when it comes to test_counter_leader_with_partial_view . I started the multiplexer on current 5.0 to see if it is flaky already or I made it like that. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790730#comment-17790730 ] Brandon Williams commented on CASSANDRA-18935: -- That's fine, I'm +1 on that plan. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790722#comment-17790722 ] Stefan Miklosovic commented on CASSANDRA-18935: --- Well ... it is flaky every now and then [https://app.circleci.com/pipelines/github/instaclustr/cassandra/3569/workflows/363967bb-7390-4aef-9dd1-b2850e3790d9/jobs/158433] I just dont have time for this for now, I will commit just the revert and we will deal with 19103 separately in trunk afterwards. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790581#comment-17790581 ] Stefan Miklosovic commented on CASSANDRA-18935: --- It seems that the byteman script we inject into node 1 and 3 in test_counter_leader_with_partial does not work. If you look closely, the rule is trying to intercept a method called "findSuitableEndpoint" (1) but there is not such method in 4.0 included. So how does this work then, huh ... findSuitableEndpoint is present only in 3.0 and 3.11. For these versions, there is this byteman injected (2). So, it seems like it does its job for 3.0 and 3.11 only. For 4.0 included it does not inject anything. The question is, if it is not injected, how is it possible that the test still passes anyway? Well, if it is not injected, then it goes by the logic currently present in 4.0+, which is - replica which is going to be a leader will be only a node for which RPC is true and it is not marked as dead by FailureDetector. So it seems like when we stop the second node in (3), two remaining nodes will detect this and they will never send any mutation to that node because it is filtered out and byteman script which forces to select it if it is not filtered out does not work anyway. So the node with partial view of the cluster is never selected. I think we need to do this to fix the tests everywhere [https://github.com/apache/cassandra-dtest/pull/246] (this will got together with 19103) The byteman injection after we stopped the node is not necessary in trunk because there is TCM machinery in place. (1) [https://github.com/apache/cassandra-dtest/blob/trunk/byteman/4.0/election_counter_leader_favor_node2.btm#L8] (2) [https://github.com/apache/cassandra-dtest/blob/trunk/byteman/pre4.0/election_counter_leader_favor_node2.btm#L8] (3) [https://github.com/apache/cassandra-dtest/blob/trunk/counter_test.py#L117-L119] (3) [https://github.com/apache/cassandra-dtest/blob/trunk/counter_test.py#L110-L132] (4) [https://github.com/apache/cassandra-dtest/blob/trunk/byteman/4.0/election_counter_leader_favor_node2.btm#L10] (5) [https://github.com/apache/cassandra-dtest/blob/trunk/counter_test.py#L117] > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790416#comment-17790416 ] Stefan Miklosovic commented on CASSANDRA-18935: --- Well, I see in trunk that test_counter_leader_with_partial_view now fails. I am trying to figure out why and fix it. https://app.circleci.com/pipelines/github/instaclustr/cassandra/3562/workflows/3aeab43a-1b13-4b93-a6a9-c6c3cca0e9ca/jobs/157646/tests#failed-test-1 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790177#comment-17790177 ] Brandon Williams commented on CASSANDRA-18935: -- I see, thanks. NativeProtocolTest will cover it in Stefan's patch, so I think we good to go here. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790174#comment-17790174 ] Runtian Liu commented on CASSANDRA-18935: - [~brandon.williams] the reason why test_counter_leader_with_partial_view didn't fail before and still does not fail currently is because "nodetool disablebinary" is not called in the test. The test is only restarting a node and delay update status of other nodes' views for that node. I would say the test is currently useful but it won't cover the case when modifying binary port with nodetool commands. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790161#comment-17790161 ] Stefan Miklosovic commented on CASSANDRA-18935: --- I created this for trunk CASSANDRA-19103 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790115#comment-17790115 ] Brandon Williams commented on CASSANDRA-18935: -- I am +1 on this. I think we should figure out why counter_leader_with_partial_view_test didn't fail (and still does not currently) when we set rpc_ready to false, otherwise it's not very useful. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789182#comment-17789182 ] Stefan Miklosovic commented on CASSANDRA-18935: --- [~mck] we are trying to fix the regression here after original 18935 was committed. Mentioning that in case you will want to move this out from 5.0.0 scope. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17789177#comment-17789177 ] Stefan Miklosovic commented on CASSANDRA-18935: --- [3.0|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3542/workflows/f44d30c4-146d-4183-87ad-2fc9ced5fe9d] [3.11|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3543/workflows/6ede27da-a6ef-4f80-846d-fd4de7292329] [4.0|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3544/workflows/c649e450-6334-4131-bfa3-a229c6dd6447] [4.1|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3545/workflows/0d27e44f-771e-4990-b3d3-3c922ca29521] [5.0|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3546/workflows/fa7e792a-ca7a-4ab7-a5d0-d1f824b76504] restarted j17 dtest for 5.0 [here|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3552/workflows/f5b5824e-95b5-40da-a43e-21017cf40928/jobs/156678] [trunk|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3547/workflows/7b99303f-08b9-45f1-a5b9-70107bbaa1c3] > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788499#comment-17788499 ] Runtian Liu commented on CASSANDRA-18935: - I think the fix looks good. It will work for us. Thanks. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788148#comment-17788148 ] Stefan Miklosovic commented on CASSANDRA-18935: --- RPC_READY is set locally _after_ native transport is started (ports are open, it is fully initialized and it listens to connections) and then it is propagated by Gossip and it will eventually reach other nodes. So by the time another node sees RPC_READY is "true" for such and such node, it is basically guaranteed that it listens to connections. (if the node which started transport service does not stop it by the time RPC_READY = true propagates in Gossip). I am not sure what is the correct solution but the current one feels like a hack. If RPC_READY is defacto irrelevant what it is set to, then we can change it to something else, questionable to what. However, this is problematic only for a corner cases like storage node where RPC is turned off. It is quite a bummer that the current code does not consider that scenario. [~curlylrt] Why is it so problematic for you to have RPC started even on data nodes? You can still firewall / blacklist them etc if you do not wish the clients to connect to it, no? This is patch for 3.0 I want to get in (1), I will continue on all branches soon and I ll provide the builds. (1) https://github.com/apache/cassandra/pull/2923/files > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2.5h > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787520#comment-17787520 ] Runtian Liu commented on CASSANDRA-18935: - I concur that with CEP 21, the issue should no longer persist. Yet, what approach should we take for versions 3.0.x to 4.1.x? My inquiry pertains to the precise meaning of RPC_READY. Does RPC_READY equate to binary being enabled? Or does it simply indicate that the node is prepared to enable the binary port for serving client traffic? If RPC_READY merely signifies readiness without guaranteeing the binary port's activation, it might be better to switch this flag to true only after gossip has stabilized, regardless of the binary port's status. Initially, we encountered a problem when attempting to initiate every node with "start_native_transport": false in cassandra.yaml, intending to activate the binary port via JMX short after the node is status NORMAL and stable. This led to a significant outage, with all counter mutations failing because the RPC_READY flag was absent on all nodes. This flag is only set to true when "start_native_transport" is true. The solution proposed in this ticket appears to resolve this issue, as enabling the binary port will trigger the RPC_READY flag to true. However, we also observed that disabling the binary port sets this flag to false. This could pose a problem for us, as we occasionally need to disable nodes for various reasons, which could obstruct counter updates. Should we consider separating the RPC_READY status from the actual state of the binary port? For example, for the setup of dedicated coordinator nodes, the storage nodes are started with "start_native_transport": false and never get binary enabled, the "RPC_READY" flag will never be true. Then, counter update for certain partitions will always fail because of the UAE. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787497#comment-17787497 ] Stefan Miklosovic commented on CASSANDRA-18935: --- I think we should just remove that one line and that's it, for the sake of not having this regression in 5.0.0. We will wait until CEP-21 (TCM) is in place, should be pretty soon. If you check TCM logic in that method, it was rewritten (1) so it does not make sense to introduce new gossip message etc, this stuff is apparently going to be handled differently quite soon. After TCM changes are in, I think we can finally decouple this rpc readiness from counter's logic and then we can fix enable/disable binary behaviour when it comes to RPC status. (1) https://github.com/apache/cassandra/blob/cep-21-tcm/src/java/org/apache/cassandra/locator/ReplicaPlans.java#L212-L221 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png, > signature.asc > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787463#comment-17787463 ] Runtian Liu commented on CASSANDRA-18935: - It will be easier to understand If you check the added [test.|https://github.com/ostefano/cassandra-dtest/commit/a1e505977a6e8f04f85d053ce1f8bb83e7b04d21] Let me explain the "full view of the cluster": We have a 3 node cluster, with a keyspace as replication factor 3. Now we restart node A. After node A restart, it will first set its own status to NORMAL then gossip this info to the rest of the cluster, Node B received the message. Before other nodes send any gossip info to node A, the endpointStateMap of node A is empty. Let's say some how the gossip info sending to node A got delayed. Now, node B received a request to do counter mutation. Node B selects node A as a leader of the counter mutation because Node A is normal, so Node B send the mutation to A. From A's point of view, the other two nodes are down nodes. The only live node is Node A itself. If consistency level is local_quorum, node A will just throw the UAE. To solve the issue, CASSANDRA-13043 made the change to wait for the last flag "RPC_READY" to be true. When RPC_READY is true, A has the "full view of the cluster" (Node B and C is UP and normal). {panel:title=Below are logs related to gossip when a node getting started:} {"@timestamp":"2023-11-18T08:03:21.358+00:00","@version":1,"message":"Starting up server gossip","logger_name":"o.a.c.service.StorageService","thread_name":"main","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.454+00:00","@version":1,"message":"Updating topology for /node1.ip.address:9042","logger_name":"o.a.c.locator.TokenMetadata","thread_name":"main","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.455+00:00","@version":1,"message":"Updating topology for /node1.ip.address:9042","logger_name":"o.a.c.locator.TokenMetadata","thread_name":"main","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.660+00:00","@version":1,"message":"Node /node2.ip.address:9042 is now part of the cluster","logger_name":"o.a.cassandra.gms.Gossiper","thread_name":"GossipStage:1","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.668+00:00","@version":1,"message":"Using saved tokens [-2221857722054656123, -3885079421067140903, -5343119608664883471, -6713057863481554638, -704218406783704283, -8255104289521151109, 1443620187740186167, 2237424365810614485, 3011306438350676741, 3823627889871018402, 4717722679840880909, 491446797270294465, 5682153714635827559, 6562206662586337529, 747920626402974, 8684646769397141191]","logger_name":"o.a.c.service.StorageService","thread_name":"main","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.688+00:00","@version":1,"message":"JOINING: Finish joining ring","logger_name":"o.a.c.service.StorageService","thread_name":"main","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.700+00:00","@version":1,"message":"Updating topology for /node2.ip.address:9042","logger_name":"o.a.c.locator.TokenMetadata","thread_name":"GossipStage:1","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.701+00:00","@version":1,"message":"Updating topology for /node2.ip.address:9042","logger_name":"o.a.c.locator.TokenMetadata","thread_name":"GossipStage:1","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.707+00:00","@version":1,"message":"Node /node1.ip.address:9042 state jump to NORMAL","logger_name":"o.a.c.service.StorageService","thread_name":"main","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.726+00:00","@version":1,"message":"Node /node1.ip.address:9042 state jump to NORMAL","logger_name":"o.a.c.service.StorageService","thread_name":"main","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.730+00:00","@version":1,"message":"Node /node3.ip.address:9042 has restarted, now UP","logger_name":"o.a.cassandra.gms.Gossiper","thread_name":"GossipStage:1","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.731+00:00","@version":1,"message":"Node /node3.ip.address:9042 state jump to NORMAL","logger_name":"o.a.c.service.StorageService","thread_name":"GossipStage:1","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.826+00:00","@version":1,"message":"Updating topology for /node3.ip.address:9042","logger_name":"o.a.c.locator.TokenMetadata","thread_name":"GossipStage:1","level":"INFO","level_value":2} \{"@timestamp":"2023-11-18T08:03:21.827+00:00","@version":1,"message":"Updating topology for /node3.ip.address:9042","logger_name":"o.a.c.locator.TokenMetadata","thread_name":"GossipStage:1","level":"INFO","level_value":2}
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787427#comment-17787427 ] Runtian Liu commented on CASSANDRA-18935: - The test won't pass if we change to check if the status of the node is normal. I think we cannot use status NORMAL to filter the leader node for counter mutation. I think the extra requirement for counter mutation leader here is to make sure the leader has full view of the cluster. But the binary port of the leader doesn't have to be enabled. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787425#comment-17787425 ] Brandon Williams commented on CASSANDRA-18935: -- bq. it will choose replicas which are NORMAL but their transports are not started yet. Which is inconsequential since nothing needs to use them. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787423#comment-17787423 ] Stefan Miklosovic commented on CASSANDRA-18935: --- btw when I read the code correctly, NORMAL value of STATUS application state will be propagated by Gossip before transports are started and RPC_READY value is set to true. So it may happen that for brief period of time, until RPC_READY is true, it will choose replicas which are NORMAL but their transports are not started yet. If NORMAL status would be set after RPC_READY was set and counters wait for NORMAL, there is a guarantee that if it is RPC_READY so queries would not fail. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787412#comment-17787412 ] Brandon Williams commented on CASSANDRA-18935: -- We should probably make a new ticket to change the behavior and just revert the one line here to avoid a regression. I'm not sure why that test didn't fail here though. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787408#comment-17787408 ] Runtian Liu commented on CASSANDRA-18935: - If that's the case, does it mean if a node's status is NORMAL, it means the node should have the full view of the cluster? Then we should change to use status to filter. Also, we have added [test|https://github.com/ostefano/cassandra-dtest/commit/a1e505977a6e8f04f85d053ce1f8bb83e7b04d21] earlier, let me check if changing the filter to use status will work with that test. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787404#comment-17787404 ] Brandon Williams commented on CASSANDRA-18935: -- bq. the gossip may have some delay so it may only have the partial view of the cluster. Gossip has mechanisms to avoid this scenario. If it was a problem then rolling restarts would break all kinds of queries all the time, nothing specific to counters. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787400#comment-17787400 ] Runtian Liu commented on CASSANDRA-18935: - [~brandon.williams] , I think the problem of "STATUS" is that when a node started, it will broadcast the NORMAL status to peers. For this just started node, the gossip may have some delay so it may only have the partial view of the cluster. This means the node will consider part of the cluster is down. What we really want in counter mutation is to make sure the leader recent started node(or a new node) is fully connected to the cluster so it has full view of the cluster. The reason we chose RPC_READY might be that this flag was set to true only when everything is done. But this flag is indicating if binary port is enabled, I don't think we should use this flag. I see 4.0 has added a checker before enabling binary port: [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L644] I think it's better to create a new flag like "FULLY_CONNECTED" so that we know the node has a full view of the cluster. Then we can use this flag in counter mutation and keep "RPC_READY" like what we have today: 1. set to true when binary port enabled. 2. set to false when binary port disabled. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787338#comment-17787338 ] Brandon Williams commented on CASSANDRA-18935: -- I think we should probably also add a test for counters in this situation so nobody accidentally "fixes" it again. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787336#comment-17787336 ] Brandon Williams commented on CASSANDRA-18935: -- We can also switch to checking STATUS here, and then someday we can restore RPC_READY to accuracy. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787332#comment-17787332 ] Stefan Miklosovic commented on CASSANDRA-18935: --- I will remove that one line [~curlylrt] showed. Too bad there is no simple fix. This will leak e.g. to nodetool gossipinfo where RPC status is visible so it may be misleading. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787322#comment-17787322 ] Brandon Williams commented on CASSANDRA-18935: -- We can, but all existing installations are still going to want RPC_READY, so it's not so simple to unring this bell. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787314#comment-17787314 ] Stefan Miklosovic commented on CASSANDRA-18935: --- OK so if they just want that flag, and it is irrelevant whether it is in fact enabled or not, then why could not we replace that with filter on STATUS being NORMAL. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787305#comment-17787305 ] Brandon Williams commented on CASSANDRA-18935: -- bq. So what is the difference if we set that RPC to false when it is really not enabled? Counter writes will fail. Whether or not RPC is actually enabled has no effect, counters just want the flag. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787304#comment-17787304 ] Stefan Miklosovic commented on CASSANDRA-18935: --- Before this fix, RPC was set to true on startup, then if somebody turned it off, RPC flag was not set to false so that logic around counters always saw RPC of that node up, even if it was not. So what is the difference if we set that RPC to false when it is really not enabled? I think what happens here is that if RPC is set to true even it is really not, then in StorageProxy.mutateCounter it will send message with counter mutation with a response handler which has "null" in "hintsOnFailure" parameter so it means that if that message fails (because RPC is false?) it will not even create hints for such mutation. But the outcome is same? (1) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L1736 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787292#comment-17787292 ] Brandon Williams commented on CASSANDRA-18935: -- The problem is not coordinator-only nodes, but storage nodes that have rpc disabled. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787289#comment-17787289 ] Stefan Miklosovic commented on CASSANDRA-18935: --- Coordinator nodes, these which will never join a ring, will be in STARTING state for ever. So if we filter only nodes with NORMAL status here (1) instead waiting for RPC, then replicas for hints will ever be just "normal" nodes carrying data. So this will not collide with rpc enable/disable as it is now? [~brandon.williams] (1) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L1764 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0-rc, 5.x > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787143#comment-17787143 ] Brandon Williams commented on CASSANDRA-18935: -- We already have a flag to indicate this, STATUS. It seems there could have been some kind of race around this when CASSANDRA-13043 was being worked on that made RPC_READY seem like a better option, but nonetheless it is the de facto way to know whether a node is ready to receive requests, and what the rest of the system uses: counters are special and reinventing that wheel here. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.12, 4.1.4, 5.0-alpha2, 5.0, 5.1 > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787013#comment-17787013 ] Runtian Liu commented on CASSANDRA-18935: - Not sure why we added this RPC_READY flag in the first place. But I think the issue https://issues.apache.org/jira/browse/CASSANDRA-13043 trying to solve is that when doing counter mutation, we do not want the node stilling bootstrapping to be the leader of the counter mutation. The trick we did is to use this flag to make sure the bootstrap is finished. I think we should not use RPC_READY flag to filter the nodes. Maybe we need to add a new flag to indicate if a node is fully bootstrapped? > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.12, 4.1.4, 5.0-alpha2, 5.0, 5.1 > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786976#comment-17786976 ] Stefan Miklosovic commented on CASSANDRA-18935: --- This is a tricky ticket! I just keep thinking about the consequences here from various angles. When you say we should never set it to false, does that hold for coordinator nodes as well? I think it should be reworded like this: never set it to false, _WHEN IT IS NOT A COORDINATOR_, once it has been set to true unless decom/drain. Why wouldnt you set rpc status to false in coordinator nodes? > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.12, 4.1.4, 5.0-alpha2, 5.0, 5.1 > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786968#comment-17786968 ] Stefan Miklosovic commented on CASSANDRA-18935: --- interesting catch about dedicated coordinators! I think we need to cover that scenario too. Why do we even need this line (1) anymore? Why not to filter only based on liveness? If a node is decommissioned or drained, is it still alive? I just dont know from head. (1) https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L1764 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.12, 4.1.4, 5.0-alpha2, 5.0, 5.1 > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786922#comment-17786922 ] Brandon Williams commented on CASSANDRA-18935: -- bq. Does this mean after this change, if a cluster has multiple nodes binary disabled for some reason, for certain partitions, the counter update will always get UnavailableException because all the replica nodes are not "RPC_READY"? I believe it does mean we will throw UE because the 13043 code is still active [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageProxy.java#L1764]. It is rather unfortunate that counters have misused this, forcing us to claim rpc is ready regardless of the actual state, but I think that's what we have to do, never set it to false once it has been set to true unless decom/drain, as you said. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.12, 4.1.4, 5.0-alpha2, 5.0, 5.1 > > Attachments: 18935-3.11.patch, image-2023-11-16-10-56-16-693.png > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786894#comment-17786894 ] Runtian Liu commented on CASSANDRA-18935: - [~smiklosovic] Does this mean after this change, if a cluster has multiple nodes binary disabled for some reason, for certain partitions, the counter update will always get UnavailableException because all the replica nodes are not "RPC_READY"? Why counter update needs to bind with native transport enabled? Also, it is mentioned in https://issues.apache.org/jira/browse/CASSANDRA-13043, "if a cluster have a setup with dedicated coordinator nodes, in which storage nodes don't even bother listening to clients, and have that disabled" counter update will always fail. Quote: " it sounds like the problem we have is that we move endpoints from 'pending' to 'natural' endpoints too quickly after bootstrap, before the node is actually fully ready, and that this is what we should be fixing" With all above being said, I think it's better to # set the RPC_READY to true when enable native transport. (This is already done in the merge PR) # Do not set RPC_READY to false if we are using nodetool to shutdown native transport. We just set the value to false when the node is decommissioned and drained. (This is added in this PR) Thoughts? > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.12, 4.1.4, 5.0-alpha2, 5.0, 5.1 > > Attachments: 18935-3.11.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778746#comment-17778746 ] Stefan Miklosovic commented on CASSANDRA-18935: --- passed https://app.circleci.com/pipelines/github/driftx/cassandra/1346/workflows/b3fec612-e901-4166-a18a-8e1c88128f42/jobs/59243 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17778662#comment-17778662 ] Brandon Williams commented on CASSANDRA-18935: -- [Upgrade tests|https://app.circleci.com/pipelines/github/driftx/cassandra/1346/workflows/b3fec612-e901-4166-a18a-8e1c88128f42]. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1329#comment-1329 ] Brandon Williams commented on CASSANDRA-18935: -- Fair enough, it's a simple patch. The failure in 5.0 is CASSANDRA-18360 and it didn't repro on trunk so nothing to be concerned about there. +1 > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1316#comment-1316 ] Stefan Miklosovic commented on CASSANDRA-18935: --- since it is the very same patch just applied to various branches, I would say that we tested it all from Java 8 to 17 :) > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1315#comment-1315 ] Stefan Miklosovic commented on CASSANDRA-18935: --- [~brandon.williams] would you please took a look and approved? > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1308#comment-1308 ] Brandon Williams commented on CASSANDRA-18935: -- Missing some JDK runs? > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1220#comment-1220 ] Stefan Miklosovic commented on CASSANDRA-18935: --- [trunk j17|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3369/workflows/b1dceb9a-e45d-4b1a-b4db-394f361f2689] [5.0 j17|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3368/workflows/8309cbb1-cd5b-4945-b590-329617e44c92] [4.1 j11|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3367/workflows/8d968041-fd54-4cba-87e2-0d939b7376ca] [4.0 j11|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3366/workflows/e4d5c713-cb48-46e5-979b-fc139a6c0733] [3.11 j8|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3350/workflows/8416bbd5-a736-49ee-aa50-0a98f5c9eee2] [3.0 j8|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3365/workflows/689b2ed4-a32b-4ee7-9b1e-f5bfd53d1d9c] > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[ https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776582#comment-17776582 ] Stefan Miklosovic commented on CASSANDRA-18935: --- I just renamed the ticket. > Fix nodetool enable/disablebinary to correctly set rpc > -- > > Key: CASSANDRA-18935 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18935 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core, Legacy/CQL >Reporter: Cameron Zemek >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x > > Attachments: 18935-3.11.patch > > Time Spent: 10m > Remaining Estimate: 0h > > > {code:java} > if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) || > (nativeFlag == null && DatabaseDescriptor.startNativeTransport())) > { > startNativeTransport(); > StorageService.instance.setRpcReady(true); > } {code} > The startup code here only sets RpcReady if native transport is enabled. If > you call > {code:java} > nodetool enablebinary{code} > then this flag doesn't get set. > But with the change from CASSANDRA-13043 it requires RpcReady set to true in > order to get a leader for the counter update. > Not sure what the correct fix is here, seems to only really use this flag for > counters. So thinking perhaps the fix is to just move this outside the if > condition. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org