Assuming that the one node doesn't have native_transport_max_negotiable_protocol_version=3 in cassandra.yaml, you could check its log for "Detected peers which do not fully support protocol V4. Capping max negotiable version to V3".
The details are in CASSANDRA-15193, but tl;dr is that a serialisation bug affecting paging in mixed version clusters means that it was/is not ideal to support V4 in a cluster containing both 2.x and 3.x nodes. Each 3.x node determines the max protocol version it should support based on the advertised versions of its peers. It's possible that the affected node missed an update regarding one of its peers and so is incorrectly enforcing the cap. If that is the case then restarting that node should prompt it to reevaluate the cap. > On 29 Jul 2021, at 07:54, Erick Ramirez <erick.rami...@datastax.com> wrote: > > Thanks, Pekka. But we know from an earlier post from Srinivas that the driver > is trying to negotiate with v4 but the node wouldn't: > > [2021-07-09 23:26:52.382 -0700] <cluster1-nio-worker-1> > com.datastax.driver.core.Connection - DEBUG: Got unsupported protocol version > error from /<ip>:<port> for version V4 server supports version V3 > [2021-07-09 23:26:52.382 -0700] <cluster1-nio-worker-1> > com.datastax.driver.core.Connection - DEBUG: Connection[//<ip>:<port> -1, > inFlight=0, closed=true] closing connection > [2021-07-09 23:26:52.382 -0700] <cluster1-nio-worker-1> > com.datastax.driver.core.Host.STATES - DEBUG: [//<ip>:<port>] > Connection[/10.39.38.166:9042-1, inFlight=0, closed=true] closed, remaining = > 0 > [2021-07-09 23:26:52.383 -0700] <main> com.datastax.driver.core.Cluster - > DEBUG: Cannot connect with protocol V4, trying V3 > > So we know it's just the one problematic node in the cluster which won't > negotiate. The SHOW VERSION in cqlsh also indicates v3 but I can't figure out > what could be triggering it. Cheers!
