Late to the party I'm afraid, but I'd agree with Abe's proposal to deprecate the dual port approach given that CASSANDRA-10559 makes it pretty much redundant. Adding further yaml options like "default ssl/no ssl" feels like another nasty band-aid that we'll have to live with for the foreseeable future.
Also, if CASSANDRA-16999 is only going to trunk, why can't we just deprecate dual ports in 5.0 (as it isn't at -rc stage yet) and remove it from trunk? That seems preferable to shoehorning something into the new system_views.peers table, which isn't going to help any existing drivers anyway as none of them will be using it. > On 12 Feb 2024, at 07:58, Štefan Miklošovič <stefan.mikloso...@gmail.com> > wrote: > > I think that the situation like > > client_encryption_options.enabled = true > client_encryption_options.optional=true > native_transport_port != native_transport_port_ssl > > is a legit bug and should be fixed. If we look here (1), when these ports are > not equal, the normal port is explicitly set to be unencrypted but it is > encrypted on _ssl port. This is not always true for _ssl port, because in (2) > we throw only if client_encryption_options' encryption_policy is UNENCRYPTED. > We do not throw if it is OPTIONAL. If we say that the normal port is always > unencrypted, why don't we also say that _ssl port is always encrypted? This > asymmetry should be fixed. > > However, I think it is too late to fix anything but trunk. Adding a column > might break clients and fixing the logic around ports might potentially break > the deployments so our best shot seems to be: > > 1) from 4.0 to trunk - apply a patch which would inform a user that it is > preferable to use single port instead of dual ports > 2) for trunk - apply a patch which adds native_port_ssl column to > system_views.peers so Cassandra Java Driver can connect to such a deployment > 3) optionally fix the bug I was describing above. > > I am not sure how to evaluate the severity of 3). I would like to see it from > 4.0 to trunk but I also understand that if it is too disruptive, we can leave > it just to trunk. > > The problems you described are the result of us using one > client_encryption_options for both ssl and non-ssl ports. I would say that > the first problem you described, even if it looks weird, is less serious, > because a user knowingly uses two ports and one of them is said to be > unencrypted and another one encrypted. So the fact that a non-ssl port still > enables unencrypted traffic is somehow expected. What is not expected is that > _ssl port might still accept unencrypted traffic. > > To sum it up for the driver, I do not think this has a nice solution for dual > ports deployments already out there so it would work just for the trunk. > Actually, because there is missing native_port_ssl in a system table, there > is currently no way to successfully use dual ports because Java driver just > can not connect to SSL-enabled nodes reliably using ssl and non-ssl ports. > > (1) > https://github.com/apache/cassandra/blob/097c1231e2466163fe3f8b36b12cdc5235eb1403/src/java/org/apache/cassandra/service/NativeTransportService.java#L94 > > (2) > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/DatabaseDescriptor.java#L905-L915 > > On Thu, Feb 8, 2024 at 9:58 PM Abe Ratnofsky <a...@aber.io > <mailto:a...@aber.io>> wrote: >> > Deprecating helps nothing for existing releases. We can’t/shouldn’t remove >> > the feature in existing releases. >> >> The deprecation I'm proposing is intended to push people to configure their >> servers in a way that is more secure and maximizes compatibility with >> clients. Deprecating can help for existing releases - the better >> configuration already exists, and it's likely that users of dual-native-port >> optional SSL can use it. At the very least, users should be made aware of >> the risks of dual-native-port operation. >> >> Currently, if a user specifies the following server configuration: >> - client_encryption_options.enabled=true >> - client_encryption_options.optional=false >> - native_transport_port != native_transport_port_ssl >> >> then the server will still handle unencrypted traffic on >> native_transport_port. This feels like a security risk: it would be >> reasonable to interpret that this configuration requires all traffic to be >> encrypted. >> >> And if a user specifies this server configuration: >> - client_encryption_options.enabled=true >> - client_encryption_options.optional=true >> - native_transport_port != native_transport_port_ssl >> >> then clients can still send unencrypted traffic to >> native_transport_port_ssl, since the server handles optional encryption on >> this port. In this case, there are two ports that accept unencrypted >> traffic, one of which also accepts encrypted traffic. >> >> In both cases: >> - Clients configured to use SSL will discover non-SSL ports from >> system.peers_v2 and fail to connect to those hosts, causing >> single-connection sessions and no load balancing >> - Clients configured to use SSL will fail to interpret server-reported >> topology and status events because those events currently include non-SSL >> ports, causing user connection pools to incorrectly track cluster changes >> >> I'm proposing that in the dual-native-port case, we log a warning to advise >> clients to move to a single port, with the same values for enabled and >> optional. With a single port, users won't need to worry about >> native_transport_port always accepting unencrypted traffic. The peers_v2 >> system table and EVENT response messages will include ports that clients >> will be able to connect to regardless of their SSL configuration. >> >> I'm open to discussing other ways we could handle this, but I think the >> requirements are: >> - Maintain compatibility with existing clients (no new tables for >> discovering peers, etc.) >> - Ensure SSL and non-SSL sessions can continue to operate (with >1 pooled >> connections) without disruption >> >> Thanks to Stefan for his efforts looking into this more closely with me >> yesterday. We found out how this came about as well - when the project moved >> support dual-native-port in CASSANDRA-9590 >> <https://issues.apache.org/jira/browse/CASSANDRA-9590>, driver >> configurations were expected to include hard-coded ports for encrypted and >> unencrypted traffic. Then, when customizable per-host native ports came out >> in CASSANDRA-7554 <https://issues.apache.org/jira/browse/CASSANDRA-7544>, >> drivers were expected to discover the right native protocol port from the >> system table instead of hard-coding it. So this has been a problem since 4.0 >> for users running dual-native-port and clients requiring SSL. >> >> -- >> Abe
