[ https://issues.apache.org/jira/browse/CASSANDRA-15980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168290#comment-17168290 ]
David Capwell commented on CASSANDRA-15980: ------------------------------------------- Sorry for the delay, been trying to validate. In 3.11 I don't see any of these logs, so this is local to 4.0 and wouldn't regress our logging. It took a bit to replicate the issue, as the document linked from Nate doesn't cause these logs to happen; in fact ssl doesn't happen (is this a known issue?). Here are the steps I took to replicate {code} Based off https://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to-server.html ``` # create non-ssl cluster ccm create -n 3 -v 3.11.6 sslverify-311 ``` Setup the certs ``` cat <<EOF > gen_ca_cert.conf [ req ] distinguished_name = req_distinguished_name prompt = no output_password = mypass default_bits = 2048 [ req_distinguished_name ] C = US ST = TX L = Austin O = TLP OU = TestCluster CN = TestClusterMasterCA emailAddress = i...@thelastpickle.com EOF openssl req -config gen_ca_cert.conf -new -x509 -keyout ca-key -out ca-cert -days 365 # should now have ca-cert and ca-key # validate they work openssl x509 -in ca-cert -text -noout # setup the public/private keys setjdk 8 keytool -genkeypair -keyalg RSA -alias node1 -keystore node1-server-keystore.jks -storepass awesomekeypass -keypass awesomekeypass -validity 365 -keysize 2048 -dname "CN=node1, OU=SSL-verification-cluster, O=TheLastPickle, C=US" keytool -genkeypair -keyalg RSA -alias node2 -keystore node2-server-keystore.jks -storepass awesomekeypass -keypass awesomekeypass -validity 365 -keysize 2048 -dname "CN=node2, OU=SSL-verification-cluster, O=TheLastPickle, C=US" keytool -genkeypair -keyalg RSA -alias node3 -keystore node3-server-keystore.jks -storepass awesomekeypass -keypass awesomekeypass -validity 365 -keysize 2048 -dname "CN=node3, OU=SSL-verification-cluster, O=TheLastPickle, C=US" # should now have node1-server-keystore.jks node2-server-keystore.jks and node3-server-keystore.jks # validate they work keytool -list -v -keystore node1-server-keystore.jks -storepass awesomekeypass keytool -keystore node1-server-keystore.jks -alias node1 -certreq -file node1_cert_sr -keypass awesomekeypass -storepass awesomekeypass keytool -keystore node2-server-keystore.jks -alias node2 -certreq -file node2_cert_sr -keypass awesomekeypass -storepass awesomekeypass keytool -keystore node3-server-keystore.jks -alias node3 -certreq -file node3_cert_sr -keypass awesomekeypass -storepass awesomekeypass # should now have node1_cert_sr and node2_cert_sr node3_cert_sr # sign openssl x509 -req -CA ca-cert -CAkey ca-key -in node1_cert_sr -out node1_cert_signed -days 365 -CAcreateserial -passin pass:mypass openssl x509 -req -CA ca-cert -CAkey ca-key -in node2_cert_sr -out node2_cert_signed -days 365 -CAcreateserial -passin pass:mypass openssl x509 -req -CA ca-cert -CAkey ca-key -in node3_cert_sr -out node3_cert_signed -days 365 -CAcreateserial -passin pass:mypass should have node1_cert_signed node2_cert_signed node3_cert_signed and ca-cert.srl # add to key store keytool -keystore node1-server-keystore.jks -alias CARoot -import -file ca-cert -noprompt -keypass awesomekeypass -storepass awesomekeypass keytool -keystore node2-server-keystore.jks -alias CARoot -import -file ca-cert -noprompt -keypass awesomekeypass -storepass awesomekeypass keytool -keystore node3-server-keystore.jks -alias CARoot -import -file ca-cert -noprompt -keypass awesomekeypass -storepass awesomekeypass should have node1-server-keystore.jks node2-server-keystore.jks and node3-server-keystore.jks keytool -keystore node1-server-keystore.jks -alias node1 -import -file node1_cert_signed -keypass awesomekeypass -storepass awesomekeypass keytool -keystore node2-server-keystore.jks -alias node2 -import -file node2_cert_signed -keypass awesomekeypass -storepass awesomekeypass keytool -keystore node3-server-keystore.jks -alias node3 -import -file node3_cert_signed -keypass awesomekeypass -storepass awesomekeypass # Build the trust store keytool -keystore generic-server-truststore.jks -alias CARoot -importcert -file ca-cert -keypass mypass -storepass truststorepass -noprompt # should have generic-server-truststore.jks ``` ``` ccm create -n 3 sslverify-trunk --install-dir=$HOME/src/github/apache/cassandra-trunk cluster="sslverify-trunk" cp node1-server-keystore.jks ~/.ccm/${cluster}/node1/conf/server-keystore.jks cp node2-server-keystore.jks ~/.ccm/${cluster}/node2/conf/server-keystore.jks cp node3-server-keystore.jks ~/.ccm/${cluster}/node3/conf/server-keystore.jks cp generic-server-truststore.jks ~/.ccm/${cluster}/node1/conf/server-truststore.jks cp generic-server-truststore.jks ~/.ccm/${cluster}/node2/conf/server-truststore.jks cp generic-server-truststore.jks ~/.ccm/${cluster}/node3/conf/server-truststore.jks # validate they are there ls -l ~/.ccm/$cluster/node?/conf/*.jks # update the yaml to be (replace $USER, and $NODE with node<num>, ccm won't) # server_encryption_options: # internode_encryption: all # keystore: /Users/$USER/.ccm/sslverify-trunk/$NODE/conf/server-keystore.jks # keystore_password: awesomekeypass # truststore: /Users/$USER/.ccm/sslverify-trunk/$NODE/conf/server-truststore.jks # truststore_password: truststorepass # #protocol: TLS # algorithm: SunX509 # store_type: JKS # cipher_suites: [TLS_RSA_WITH_AES_256_CBC_SHA] # require_client_auth: true # # not from nates doc # enable_legacy_ssl_storage_port: true # enabled: true # optional: false # protocol: TLSv1.2 vim ~/.ccm/$cluster/node?/conf/cassandra.yaml ``` Now to test Jon's code ``` ccm create -n 3 sslverify-c15980 --install-dir=$HOME/src/github/apache/team/jonmeredith cluster="sslverify-c15980" cp node1-server-keystore.jks ~/.ccm/${cluster}/node1/conf/server-keystore.jks cp node2-server-keystore.jks ~/.ccm/${cluster}/node2/conf/server-keystore.jks cp node3-server-keystore.jks ~/.ccm/${cluster}/node3/conf/server-keystore.jks cp generic-server-truststore.jks ~/.ccm/${cluster}/node1/conf/server-truststore.jks cp generic-server-truststore.jks ~/.ccm/${cluster}/node2/conf/server-truststore.jks cp generic-server-truststore.jks ~/.ccm/${cluster}/node3/conf/server-truststore.jks # validate they are there ls -l ~/.ccm/$cluster/node?/conf/*.jks # update the yaml to be (replace $USER, and $NODE with node<num>, ccm won't) # server_encryption_options: # internode_encryption: all # keystore: /Users/$USER/.ccm/sslverify-trunk/$NODE/conf/server-keystore.jks # keystore_password: awesomekeypass # truststore: /Users/$USER/.ccm/sslverify-trunk/$NODE/conf/server-truststore.jks # truststore_password: truststorepass # #protocol: TLS # algorithm: SunX509 # store_type: JKS # cipher_suites: [TLS_RSA_WITH_AES_256_CBC_SHA] # require_client_auth: true # # not from nates doc # enable_legacy_ssl_storage_port: true # enabled: true # optional: false # protocol: TLSv1.2 vim ~/.ccm/$cluster/node?/conf/cassandra.yaml ``` {code} Overall LGTM, left a few small comments in the PR. > Improve log messages for socket connection/disconnection > -------------------------------------------------------- > > Key: CASSANDRA-15980 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15980 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging > Reporter: Jon Meredith > Assignee: Jon Meredith > Priority: Normal > Fix For: 4.0-beta > > Time Spent: 20m > Remaining Estimate: 0h > > Logging for inbound SSL connections can take place before protocol > negotiation has taken place and logs a misleading cipher that could cause > problems for security auditing. > > > {code:java} > INFO 2020-07-03T13:57:58,380 [Messaging-EventLoop-3-1] > org.apache.cassandra.net.InboundConnectionInitiator:242 - connection from > peer /1.1.1.1:57899 to /2.2.2.2:7000, protocol = TLSv1.2, cipher suite = > SSL_NULL_WITH_NULL_NULL > {code} > > Instead Cassandra should log the connection & protocol, then once the cipher > has been negotiated log the agreed upon cipher. > > > If the inbound SSL connection does not present a client certificate, > Cassandra logs this error, even if the client wasn't required to. > {code:java} > ERROR 2020-07-14T11:58:45,925 [Native-Transport-Requests-1] > org.apache.cassandra.transport.ServerConnection:140 - Failed to get peer > certificates for peer /4.3.2.1:59263 > {code} > > Logging the absense of verified certificates should be a concern of the > SaslNegotiator if it requires it, and not something worth alerting the > operator for generally. Downgrade to debug message to make investigation > possible if needed. > > > Finally, to help with logging issues related to disconnection, add a log > statement when an instance decides it no longer needs to keep a gossip > connection open when cleaning up connections in > org.apache.cassandra.net.OutboundConnections.UnusedConnectionMonitor#closeUnusedSinceLastRun -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org