ramrock2008 opened a new pull request #29171:
URL: https://github.com/apache/spark/pull/29171
Hello Spark developers,
I'm running the following basic spark job on YARN with SSL enabled:
```
spark-submit --class org.apache.spark.examples.SparkPi --master yarn
--deploy-mode client /usr/lib/spark/examples/jars/spark-examples.jar 3
```
On my cluster, SSL was enabled for spark and it contains the following
configurations:
```
$ cat /etc/spark/conf/spark-defaults.conf
spark.network.crypto.enabled true
spark.network.crypto.keyFactoryAlgorithm PBKDF2WithHmacSHA256
spark.ssl.protocol TLSv1.2
spark.ssl.keyStorePassword *********
spark.ssl.keyStore /tmp/abc/keystore.jks
spark.authenticate true
spark.network.crypto.keyLength 256
spark.network.crypto.saslFallback true
spark.ssl.keyPassword *********
spark.ssl.enabled true
spark.ssl.enabledAlgorithms
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA
spark.ssl.trustStore /tmp/abc/truststore.jks
spark.ssl.trustStorePassword **********
```
Although the keystores specified in the above configuration contain expired
certificate, spark job still runs fine without throwing any error.
Any reason why spark job is not verifying the expiry date on certificates.
```
hadoop@ip-172-31-13-56 ~]$ keytool -list -v -keystore /tmp/abc/truststore.jks
Enter keystore password:
***************** WARNING WARNING WARNING *****************
* The integrity of the information stored in your keystore *
* has NOT been verified! In order to verify its integrity, *
* you must provide your keystore password. *
***************** WARNING WARNING WARNING *****************
Keystore type: jks
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: test
Creation date: Jul 16, 2020
Entry type: trustedCertEntry
Owner: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Issuer: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Serial number: 42271acc
Valid from: Wed Jul 15 01:22:07 UTC 2020 until: Thu Jul 16 01:22:07 UTC 2020
Certificate fingerprints:
MD5: EB:B0:1E:C7:2A:81:1C:EB:25:DC:FD:47:FC:5D:9B:F6
SHA1: 15:C5:3F:E4:37:D0:F7:1B:7F:4D:13:B0:03:C6:18:FF:F1:6E:20:1A
SHA256:
08:0F:63:4B:29:B5:54:E7:24:46:C1:A3:9F:A9:45:D8:50:BF:49:38:6E:EA:8C:F7:2D:1C:30:B2:F5:72:9B:51
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 8D 2B E1 AC 3C B1 81 9D 79 FE 44 D8 EB BC E8 7F .+..<...y.D.....
0010: F7 0B F7 3D ...=
]
]
*******************************************
*******************************************
[hadoop@ip-172-31-13-56 ~]$ keytool -list -v -keystore /tmp/abc/keystore.jks
Enter keystore password:
***************** WARNING WARNING WARNING *****************
* The integrity of the information stored in your keystore *
* has NOT been verified! In order to verify its integrity, *
* you must provide your keystore password. *
***************** WARNING WARNING WARNING *****************
Keystore type: jks
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: test
Creation date: Jul 15, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Issuer: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Serial number: 42271acc
Valid from: Wed Jul 15 01:22:07 UTC 2020 until: Thu Jul 16 01:22:07 UTC 2020
Certificate fingerprints:
MD5: EB:B0:1E:C7:2A:81:1C:EB:25:DC:FD:47:FC:5D:9B:F6
SHA1: 15:C5:3F:E4:37:D0:F7:1B:7F:4D:13:B0:03:C6:18:FF:F1:6E:20:1A
SHA256:
08:0F:63:4B:29:B5:54:E7:24:46:C1:A3:9F:A9:45:D8:50:BF:49:38:6E:EA:8C:F7:2D:1C:30:B2:F5:72:9B:51
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 8D 2B E1 AC 3C B1 81 9D 79 FE 44 D8 EB BC E8 7F .+..<...y.D.....
0010: F7 0B F7 3D ...=
]
]
*******************************************
*******************************************
Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to
PKCS12 which is an industry standard format using "keytool -importkeystore
-srckeystore /tmp/abc/keystore.jks -destkeystore /tmp/abc/keystore.jks
-deststoretype pkcs12".
```
Is there something I'm missing here or is that the default behaviour of
spark. Is there a way to make this authentication strict if possible
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
