[
https://issues.apache.org/jira/browse/SPARK-23527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381258#comment-16381258
]
Gabor Somogyi commented on SPARK-23527:
---------------------------------------
Yeah, I agree with Yuming. In the first case host not found. In the second case
the server certificate is most probably self signed.
All you need to do is to add the server certificate to your trusted Java key
store.
> Error with spark-submit and kerberos with TLS-enabled Hadoop cluster
> --------------------------------------------------------------------
>
> Key: SPARK-23527
> URL: https://issues.apache.org/jira/browse/SPARK-23527
> Project: Spark
> Issue Type: Bug
> Components: Spark Submit
> Affects Versions: 2.2.1
> Environment: core-site.xml
> <property>
> <name>hadoop.security.key.provider.path</name>
> <value>kms://ht...@host1.domain.com;host2.domain.com:16000/kms</value>
> </property>
> hdfs-site.xml
> <property>
> <name>dfs.encryption.key.provider.uri</name>
> <value>kms://ht...@host1.domain.com;host2.domain.com:16000/kms</value>
> </property>
> Reporter: Ron Gonzalez
> Priority: Critical
>
> For current configuration of our enterprise cluster, I submit using
> spark-submit:
> ./spark-submit --master yarn --deploy-mode cluster --class
> org.apache.spark.examples.SparkPi --conf
> spark.yarn.jars=hdfs:/user/user1/spark/lib/*.jar
> ../examples/jars/spark-examples_2.11-2.2.1.jar 10
> I am getting the following problem:
>
> 18/02/27 21:03:48 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token
> 3351181 for svchdc236d on ha-hdfs:nameservice1
> Exception in thread "main" java.lang.IllegalArgumentException:
> java.net.UnknownHostException: host1.domain.com;host2.domain.com
> at
> org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.getDelegationTokenService(KMSClientProvider.java:825)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.addDelegationTokens(KMSClientProvider.java:781)
> at
> org.apache.hadoop.crypto.key.KeyProviderDelegationTokenExtension.addDelegationTokens(KeyProviderDelegationTokenExtension.java:86)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.addDelegationTokens(DistributedFileSystem.java:2046)
> at
> org.apache.spark.deploy.yarn.security.HadoopFSCredentialProvider$$anonfun$obtainCredentials$1.apply(HadoopFSCredentialProvider.scala:52)
>
> If I get rid of the other host for the properties so instead of
> kms://ht...@host1.domain.com;host2.domain.com:16000/kms, I convert it to:
> kms://ht...@host1.domain.com:16000/kms
> it fails with a different error:
> java.io.IOException: javax.net.ssl.SSLHandshakeException:
> sun.security.validator.ValidatorException: PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target
> If I do the same thing using spark 1.6, it works so it seems like a
> regression...
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
