[
https://issues.apache.org/jira/browse/YARN-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184700#comment-16184700
]
Robert Kanter commented on YARN-6457:
-------------------------------------
[~vrozov], [~sanjaypujare] we were doing some testing and found that this
change breaks a setup with HDFS HA + SSL + Hadoop Credstore. In that setup,
the RM will fail to startup with a stack trace like this:
{noformat}
Error starting ResourceManager
java.lang.IllegalArgumentException: java.net.UnknownHostException: ns1
at
org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:444)
at
org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:132)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:341)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:285)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:163)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3258)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:123)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3307)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3275)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:476)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:467)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at
org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at
org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
at
org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2157)
at
org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2095)
at
org.apache.hadoop.yarn.webapp.util.WebAppUtils.getPassword(WebAppUtils.java:431)
at
org.apache.hadoop.yarn.webapp.util.WebAppUtils.loadSslConfiguration(WebAppUtils.java:409)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:312)
at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:401)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:1119)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1229)
at
org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1425)
Caused by: java.net.UnknownHostException: ns1
... 28 more
{noformat}
I did some digging, and the problem is that with HDFS HA, we have a logical
name (i.e. "ns1") instead of an actual hostname. So when the Credstore (i.e.
{{Configuration.getPassword}}) tries to read the password, it needs to resolve
the logical name into a hostname; however, that information is now missing
because of this JIRA. If I change it so that we do {{new Configuration()}}
instead of {{new Configuration(false)}}, so we'll load hdfs-site (and others),
and that fixes the problem.
Why do we need to set {{loadDefaults}} to {{false}}?
> Allow custom SSL configuration to be supplied in WebApps
> --------------------------------------------------------
>
> Key: YARN-6457
> URL: https://issues.apache.org/jira/browse/YARN-6457
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: webapp, yarn
> Reporter: Sanjay M Pujare
> Assignee: Sanjay M Pujare
> Fix For: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
>
> Attachments: YARN-6457.00.patch, YARN-6457.01.patch
>
> Original Estimate: 96h
> Remaining Estimate: 96h
>
> Currently a custom SSL store cannot be passed on to WebApps which forces the
> embedded web-server to use the default keystore set up in ssl-server.xml for
> the whole Hadoop cluster. There are cases where the Hadoop app needs to use
> its own/custom keystore.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
