[
https://issues.apache.org/jira/browse/YARN-5701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15576040#comment-15576040
]
Billie Rinaldi commented on YARN-5701:
--------------------------------------
Here are details on the bugs I am attempting to fix in this patch.
The changes in SliderClient are to use System.console() instead of a
BufferedReader to prompt the user for passwords when the app requests passwords
in the appConfig. When using the old slider python script that launched the
SliderClient, System.console did not work, so we used BufferedReader and masked
the passwords in the slider script. When running with the yarn script instead
of slider, we need to use System.console() (as is done in CredentialShell)
because the yarn script is not masking the passwords.
Another issue that came up when testing an assembly app that requests passwords
is that the assembly is not inheriting the credentials request of the child /
external apps. This is fixed in the InstanceBuilder.java changes.
In ProviderUtils.filterSiteOptions, the return map is being used as
substitution tokens, so the keys should be in the format $\{key\}. This is
fixing a problem where tokens are not being substituted properly.
In ProviderUtils.createConfigFile and ProviderUtils.localizeConfigFile, there
is a concurrency problem in the creation of config files that occurs when
multiple containers of the same type are being launched at once. This part of
the patch fixes those.
The new methods in ProviderUtils, getHostNamesList and getIPsList, are used to
populate substitution tokens COMPONENTNAME_HOSTNAME and COMPONENTNAME_IP. These
are needed in addition to COMPONENTNAME_HOST token which provides the name of
the host where a container is running. In the docker container case, the
HOSTNAME substitution will be the docker hostname of the container (we may want
to change this later to be the DNS hostname in the case when RegistryDNS is
enabled). This patch also adds handling for the nested assembly case when
role.prefix is populated.
DockerProviderService line 374 fixes a typo; it should have been using
replaceAll instead of replace.
DockerProviderService.buildMonitorDetails and
DockerProviderService.buildRoleHostDetails are needed to populate information
on the AM UI page. These methods are copied from the original
AgentProviderService.
The .keep file needs to be added to make the Slider AM web service work. It is
hard to add this file. You have to run "git add -f
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-slider/hadoop-yarn-slider-core/src/main/resources/webapps/slideram/.keep"
before doing the git commit.
Lastly, clusterName has been added as a parameter to a couple of methods where
it was missing. CLUSTER_NAME was not being substituted properly in exports
because the parameter was not being passed.
> Fix issues in yarn native services apps-of-apps
> -----------------------------------------------
>
> Key: YARN-5701
> URL: https://issues.apache.org/jira/browse/YARN-5701
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Billie Rinaldi
> Assignee: Billie Rinaldi
> Attachments: YARN-5701-yarn-native-services-001.patch,
> YARN-5701-yarn-native-services.001.patch
>
>
> In testing out complex apps-of-apps with the docker provider, I found a few
> issues primarily related to exports, as well as a couple of issues in config
> generation and CredentialProvider handling.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
