Billie Rinaldi commented on YARN-5701:

Here are details on the bugs I am attempting to fix in this patch.

The changes in SliderClient are to use System.console() instead of a 
BufferedReader to prompt the user for passwords when the app requests passwords 
in the appConfig. When using the old slider python script that launched the 
SliderClient, System.console did not work, so we used BufferedReader and masked 
the passwords in the slider script. When running with the yarn script instead 
of slider, we need to use System.console() (as is done in CredentialShell) 
because the yarn script is not masking the passwords.

Another issue that came up when testing an assembly app that requests passwords 
is that the assembly is not inheriting the credentials request of the child / 
external apps. This is fixed in the InstanceBuilder.java changes.

In ProviderUtils.filterSiteOptions, the return map is being used as 
substitution tokens, so the keys should be in the format $\{key\}. This is 
fixing a problem where tokens are not being substituted properly.

In ProviderUtils.createConfigFile and ProviderUtils.localizeConfigFile, there 
is a concurrency problem in the creation of config files that occurs when 
multiple containers of the same type are being launched at once. This part of 
the patch fixes those.

The new methods in ProviderUtils, getHostNamesList and getIPsList, are used to 
populate substitution tokens COMPONENTNAME_HOSTNAME and COMPONENTNAME_IP. These 
are needed in addition to COMPONENTNAME_HOST token which provides the name of 
the host where a container is running. In the docker container case, the 
HOSTNAME substitution will be the docker hostname of the container (we may want 
to change this later to be the DNS hostname in the case when RegistryDNS is 
enabled). This patch also adds handling for the nested assembly case when 
role.prefix is populated.

DockerProviderService line 374 fixes a typo; it should have been using 
replaceAll instead of replace.

DockerProviderService.buildMonitorDetails and 
DockerProviderService.buildRoleHostDetails are needed to populate information 
on the AM UI page. These methods are copied from the original 

The .keep file needs to be added to make the Slider AM web service work. It is 
hard to add this file. You have to run "git add -f 
 before doing the git commit.

Lastly, clusterName has been added as a parameter to a couple of methods where 
it was missing. CLUSTER_NAME was not being substituted properly in exports 
because the parameter was not being passed.

> Fix issues in yarn native services apps-of-apps
> -----------------------------------------------
>                 Key: YARN-5701
>                 URL: https://issues.apache.org/jira/browse/YARN-5701
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>         Attachments: YARN-5701-yarn-native-services-001.patch, 
> YARN-5701-yarn-native-services.001.patch
> In testing out complex apps-of-apps with the docker provider, I found a few 
> issues primarily related to exports, as well as a couple of issues in config 
> generation and CredentialProvider handling.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to