[
https://issues.apache.org/jira/browse/YARN-11255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603687#comment-17603687
]
ASF GitHub Bot commented on YARN-11255:
---------------------------------------
ashutoshcipher opened a new pull request, #4884:
URL: https://github.com/apache/hadoop/pull/4884
### Description of PR
Support loading alternative docker client config from system environment
When using YARN docker support, although the hadoop shell supported
```
-docker_client_config
```
to pass the client config file that contains security token to generate the
docker config for each job as a temporary file.
For other applications that submit jobs to YARN, e.g. Spark, which loads the
docker setting via system environment e.g.
```
spark.executorEnv.*
```
will not be able to add those authorization token because this system
environment isn't considered in YARN.
Add genetic solution to handle these kind of cases without making changes in
spark code or others
### How was this patch tested?
Unit tests and manually verified by running on EMR.
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Support loading alternative docker client config from system environment
> ------------------------------------------------------------------------
>
> Key: YARN-11255
> URL: https://issues.apache.org/jira/browse/YARN-11255
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Ashutosh Gupta
> Assignee: Ashutosh Gupta
> Priority: Major
>
> When using YARN docker support, although the hadoop shell supported
> {code:java}
> -docker_client_config{code}
> to pass the client config file that contains security token to generate the
> docker config for each job as a temporary file.
> For other applications that submit jobs to YARN, e.g. Spark, which loads the
> docker setting via system environment e.g.
> {code:java}
> spark.executorEnv.* {code}
> will not be able to add those authorization token because this system
> environment isn't considered in YARN.
> Add genetic solution to handle these kind of cases without making changes in
> spark code or others
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
