[
https://issues.apache.org/jira/browse/YARN-11960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated YARN-11960:
----------------------------------
Labels: pull-request-available (was: )
> Fix NodeManager OOM caused by large client.json
> -----------------------------------------------
>
> Key: YARN-11960
> URL: https://issues.apache.org/jira/browse/YARN-11960
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: KWON BYUNGCHANG
> Priority: Major
> Labels: pull-request-available
> Attachments: YARN-11960.001.patch
>
>
> In Spark or MapReduce jobs, when accessing private Docker registry, Docker
> config.json path can be set by below environment variable:
> {noformat}
> YARN_CONTAINER_RUNTIME_DOCKER_CLIENT_CONFIG=hdfs://ns/user/foo/config.json
> {noformat}
> NodeManager reads this file and writes to local filesystem as config.json.
> But current implementation reads whole file to memory and do JSON parsing,
> then writes it.
> If large file is configured, NodeManager can occur OOM.
> So file size limit should be added.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
