[
https://issues.apache.org/jira/browse/HDDS-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943351#comment-16943351
]
Xiaoyu Yao edited comment on HDDS-2230 at 10/3/19 5:17 PM:
-----------------------------------------------------------
The invalid format is ignored by the script.
The hadoop-runner image set /opt/hadoop to be owned by hadoop:users.
When patch 001 enable Yarn LinuxContainer, it requires the
/opt/hadoop/etc/hadoop to be owned by root and not modifiable by other users,
which prevents the start.sh from changing the configurations based on
docker-config. As a result, the NM failed to start with these settings.
Attach a patch that comment(skip) only the following three settings related to
LinuxContainer. I tested patch v02 locally and the MR jobs finished
successfully.
+#YARN-SITE.XML_yarn.nodemanager.container-executor.class=org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor
+#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.path=/opt/hadoop/bin/container-executor
+#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.group=hadoop
was (Author: xyao):
The invalid format is ignored by the script. With patch 001, the Yarn
LinuxContainer requires the /opt/hadoop/etc/hadoop to be owned by root and not
modifiable by other users, which prevents the start.sh from changing the
configurations based on docker-config.
Attach a patch that comment(skip) only the following three settings related to
LinuxContainer. I tested patch v02 locally and the MR jobs finished
successfully.
+#YARN-SITE.XML_yarn.nodemanager.container-executor.class=org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor
+#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.path=/opt/hadoop/bin/container-executor
+#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.group=hadoop
> Invalid entries in ozonesecure-mr config
> ----------------------------------------
>
> Key: HDDS-2230
> URL: https://issues.apache.org/jira/browse/HDDS-2230
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: docker
> Reporter: Attila Doroszlai
> Assignee: Attila Doroszlai
> Priority: Minor
> Attachments: HDDS-2230.001.patch, HDDS-2230.002.patch
>
>
> Some of the entries in {{ozonesecure-mr/docker-config}} are in invalid
> format, thus they end up missing from the generated config files.
> {noformat}
> $ cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozonesecure-mr
> $ ./test.sh # configs are generated during container startup
> $ cd ../..
> $ grep -c 'ozone.administrators' compose/ozonesecure-mr/docker-config
> 1
> $ grep -c 'ozone.administrators' etc/hadoop/ozone-site.xml
> 0
> $ grep -c 'yarn.timeline-service' compose/ozonesecure-mr/docker-config
> 5
> $ grep -c 'yarn.timeline-service' etc/hadoop/yarn-site.xml
> 2
> $ grep -c 'container-executor' compose/ozonesecure-mr/docker-config
> 3
> $ grep -c 'container-executor' etc/hadoop/yarn-site.xml
> 0
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
