[ https://issues.apache.org/jira/browse/HDDS-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943351#comment-16943351 ]
Xiaoyu Yao edited comment on HDDS-2230 at 10/3/19 5:17 PM: ----------------------------------------------------------- The invalid format is ignored by the script. The hadoop-runner image set /opt/hadoop to be owned by hadoop:users. When patch 001 enable Yarn LinuxContainer, it requires the /opt/hadoop/etc/hadoop to be owned by root and not modifiable by other users, which prevents the start.sh from changing the configurations based on docker-config. As a result, the NM failed to start with these settings. Attach a patch that comment(skip) only the following three settings related to LinuxContainer. I tested patch v02 locally and the MR jobs finished successfully. +#YARN-SITE.XML_yarn.nodemanager.container-executor.class=org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor +#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.path=/opt/hadoop/bin/container-executor +#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.group=hadoop was (Author: xyao): The invalid format is ignored by the script. With patch 001, the Yarn LinuxContainer requires the /opt/hadoop/etc/hadoop to be owned by root and not modifiable by other users, which prevents the start.sh from changing the configurations based on docker-config. Attach a patch that comment(skip) only the following three settings related to LinuxContainer. I tested patch v02 locally and the MR jobs finished successfully. +#YARN-SITE.XML_yarn.nodemanager.container-executor.class=org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor +#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.path=/opt/hadoop/bin/container-executor +#YARN-SITE.XML_yarn.nodemanager.linux-container-executor.group=hadoop > Invalid entries in ozonesecure-mr config > ---------------------------------------- > > Key: HDDS-2230 > URL: https://issues.apache.org/jira/browse/HDDS-2230 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: docker > Reporter: Attila Doroszlai > Assignee: Attila Doroszlai > Priority: Minor > Attachments: HDDS-2230.001.patch, HDDS-2230.002.patch > > > Some of the entries in {{ozonesecure-mr/docker-config}} are in invalid > format, thus they end up missing from the generated config files. > {noformat} > $ cd hadoop-ozone/dist/target/ozone-0.5.0-SNAPSHOT/compose/ozonesecure-mr > $ ./test.sh # configs are generated during container startup > $ cd ../.. > $ grep -c 'ozone.administrators' compose/ozonesecure-mr/docker-config > 1 > $ grep -c 'ozone.administrators' etc/hadoop/ozone-site.xml > 0 > $ grep -c 'yarn.timeline-service' compose/ozonesecure-mr/docker-config > 5 > $ grep -c 'yarn.timeline-service' etc/hadoop/yarn-site.xml > 2 > $ grep -c 'container-executor' compose/ozonesecure-mr/docker-config > 3 > $ grep -c 'container-executor' etc/hadoop/yarn-site.xml > 0 > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org