[
https://issues.apache.org/jira/browse/HDDS-524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652426#comment-16652426
]
Elek, Marton commented on HDDS-524:
-----------------------------------
The problem could be reproduced by changing the image from flokkr/hadoop to
apache/hadoop:3 (in compose/ozone-hdfs/docker-compose.yaml)
And execute a test (or just start a cluster).
{code:java}
./test.sh --keep --env ozone-hdfs basic
{code}
The flokkr/hadoop image doesn't include the robot tests so it would be better
to use the official image. But it was not possible to use apache/hadoop:3
because this problem (cluster can't be started)
I was think a lot about this patch (sorry for the late answer). There are two
main approaches:
1. fix the permission at runtime (this patch does it very well)
2. fix the permission at runtime (the same command but add it to the
Dockerfile: RUN sudo chown ...)
First I thought that both of them are working. But yesterday we found an
(other) problem with Bharat and I learned that sometimes how hard is to execute
the docker clusters with different user.
My user id is 1000 (outside) and the hadoop id is 1000 (inside the docker
container) so in my case the container could read/write everything from my
hadoop-ozone/dist/taget/ozone-0.3-SNAPSHOT (which is mounted as /opt/hadoop).
But in case of docker-machine the user id inside could be different from
outside. In this very specific case the only safe method is fixing permissions
but not the owner.
For example with this patch if the external/real/osx user is id=510 and the
hadoop user in the container is id=1000, the owner for all the files from
hadoop-ozone/dist/target/ozone-0.3-SNAPSHOT will be id=1000 (external). So you
can't do a maven clean any more as you can't delete the cluster.
I don't know if it was clean enough (let's discuss it offline if not) but
finally I prefer the second approach. To fix the owner of log4j.properties with
executing the chown from the Dockerfile.
> log4j is added with root to apache/hadoop:2 and apache/hadoop:3 images
> ----------------------------------------------------------------------
>
> Key: HDDS-524
> URL: https://issues.apache.org/jira/browse/HDDS-524
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Elek, Marton
> Assignee: Dinesh Chitlangia
> Priority: Major
> Labels: newbie
> Attachments: HDDS-524-docker-hadoop-runner.001.patch
>
>
> {code}
> docker run -it apache/hadoop:2 ls -lah /opt/hadoop/etc/hadoop
> total 152K
> drwxr-xr-x 1 hadoop users 4.0K Aug 13 17:08 .
> drwxr-xr-x 1 hadoop users 4.0K Nov 13 2017 ..
> -rw-r--r-- 1 hadoop users 7.7K Nov 13 2017 capacity-scheduler.xml
> ...
> -rw-r--r-- 1 hadoop users 5.8K Nov 13 2017 kms-site.xml
> -rw-r--r-- 1 root root 1023 Aug 13 17:04 log4j.properties
> -rw-r--r-- 1 hadoop users 1.1K Nov 13 2017 mapred-env.cmd
> ...
> {code}
> The owner of the log4j is root instead of hadoop. For this reason we can't
> use the images for acceptance tests as the launcher script can't overwrite
> log4j properties based on the environment variables.
> Same is true with
> {code}
> docker run -it apache/hadoop:2 ls -lah /opt/hadoop/etc/hadoop
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
