GitHub user rvesse opened a pull request: https://github.com/apache/spark/pull/23017
[WIP][SPARK-26015][K8S] Set a default UID for Spark on K8S Images ## What changes were proposed in this pull request? Adds USER directives to the Dockerfiles which is configurable via build argument (`spark_uid`) for easy customisation. A `-u` flag is added to `bin/docker-image-tool.sh` to make it easy to customise this e.g. ``` > bin/docker-image-tool.sh -r rvesse -t uid -u 185 build > bin/docker-image-tool.sh -r rvesse -t uid push ``` If no UID is explicitly specified it defaults to `185` - this is per @skonto's suggestion to align with the OpenShift standard reserved UID for Java apps ( https://lists.openshift.redhat.com/openshift-archives/users/2016-March/msg00283.html) Notes: - We have to make the `WORKDIR` writable by the root group or otherwise jobs will fail with `AccessDeniedException` - We have to specify `HOME` to set it to the `WORKDIR` to avoid an Ivy error caused by `user.home` not having a valid value otherwise To Do: - [ ] Debug and resolve issue with client mode test - [ ] Consider whether to always propagate `SPARK_USER_NAME` to environment of driver and executor pods so `entrypoint.sh` can insert that into `/etc/passwd` entry - [ ] Rebase once PR #23013 is merged and update documentation accordingly ## How was this patch tested? Built the Docker images with the new Dockerfiles that include the `USER` directives. Ran the Spark on K8S integration tests against the new images. All pass except client mode which I am currently debugging further. Also manually dropped myself into the resulting container images via `docker run` and checked `id -u` output to see that UID is as expected. Tried customising the UID from the default via the new `-u` argument to `docker-image-tool.sh` and again checked the resulting image for the correct runtime UID. cc @felixcheung @skonto @vanzin You can merge this pull request into a Git repository by running: $ git pull https://github.com/rvesse/spark SPARK-26015 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/23017.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #23017 ---- commit 92e22f76bbe61035dcde54c5fb2990cb7099a98d Author: Rob Vesse <rvesse@...> Date: 2018-11-12T13:44:03Z [SPARK-26015][K8S] Set a default UID for Spark on K8S Images Adds USER directives to the Dockerfiles which is configurable via build argument for easy customisation. A -u flag is added to bin/docker-image-tool.sh to make it easy to customise this. ---- --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org