nchammas commented on issue #27534: [SPARK-30879][DOCS] Refine workflow for building docs URL: https://github.com/apache/spark/pull/27534#issuecomment-606780868 > We can manually test it by changing the `ENTRYPOINT` in `Dockerfile` to `/bin/bash` and check if the python and ruby are installed correctly. > > The release script doesn't work anymore after this patch because we use a non-standard way to install python/ruby (download a bash script and run it): > > 1. The actual work is done by user `spark-rm`, which can't access `/root/.pyenv` > 2. Even if we fix the permission issue, the installed python doesn't work well with the system libraries, and have errors like `ImportError: No module named threading` > 3. It's wrong to append the `PATH`, and we should prepend it. Otherwise `python` still points to the system default which is 2.7. This is strange since I tested this successfully via `docker build .` from within the folder containing the Dockerfile. What I couldn't do was test building the image via `do-release-docker.sh`, which I called out in the PR description and again [in the comments](https://github.com/apache/spark/pull/27534#issuecomment-592144594). I think the reason my test didn't see these issues is because the `do-release-docker.sh` script builds the image with some options that change the executing user from `root`. ``` run_silent "Building spark-rm image with tag $IMGTAG..." "docker-build.log" \ docker build --no-cache -t "spark-rm:$IMGTAG" --build-arg UID=$UID "$SELF/spark-rm" ``` I guess the moral of the story is that I couldn't test `do-release-docker.sh` directly, and my substitute test of `docker build .` was not a good test since it didn't account for the `root` vs. `spark-rm` user change. I should have more carefully tried to duplicate the `docker build` command as it appeared in `do-release-docker.sh`, and insisted more forcefully that we not merge this PR in until a committer had had a chance to try out `do-release-docker.sh` directly. Sorry.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
