Yikun commented on code in PR #52: URL: https://github.com/apache/spark-docker/pull/52#discussion_r1274663861
########## 3.4.1/scala2.12-java11-ubuntu/Dockerfile: ########## @@ -36,15 +36,16 @@ RUN set -ex; \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.4.1/spark-3.4.1-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.4.1/spark-3.4.1-bin-hadoop3.tgz.asc \ +ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.4.1/spark-3.4.1-bin- \ + SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.4.1/spark-3.4.1-bin- \ GPG_KEY=F28C9C925C188C35E345614DEDA00CE834F0FC5C RUN set -ex; \ + if [ "2.12" = "2.13" ]; then export BIN_FILE_SUFFIX="hadoop3-scala2.13.tgz"; else export BIN_FILE_SUFFIX="hadoop3.tgz"; fi; \ Review Comment: As I mentioned in `Dockerfile.template`, if we address suffix concat in template.py then this dockerfile will not be changed. ########## versions.json: ########## @@ -22,8 +22,16 @@ "path": "3.4.1/scala2.12-java11-ubuntu", "tags": [ "3.4.1-scala2.12-java11-ubuntu", - "3.4.1-scala", - "scala" + "3.4.1-scala2.12", + "scala2.12" Review Comment: ```suggestion "scala2.12", "3.4.1-scala", "scala" ``` I believe the `scala`, `3.4.1-scala` tag should also be kept. ########## add-dockerfiles.sh: ########## @@ -33,6 +33,7 @@ scala2.12-java11-python3-r-ubuntu scala2.12-java11-python3-ubuntu scala2.12-java11-r-ubuntu scala2.12-java11-ubuntu +scala2.13-java11-ubuntu Review Comment: You only add the scala one, do you think python3/r/all should also be added in sometime? (Yep, just a question, IMO we can add scala2.13 now, and add others on demand in future) ########## Dockerfile.template: ########## @@ -36,15 +36,16 @@ RUN set -ex; \ # Install Apache Spark # https://downloads.apache.org/spark/KEYS -ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-{{ SPARK_VERSION }}/spark-{{ SPARK_VERSION }}-bin-hadoop3.tgz \ - SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-{{ SPARK_VERSION }}/spark-{{ SPARK_VERSION }}-bin-hadoop3.tgz.asc \ Review Comment: ``` ENV SPARK_TGZ_URL={{ SPARK_TGZ_URL }} \ SPARK_TGZ_ASC_URL= {{ SPARK_TGZ_ASC_URL }} \ ``` Could we only change the template in here, and address suffix and prefix in https://github.com/apache/spark-docker/blob/master/tools/template.py -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
