MonkeyCanCode commented on code in PR #2957: URL: https://github.com/apache/polaris/pull/2957#discussion_r2486614249
########## plugins/spark/v3.5/regtests/Dockerfile: ########## @@ -18,31 +18,29 @@ # FROM docker.io/apache/spark:3.5.6-java17 -ARG POLARIS_HOST=polaris -ENV POLARIS_HOST=$POLARIS_HOST -ENV SPARK_HOME=/opt/spark -ENV CURRENT_SCALA_VERSION='2.12' -ENV LANGUAGE='en_US:en' + +ARG POLARIS_HOST=polaris \ + CURRENT_SCALA_VERSION=2.12 Review Comment: So we actually have both 2.12 and 2.13 plugin jar for spark 3.5 (supported by our CI as well in this dir for the plugin checks). This is used in the bash script under this path if we build 2.13. However, the existing getting start example is using 2.12 only. This ARG is needed as I will need to push the right jar into the image (which matches the corresponding scala version used on the spark base image...also, the spark image we are using for 3.5 is using scala 2.12 as well). But I guess we can remove this and use 2.12 for now as that is what the spark image is using? For spark 4.x, we will need to change to 2.13 as support for scala 2.12 is dropped. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
