Ok thanks for the correction. The docker pull line shows as follows:
docker pull apache/spark:v3.2.1 So this only tells me the version of Spark 3.2.1 I thought we discussed deciding on the docker naming conventions in detail, and broadly agreed on what needs to be in the naming convention. For example, in this thread: Time to start publishing Spark Docker Images? - mich.talebza...@gmail.com - Gmail (google.com) <https://mail.google.com/mail/u/0/?hl=en-GB#search/publishing/FMfcgzGkZQSzbXWQDWfddGDNRDQfPCpg> dated 22nd July 2021 Referring to that, I think the broad agreement was that the docker image name should be of the form: The name of the file provides: - Built for spark or spark-py (PySpark) spark-r - Spark version: 3.1.1, 3.1.2, 3.2.1 etc. - Scala version; 2.1.2 - The OS version based on JAVA: 8-jre-slim-buster, 11-jre-slim-buster meaning JAVA 8 and JAVA 11 respectively I believe it is a good thing and we ought to adopt that convention. For example: spark-py-3.2.1-scala_2.12-11-jre-slim-buster HTH view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Mon, 21 Feb 2022 at 21:58, Holden Karau <hol...@pigscanfly.ca> wrote: > My bad, the correct link is: > > https://hub.docker.com/r/apache/spark/tags > > On Mon, Feb 21, 2022 at 1:17 PM Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > >> well that docker link is not found! may be permission issue >> >> [image: image.png] >> >> >> >> >> view my Linkedin profile >> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >> >> >> https://en.everybodywiki.com/Mich_Talebzadeh >> >> >> >> *Disclaimer:* Use it at your own risk. Any and all responsibility for >> any loss, damage or destruction of data or any other property which may >> arise from relying on this email's technical content is explicitly >> disclaimed. The author will in no case be liable for any monetary damages >> arising from such loss, damage or destruction. >> >> >> >> >> On Mon, 21 Feb 2022 at 21:09, Holden Karau <hol...@pigscanfly.ca> wrote: >> >>> We are happy to announce the availability of Spark 3.1.3! >>> >>> Spark 3.1.3 is a maintenance release containing stability fixes. This >>> release is based on the branch-3.1 maintenance branch of Spark. We >>> strongly >>> recommend all 3.1 users to upgrade to this stable release. >>> >>> To download Spark 3.1.3, head over to the download page: >>> https://spark.apache.org/downloads.html >>> >>> To view the release notes: >>> https://spark.apache.org/releases/spark-release-3-1-3.html >>> >>> We would like to acknowledge all community members for contributing to >>> this >>> release. This release would not have been possible without you. >>> >>> *New Dockerhub magic in this release:* >>> >>> We've also started publishing docker containers to the Apache Dockerhub, >>> these contain non-ASF artifacts that are subject to different license >>> terms than the >>> Spark release. The docker containers are built for Linux x86 and ARM64 >>> since that's >>> what I have access to (thanks to NV for the ARM64 machines). >>> >>> You can get them from https://hub.docker.com/apache/spark (and spark-r >>> and spark-py) :) >>> (And version 3.2.1 is also now published on Dockerhub). >>> >>> Holden >>> >>> -- >>> Twitter: https://twitter.com/holdenkarau >>> Books (Learning Spark, High Performance Spark, etc.): >>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>> >> > > -- > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >
