Worth noting that these official dockerfiles
https://hub.docker.com/_/spark were
created with a valid java version *openjdk version "11.0.19" 2023-04-18*
docker run -it* apache/spark:3.4.1-scala2.12-java11-r-ubuntu* /bin/bash ##
downloaded from the above repository
spark@23f0acf9296d:/opt/spark/work-dir$ java -version
*openjdk version "11.0.19" 2023-04-18*
OpenJDK Runtime Environment Temurin-11.0.19+7 (build 11.0.19+7)
OpenJDK 64-Bit Server VM Temurin-11.0.19+7 (build 11.0.19+7, mixed mode,
sharing)
Thus, the released file
${SPARK_HOME}/kubernetes/dockerfiles/spark/Dockerfile
Needs to be modified to replace the following line for those that need
to create bespoke images
ARG java_image_tag=17-jre
FROM eclipse-temurin:${java_image_tag}
with
ARG java_image_tag=11-jre-slim
FROM openjdk:${java_image_tag}
Otherwise it will fail.
This bespoke works
docker run -it
eu.gcr.io/<PROJECT>/spark:3.4.1-scala_2.12-11-jre-slim-buster-java11PlusPackages
bash
java --version
*openjdk 11.0.16 2022-07-19*
OpenJDK Runtime Environment 18.9 (build 11.0.16+8)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.16+8, mixed mode, sharing)
HTH
Mich Talebzadeh,
Solutions Architect/Engineering Lead
Palantir Technologies Limited
London
United Kingdom
view my Linkedin profile
<https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
https://en.everybodywiki.com/Mich_Talebzadeh
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
On Sat, 22 Jul 2023 at 18:48, Mich Talebzadeh <mich.talebza...@gmail.com>
wrote:
> Yes thanks, I know the answer. That was not what I was looking for. The
> provided script should be working one way or another which is not. good
> that someone has raised the issue already. that Jira was raised in Oct 2022
>
> Mich Talebzadeh,
> Solutions Architect/Engineering Lead
> Palantir Technologies Limited
> London
> United Kingdom
>
>
> view my Linkedin profile
> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>
>
> https://en.everybodywiki.com/Mich_Talebzadeh
>
>
>
> *Disclaimer:* Use it at your own risk. Any and all responsibility for any
> loss, damage or destruction of data or any other property which may arise
> from relying on this email's technical content is explicitly disclaimed.
> The author will in no case be liable for any monetary damages arising from
> such loss, damage or destruction.
>
>
>
>
> On Sat, 22 Jul 2023 at 18:29, Bjørn Jørgensen <bjornjorgen...@gmail.com>
> wrote:
>
>> https://hub.docker.com/_/openjdk
>> DEPRECATION NOTICE
>>
>> This image is officially deprecated and all users are recommended to find
>> and use suitable replacements ASAP. Some examples of other Official Image
>> alternatives (listed in alphabetical order with no intentional or implied
>> preference):
>>
>> - amazoncorretto <https://hub.docker.com/_/amazoncorretto>
>> - eclipse-temurin <https://hub.docker.com/_/eclipse-temurin>
>> - ibm-semeru-runtimes <https://hub.docker.com/_/ibm-semeru-runtimes>
>> - ibmjava <https://hub.docker.com/_/ibmjava>
>> - sapmachine <https://hub.docker.com/_/sapmachine>
>>
>> See docker-library/openjdk#505
>> <https://github.com/docker-library/openjdk/issues/505> for more
>> information.
>>
>>
>> [SPARK-40941][K8S] Use Java 17 in K8s Dockerfile by default and remove
>> Dockerfile.java17 <https://github.com/apache/spark/pull/38417>
>>
>>
>> lør. 22. juli 2023 kl. 17:33 skrev Mich Talebzadeh <
>> mich.talebza...@gmail.com>:
>>
>>> Hi,
>>>
>>> I was checking the contents of Dockerfile for JAVA in Spark directory,
>>> .i.e
>>>
>>> ${SPARK_HOME}/kubernetes/dockerfiles/spark/Dockerfile
>>>
>>> in version 3.4.1
>>>
>>> I recall that in 3.4.0, I made adjustment to Dockerfile content
>>> replacing
>>>
>>> #ARG java_image_tag=17-jre
>>> #FROM eclipse-temurin:${java_image_tag}
>>>
>>> with
>>>
>>>
>>> *ARG java_image_tag=11-jre-slim*
>>>
>>> *FROM openjdk:${java_image_tag}*
>>>
>>> This worked and dockerfile was created
>>>
>>> With version 3.4.1, the same issue seems to exist
>>>
>>> Sending build context to Docker daemon 466.1MB
>>> Step 1/18 : ARG java_image_tag=17-jre
>>> Step 2/18 : FROM eclipse-temurin:${java_image_tag}
>>> *manifest for eclipse-temurin:11-jre-slim not found: manifest unknown:
>>> manifest unknown*
>>> *Failed to build Spark JVM Docker image, please refer to Docker build
>>> output for details.*
>>>
>>> Thanks
>>>
>>> Mich Talebzadeh,
>>> Solutions Architect/Engineering Lead
>>> Palantir Technologies Limited
>>> London
>>> United Kingdom
>>>
>>>
>>> view my Linkedin profile
>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/>
>>>
>>>
>>> https://en.everybodywiki.com/Mich_Talebzadeh
>>>
>>>
>>>
>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for
>>> any loss, damage or destruction of data or any other property which may
>>> arise from relying on this email's technical content is explicitly
>>> disclaimed. The author will in no case be liable for any monetary damages
>>> arising from such loss, damage or destruction.
>>>
>>>
>>>
>>
>>
>> --
>> Bjørn Jørgensen
>> Vestre Aspehaug 4, 6010 Ålesund
>> Norge
>>
>> +47 480 94 297
>>
>
