Jose Luis Pedrosa created SPARK-28258:
-----------------------------------------
Summary: Incopatibility betweek spark docker image and hadoop 3.2
and azure tools
Key: SPARK-28258
URL: https://issues.apache.org/jira/browse/SPARK-28258
Project: Spark
Issue Type: Bug
Components: Kubernetes
Affects Versions: 2.4.3
Reporter: Jose Luis Pedrosa
Currently the docker images generated by the distro uses openjdk8 based on
alpine.
This means that the version shipped of libssl is 1.1.1b-r1:
{noformat}
sh-4.4# apk list | grep ssl
libssl1.1-1.1.1b-r1 x86_64 {openssl} (OpenSSL) [installed]
{noformat}
The hadoop distro ships wildfly-openssl-1.0.4.Final.jar, which is affected by
https://issues.jboss.org/browse/JBEAP-16425.
This results on error running on the executor:
{noformat}
2019-07-04 22:32:40,339 INFO openssl.SSL: WFOPENSSL0002 OpenSSL Version OpenSSL
1.1.1b 26 Feb 2019
2019-07-04 22:32:40,363 WARN streaming.FileStreamSink: Error while looking for
metadata directory.
Exception in thread "main" java.lang.NullPointerException
at
org.wildfly.openssl.CipherSuiteConverter.toJava(CipherSuiteConverter.java:284)
{noformat}
In my tests creating a Docker image with an updated version of wildfly, solves
the issue: 1.0.7.Final
Not sure if this is Spark problem, if so, where would be the right place to
solve it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
