Kevin Bowman created SPARK-50164:
------------------------------------
Summary: Spark Kubernetes support requires vulnerable okhttp
dependency
Key: SPARK-50164
URL: https://issues.apache.org/jira/browse/SPARK-50164
Project: Spark
Issue Type: Bug
Components: Kubernetes
Affects Versions: 3.5.3, 3.4.1
Environment: Spark 3.4.1/Spark 3.5.3, on Ubuntu
Reporter: Kevin Bowman
We have been trying to resolve the following CVE flagged in our Spark install:
[https://www.cve.org/CVERecord?id=CVE-2023-0833]
The vulnerability is on {{{}okhttp-3.12.12.jar{}}}. It's pulled in by the
fabric8 kubernetes client: {{{}kubernetes-client-6.7.2.jar{}}}.
[The fabric8 kubernetes client does support using other HTTP
clients|https://blog.marcnuri.com/kubernetes-client-6-httpclient-how-to#vanilla-java-jdk-HttpClient].
However, Spark is explicitly initializing it with a bespoke okhttp client
instance. We are using 3.4.1, but the problem is still there in the master
branch.
# [Creating an okhttp dispatcher with a custom apache thread
pool|https://github.com/apache/spark/blob/cfe14c9c275b75f45b8d810333174cfd08c3af61/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala#L83C1-L84C1]
# [Building an okhttp client factory with that
dispatcher|https://github.com/apache/spark/blob/cfe14c9c275b75f45b8d810333174cfd08c3af61/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala#L120C1-L121C1]
# [Initializing fabric8 kubernetes client with the custom okhttp client
factory|https://github.com/apache/spark/blob/95b2d27079c2e012ab5bfb8c1dd83b11d7848258/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala#L127C1-L128C1]
Furthermore, we are unable to upgrade the okhttp client library itself.
The earliest version of okhttp that does not have the CVE, and is also not
dependent on a version of the Kotlin standard library that doesn't have CVEs,
appears to be {{okhttp:4.10.0}} (with {{{}kotlin-stdlib:1.6.20{}}}). However,
the latest release of the fabric8 kubernetes client
({{{}}kubernetes-client:6.13.4{}}}) is still dependent on okhttp 3.12.12. It
cannot run with okhttp 4.10.
In summary: Because it's hard-coded into Spark we are unable to replace okhttp,
and because fabric8 kubernetes client stopped support for newer versions of
okhttp we are unable to upgrade it to a non-vulnerable version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
