roczei opened a new pull request, #47795:
URL: https://github.com/apache/spark/pull/47795
### What changes were proposed in this pull request?
This PR aims to upgrade `okhttp` to 4.12.0 and `okio` to 3.6.0
### Why are the changes needed?
okhttp depends on okio which has to be upgraded as well. The new okhttp
version fixes the following vulnerabilities:
1. CVE-2023-0833 - A flaw was found in Red Hat's AMQ-Streams, which ships a
version of the OKHttp component with an
information disclosure flaw via an exception triggered by a header
containing an illegal value. This issue could allow
an authenticated attacker to access information outside of their regular
permissions.
CVSSv3 Score:- 5.5(Medium)
https://nvd.nist.gov/vuln/detail/CVE-2023-0833
2. CVE-2021-0341 - In verifyHostName of OkHostnameVerifier.java, there is a
possible way to accept a certificate for the wrong domain due to improperly
used crypto. This could lead to remote information disclosure with no
additional execution privileges needed. User interaction is not needed for
exploitation.
CVSSv3 Score:- 7.5(High)
https://nvd.nist.gov/vuln/detail/CVE-2021-0341
https://github.com/square/okhttp/issues/6724
There are two places in the Spark repository where the okhttp dependency
comes in as transitive dependency:
1. hadoop-huaweicloud
```
[INFO] +- org.apache.hadoop:hadoop-cloud-storage:jar:3.4.0:compile
[INFO] | +- org.apache.hadoop:hadoop-annotations:jar:3.4.0:compile
[INFO] | +- org.apache.hadoop:hadoop-aliyun:jar:3.4.0:compile
[INFO] | | +- com.aliyun.oss:aliyun-sdk-oss:jar:3.13.2:compile
[INFO] | | | +- org.jdom:jdom2:jar:2.0.6:compile
[INFO] | | | +- com.aliyun:aliyun-java-sdk-core:jar:4.5.10:compile
[INFO] | | | | +- org.ini4j:ini4j:jar:0.5.4:compile
[INFO] | | | | +- io.opentracing:opentracing-api:jar:0.33.0:compile
[INFO] | | | | \- io.opentracing:opentracing-util:jar:0.33.0:compile
[INFO] | | | | \- io.opentracing:opentracing-noop:jar:0.33.0:compile
[INFO] | | | +- com.aliyun:aliyun-java-sdk-ram:jar:3.1.0:compile
[INFO] | | | \- com.aliyun:aliyun-java-sdk-kms:jar:2.11.0:compile
[INFO] | | \- org.codehaus.jettison:jettison:jar:1.5.4:compile
[INFO] | +- org.apache.hadoop:hadoop-azure-datalake:jar:3.4.0:compile
[INFO] | | \-
com.microsoft.azure:azure-data-lake-store-sdk:jar:2.3.9:compile
[INFO] | \- org.apache.hadoop:hadoop-huaweicloud:jar:3.4.0:compile
[INFO] | \- com.huaweicloud:esdk-obs-java:jar:3.20.4.2:compile
[INFO] | +- com.jamesmurty.utils:java-xmlbuilder:jar:1.2:compile
[INFO] | +- com.squareup.okhttp3:okhttp:jar:3.14.2:compile
[INFO] | \- com.squareup.okio:okio:jar:1.17.2:compile
```
The Hadoop team has attempted to remove okhttp from their codebase:
remove okhttp usage: https://issues.apache.org/jira/browse/HADOOP-18890
Unfortunately the hadoop-huaweicloud dependency is still there which
pulls in the vulnerable okhttp 3.x version:
https://github.com/apache/hadoop/blob/trunk/hadoop-cloud-storage-project/hadoop-cloud-storage/pom.xml#L137C19-L137C37
3.4.0 is the only available version here:
https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-huaweicloud
2. kubernetes-client
```
[INFO] +- org.apache.spark:spark-kubernetes_2.13:jar:4.0.0-SNAPSHOT:compile
[INFO] | +- io.fabric8:kubernetes-httpclient-okhttp:jar:6.13.2:compile
[INFO] | | +- io.fabric8:kubernetes-client-api:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-core:jar:6.13.2:compile
[INFO] | | | | \- io.fabric8:kubernetes-model-common:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-gatewayapi:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-resource:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-rbac:jar:6.13.2:compile
[INFO] | | | +-
io.fabric8:kubernetes-model-admissionregistration:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-apps:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-autoscaling:jar:6.13.2:compile
[INFO] | | | +-
io.fabric8:kubernetes-model-apiextensions:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-batch:jar:6.13.2:compile
[INFO] | | | +-
io.fabric8:kubernetes-model-certificates:jar:6.13.2:compile
[INFO] | | | +-
io.fabric8:kubernetes-model-coordination:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-discovery:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-events:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-extensions:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-flowcontrol:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-networking:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-metrics:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-policy:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-scheduling:jar:6.13.2:compile
[INFO] | | | +-
io.fabric8:kubernetes-model-storageclass:jar:6.13.2:compile
[INFO] | | | +- io.fabric8:kubernetes-model-node:jar:6.13.2:compile
[INFO] | | | \- org.snakeyaml:snakeyaml-engine:jar:2.7:compile
[INFO] | | +- com.squareup.okhttp3:okhttp:jar:3.12.12:compile
[INFO] | | | \- com.squareup.okio:okio:jar:1.15.0:compile
[INFO] | | \- com.squareup.okhttp3:logging-interceptor:jar:3.12.12:compile
````
The kubernetes-client's maintainers do not want upgrade to okhttp 4.x
because it's based on Kotlin, they recommend to exclude 3.x. Related
documentation:
https://github.com/fabric8io/kubernetes-client/blob/main/doc/KubernetesClientWithIPv6Clusters.md
My proposed solution based on the above findings:
I think we should exclude the 3.x version and switch to use okhttp 4.x. It
is binary backwards compatible with okhttp 3.x. More details are here:
https://square.github.io/okhttp/upgrading_to_okhttp_4/
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
