This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 3215ca7628d9 [SPARK-56441][BUILD] Upgrade `lz4-java` to 1.11.0
3215ca7628d9 is described below
commit 3215ca7628d9d883bd933731ff423be511e0fc51
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sat Apr 11 18:35:04 2026 +0800
[SPARK-56441][BUILD] Upgrade `lz4-java` to 1.11.0
### What changes were proposed in this pull request?
This PR aims to upgrade `at.yawk.lz4:lz4-java` to 1.11.0.
### Why are the changes needed?
To bring new improvements of `lz4-java` library.
- https://github.com/yawkat/lz4-java/releases/tag/v1.11.0
- https://github.com/yawkat/lz4-java/pull/44
### Does this PR introduce _any_ user-facing change?
No, this is a dependency-only change.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.6
Closes #55295 from dongjoon-hyun/SPARK-56441.
Lead-authored-by: Dongjoon Hyun <[email protected]>
Co-authored-by: dongjoon-hyun <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
core/benchmarks/LZ4TPCDSDataBenchmark-jdk21-results.txt | 8 ++++----
core/benchmarks/LZ4TPCDSDataBenchmark-jdk25-results.txt | 8 ++++----
core/benchmarks/LZ4TPCDSDataBenchmark-results.txt | 12 ++++++------
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
5 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/core/benchmarks/LZ4TPCDSDataBenchmark-jdk21-results.txt
b/core/benchmarks/LZ4TPCDSDataBenchmark-jdk21-results.txt
index 031c00ec96bd..454d52f11985 100644
--- a/core/benchmarks/LZ4TPCDSDataBenchmark-jdk21-results.txt
+++ b/core/benchmarks/LZ4TPCDSDataBenchmark-jdk21-results.txt
@@ -2,16 +2,16 @@
Benchmark LZ4CompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.17.0-1010-azure
AMD EPYC 7763 64-Core Processor
Compression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Compression 4 times 2376 2382
9 0.0 593993111.8 1.0X
+Compression 4 times 2342 2349
9 0.0 585571861.0 1.0X
-OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.17.0-1010-azure
AMD EPYC 7763 64-Core Processor
Decompression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Decompression 4 times 731 757
29 0.0 182787977.2 1.0X
+Decompression 4 times 748 776
24 0.0 187029239.5 1.0X
diff --git a/core/benchmarks/LZ4TPCDSDataBenchmark-jdk25-results.txt
b/core/benchmarks/LZ4TPCDSDataBenchmark-jdk25-results.txt
index 05898aa539e8..7a6736919653 100644
--- a/core/benchmarks/LZ4TPCDSDataBenchmark-jdk25-results.txt
+++ b/core/benchmarks/LZ4TPCDSDataBenchmark-jdk25-results.txt
@@ -2,16 +2,16 @@
Benchmark LZ4CompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 25.0.2+10-LTS on Linux 6.14.0-1017-azure
+OpenJDK 64-Bit Server VM 25.0.2+10-LTS on Linux 6.17.0-1010-azure
AMD EPYC 7763 64-Core Processor
Compression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Compression 4 times 2373 2380
10 0.0 593284150.8 1.0X
+Compression 4 times 2343 2353
15 0.0 585680834.0 1.0X
-OpenJDK 64-Bit Server VM 25.0.2+10-LTS on Linux 6.14.0-1017-azure
+OpenJDK 64-Bit Server VM 25.0.2+10-LTS on Linux 6.17.0-1010-azure
AMD EPYC 7763 64-Core Processor
Decompression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Decompression 4 times 694 710
23 0.0 173453930.5 1.0X
+Decompression 4 times 721 729
9 0.0 180327142.3 1.0X
diff --git a/core/benchmarks/LZ4TPCDSDataBenchmark-results.txt
b/core/benchmarks/LZ4TPCDSDataBenchmark-results.txt
index 94d8848325e0..6227a0f58b61 100644
--- a/core/benchmarks/LZ4TPCDSDataBenchmark-results.txt
+++ b/core/benchmarks/LZ4TPCDSDataBenchmark-results.txt
@@ -2,16 +2,16 @@
Benchmark LZ4CompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.14.0-1017-azure
-AMD EPYC 7763 64-Core Processor
+OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.17.0-1010-azure
+AMD EPYC 9V74 80-Core Processor
Compression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Compression 4 times 2376 2383
10 0.0 593917596.8 1.0X
+Compression 4 times 2543 2547
6 0.0 635681307.2 1.0X
-OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.14.0-1017-azure
-AMD EPYC 7763 64-Core Processor
+OpenJDK 64-Bit Server VM 17.0.18+8-LTS on Linux 6.17.0-1010-azure
+AMD EPYC 9V74 80-Core Processor
Decompression: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Decompression 4 times 759 761
2 0.0 189753707.8 1.0X
+Decompression 4 times 767 768
2 0.0 191709504.8 1.0X
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 2cf75c0982ce..daefcd85972e 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -189,7 +189,7 @@ log4j-api/2.25.4//log4j-api-2.25.4.jar
log4j-core/2.25.4//log4j-core-2.25.4.jar
log4j-layout-template-json/2.25.4//log4j-layout-template-json-2.25.4.jar
log4j-slf4j2-impl/2.25.4//log4j-slf4j2-impl-2.25.4.jar
-lz4-java/1.10.4//lz4-java-1.10.4.jar
+lz4-java/1.11.0//lz4-java-1.11.0.jar
metrics-core/4.2.37//metrics-core-4.2.37.jar
metrics-graphite/4.2.37//metrics-graphite-4.2.37.jar
metrics-jmx/4.2.37//metrics-jmx-4.2.37.jar
diff --git a/pom.xml b/pom.xml
index d6d40a08eeef..b17dfe7ebbff 100644
--- a/pom.xml
+++ b/pom.xml
@@ -889,7 +889,7 @@
<dependency>
<groupId>at.yawk.lz4</groupId>
<artifactId>lz4-java</artifactId>
- <version>1.10.4</version>
+ <version>1.11.0</version>
</dependency>
<dependency>
<groupId>com.github.luben</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
