This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 7b54d9c554fb [SPARK-49466][BUILD] Upgrade `zstd-jni` to 1.5.6-5
7b54d9c554fb is described below
commit 7b54d9c554fb0649c3ab961b95ee4fe8360b1c1a
Author: panbingkun <[email protected]>
AuthorDate: Fri Aug 30 09:19:35 2024 -0700
[SPARK-49466][BUILD] Upgrade `zstd-jni` to 1.5.6-5
### What changes were proposed in this pull request?
The pr aims to upgrade `zstd-jni` from `1.5.6-4` to `1.5.6-5`.
### Why are the changes needed?
1.v1.5.6-4 VS v1.5.6-5
https://github.com/luben/zstd-jni/compare/v1.5.6-4...v1.5.6-5
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #47934 from panbingkun/SPARK-49466.
Authored-by: panbingkun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../ZStandardBenchmark-jdk21-results.txt | 48 +++++++++++-----------
core/benchmarks/ZStandardBenchmark-results.txt | 48 +++++++++++-----------
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
4 files changed, 50 insertions(+), 50 deletions(-)
diff --git a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
index 40b999e7ee08..b3bffea826e5 100644
--- a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt
@@ -6,44 +6,44 @@ OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux
6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 654
667 12 0.0 65384.8 1.0X
-Compression 10000 times at level 2 without buffer pool 701
702 0 0.0 70133.1 0.9X
-Compression 10000 times at level 3 without buffer pool 798
799 1 0.0 79817.3 0.8X
-Compression 10000 times at level 1 with buffer pool 593
596 3 0.0 59339.9 1.1X
-Compression 10000 times at level 2 with buffer pool 629
634 7 0.0 62857.3 1.0X
-Compression 10000 times at level 3 with buffer pool 737
738 1 0.0 73690.9 0.9X
+Compression 10000 times at level 1 without buffer pool 657
670 14 0.0 65699.2 1.0X
+Compression 10000 times at level 2 without buffer pool 697
697 1 0.0 69673.4 0.9X
+Compression 10000 times at level 3 without buffer pool 799
802 3 0.0 79855.2 0.8X
+Compression 10000 times at level 1 with buffer pool 593
595 1 0.0 59326.9 1.1X
+Compression 10000 times at level 2 with buffer pool 622
624 3 0.0 62194.1 1.1X
+Compression 10000 times at level 3 with buffer pool 732
733 1 0.0 73178.6 0.9X
OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 821
822 2 0.0 82050.4 1.0X
-Decompression 10000 times from level 2 without buffer pool 820
821 1 0.0 82038.1 1.0X
-Decompression 10000 times from level 3 without buffer pool 817
819 2 0.0 81732.1 1.0X
-Decompression 10000 times from level 1 with buffer pool 745
746 1 0.0 74456.2 1.1X
-Decompression 10000 times from level 2 with buffer pool 746
747 1 0.0 74590.2 1.1X
-Decompression 10000 times from level 3 with buffer pool 746
747 1 0.0 74593.1 1.1X
+Decompression 10000 times from level 1 without buffer pool 813
820 11 0.0 81273.2 1.0X
+Decompression 10000 times from level 2 without buffer pool 810
813 3 0.0 80986.2 1.0X
+Decompression 10000 times from level 3 without buffer pool 812
813 2 0.0 81183.1 1.0X
+Decompression 10000 times from level 1 with buffer pool 746
747 2 0.0 74568.7 1.1X
+Decompression 10000 times from level 2 with buffer pool 744
746 2 0.0 74414.5 1.1X
+Decompression 10000 times from level 3 with buffer pool 745
746 1 0.0 74538.6 1.1X
OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 49 49
1 0.0 379018.5 1.0X
-Parallel Compression with 1 workers 35 37
4 0.0 271777.5 1.4X
-Parallel Compression with 2 workers 34 38
2 0.0 261820.6 1.4X
-Parallel Compression with 4 workers 37 39
2 0.0 285987.9 1.3X
-Parallel Compression with 8 workers 39 41
1 0.0 303005.9 1.3X
-Parallel Compression with 16 workers 43 45
1 0.0 337834.5 1.1X
+Parallel Compression with 0 workers 48 49
1 0.0 374256.1 1.0X
+Parallel Compression with 1 workers 34 36
3 0.0 267557.3 1.4X
+Parallel Compression with 2 workers 34 38
2 0.0 263684.3 1.4X
+Parallel Compression with 4 workers 37 39
2 0.0 289956.1 1.3X
+Parallel Compression with 8 workers 39 41
1 0.0 306975.2 1.2X
+Parallel Compression with 16 workers 44 45
1 0.0 340992.0 1.1X
OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 156 157
1 0.0 1215755.2 1.0X
-Parallel Compression with 1 workers 186 187
3 0.0 1449769.5 0.8X
-Parallel Compression with 2 workers 111 116
5 0.0 865458.3 1.4X
-Parallel Compression with 4 workers 105 110
3 0.0 821557.7 1.5X
-Parallel Compression with 8 workers 111 114
2 0.0 868777.0 1.4X
-Parallel Compression with 16 workers 110 115
2 0.0 859766.8 1.4X
+Parallel Compression with 0 workers 156 158
1 0.0 1220760.5 1.0X
+Parallel Compression with 1 workers 191 192
2 0.0 1495168.2 0.8X
+Parallel Compression with 2 workers 111 117
5 0.0 864459.9 1.4X
+Parallel Compression with 4 workers 106 109
2 0.0 831025.5 1.5X
+Parallel Compression with 8 workers 112 115
2 0.0 875732.7 1.4X
+Parallel Compression with 16 workers 110 114
2 0.0 858160.9 1.4X
diff --git a/core/benchmarks/ZStandardBenchmark-results.txt
b/core/benchmarks/ZStandardBenchmark-results.txt
index 6b67147e6a63..b230f825feca 100644
--- a/core/benchmarks/ZStandardBenchmark-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-results.txt
@@ -6,44 +6,44 @@ OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux
6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 641
649 10 0.0 64087.9 1.0X
-Compression 10000 times at level 2 without buffer pool 688
690 2 0.0 68761.5 0.9X
-Compression 10000 times at level 3 without buffer pool 777
777 1 0.0 77675.7 0.8X
-Compression 10000 times at level 1 with buffer pool 574
575 0 0.0 57407.8 1.1X
-Compression 10000 times at level 2 with buffer pool 604
605 1 0.0 60366.5 1.1X
-Compression 10000 times at level 3 with buffer pool 708
708 1 0.0 70794.2 0.9X
+Compression 10000 times at level 1 without buffer pool 638
638 0 0.0 63765.0 1.0X
+Compression 10000 times at level 2 without buffer pool 675
676 1 0.0 67529.4 0.9X
+Compression 10000 times at level 3 without buffer pool 775
783 11 0.0 77531.6 0.8X
+Compression 10000 times at level 1 with buffer pool 572
573 1 0.0 57223.2 1.1X
+Compression 10000 times at level 2 with buffer pool 603
605 1 0.0 60323.7 1.1X
+Compression 10000 times at level 3 with buffer pool 720
727 6 0.0 71980.9 0.9X
OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 585
586 1 0.0 58531.7 1.0X
-Decompression 10000 times from level 2 without buffer pool 585
587 2 0.0 58496.8 1.0X
-Decompression 10000 times from level 3 without buffer pool 588
589 1 0.0 58831.8 1.0X
-Decompression 10000 times from level 1 with buffer pool 533
534 1 0.0 53331.8 1.1X
-Decompression 10000 times from level 2 with buffer pool 533
534 0 0.0 53324.1 1.1X
-Decompression 10000 times from level 3 with buffer pool 533
534 0 0.0 53303.4 1.1X
+Decompression 10000 times from level 1 without buffer pool 584
585 1 0.0 58381.0 1.0X
+Decompression 10000 times from level 2 without buffer pool 585
585 0 0.0 58465.9 1.0X
+Decompression 10000 times from level 3 without buffer pool 585
586 1 0.0 58499.5 1.0X
+Decompression 10000 times from level 1 with buffer pool 534
534 0 0.0 53375.7 1.1X
+Decompression 10000 times from level 2 with buffer pool 533
533 0 0.0 53312.3 1.1X
+Decompression 10000 times from level 3 with buffer pool 533
533 1 0.0 53255.1 1.1X
OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 3: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 47 48
1 0.0 364123.8 1.0X
-Parallel Compression with 1 workers 34 36
3 0.0 268638.6 1.4X
-Parallel Compression with 2 workers 32 36
2 0.0 252026.9 1.4X
-Parallel Compression with 4 workers 35 38
4 0.0 271762.4 1.3X
-Parallel Compression with 8 workers 38 40
1 0.0 298137.9 1.2X
-Parallel Compression with 16 workers 42 44
1 0.0 324881.0 1.1X
+Parallel Compression with 0 workers 46 48
1 0.0 360483.5 1.0X
+Parallel Compression with 1 workers 34 36
2 0.0 265816.1 1.4X
+Parallel Compression with 2 workers 33 36
2 0.0 254525.8 1.4X
+Parallel Compression with 4 workers 34 37
1 0.0 266270.8 1.4X
+Parallel Compression with 8 workers 37 39
1 0.0 289289.2 1.2X
+Parallel Compression with 16 workers 41 43
1 0.0 320243.3 1.1X
OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
Parallel Compression at level 9: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parallel Compression with 0 workers 155 157
1 0.0 1210833.3 1.0X
-Parallel Compression with 1 workers 192 193
3 0.0 1500386.2 0.8X
-Parallel Compression with 2 workers 114 121
9 0.0 888645.9 1.4X
-Parallel Compression with 4 workers 106 109
2 0.0 830468.4 1.5X
-Parallel Compression with 8 workers 110 113
2 0.0 857123.0 1.4X
-Parallel Compression with 16 workers 109 114
3 0.0 854349.3 1.4X
+Parallel Compression with 0 workers 154 156
2 0.0 1205934.0 1.0X
+Parallel Compression with 1 workers 191 194
4 0.0 1495729.9 0.8X
+Parallel Compression with 2 workers 110 114
5 0.0 859158.9 1.4X
+Parallel Compression with 4 workers 105 108
3 0.0 822932.2 1.5X
+Parallel Compression with 8 workers 109 113
2 0.0 851560.0 1.4X
+Parallel Compression with 16 workers 111 115
2 0.0 870695.9 1.4X
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index e93e8e94a993..f48d2f2314f7 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -275,4 +275,4 @@ xz/1.10//xz-1.10.jar
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
zookeeper-jute/3.9.2//zookeeper-jute-3.9.2.jar
zookeeper/3.9.2//zookeeper-3.9.2.jar
-zstd-jni/1.5.6-4//zstd-jni-1.5.6-4.jar
+zstd-jni/1.5.6-5//zstd-jni-1.5.6-5.jar
diff --git a/pom.xml b/pom.xml
index 7a163f3d02dd..c0a6eb5fc783 100644
--- a/pom.xml
+++ b/pom.xml
@@ -816,7 +816,7 @@
<dependency>
<groupId>com.github.luben</groupId>
<artifactId>zstd-jni</artifactId>
- <version>1.5.6-4</version>
+ <version>1.5.6-5</version>
</dependency>
<dependency>
<groupId>com.clearspring.analytics</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
