This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ebb20a3871e [SPARK-43294][BUILD] Upgrade zstd-jni to 1.5.5-2
ebb20a3871e is described below
commit ebb20a3871e111df141634a8516799d8e52ddb03
Author: yangjie01 <[email protected]>
AuthorDate: Thu May 11 01:20:20 2023 -0700
[SPARK-43294][BUILD] Upgrade zstd-jni to 1.5.5-2
### What changes were proposed in this pull request?
This pr aims upgrade `zstd-jni` from 1.5.5-1 to 1.5.5-2.
### Why are the changes needed?
New version includes one new improvement `Added support for initialising a
"dict" from a direct ByteBuffer`:
- https://github.com/luben/zstd-jni/pull/255
Other changes as follows:
- https://github.com/luben/zstd-jni/compare/v1.5.5-1...v1.5.5-2
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
Closes #41135 from LuciferYang/SPARK-43294-2.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../ZStandardBenchmark-jdk11-results.txt | 32 +++++++++++-----------
.../ZStandardBenchmark-jdk17-results.txt | 32 +++++++++++-----------
core/benchmarks/ZStandardBenchmark-results.txt | 32 +++++++++++-----------
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
5 files changed, 50 insertions(+), 50 deletions(-)
diff --git a/core/benchmarks/ZStandardBenchmark-jdk11-results.txt
b/core/benchmarks/ZStandardBenchmark-jdk11-results.txt
index da037d631a2..96aa17e2138 100644
--- a/core/benchmarks/ZStandardBenchmark-jdk11-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-jdk11-results.txt
@@ -2,26 +2,26 @@
Benchmark ZStandardCompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 11.0.18+10 on Linux 5.15.0-1035-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+OpenJDK 64-Bit Server VM 11.0.18+10 on Linux 5.15.0-1036-azure
+Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 523
608 96 0.0 52324.2 1.0X
-Compression 10000 times at level 2 without buffer pool 597
598 1 0.0 59668.8 0.9X
-Compression 10000 times at level 3 without buffer pool 792
794 2 0.0 79185.9 0.7X
-Compression 10000 times at level 1 with buffer pool 332
333 1 0.0 33188.3 1.6X
-Compression 10000 times at level 2 with buffer pool 398
399 1 0.0 39798.4 1.3X
-Compression 10000 times at level 3 with buffer pool 589
590 1 0.0 58927.7 0.9X
+Compression 10000 times at level 1 without buffer pool 1074
1076 3 0.0 107421.2 1.0X
+Compression 10000 times at level 2 without buffer pool 1056
1056 0 0.0 105604.6 1.0X
+Compression 10000 times at level 3 without buffer pool 1297
1298 1 0.0 129748.3 0.8X
+Compression 10000 times at level 1 with buffer pool 305
317 13 0.0 30513.4 3.5X
+Compression 10000 times at level 2 with buffer pool 386
408 17 0.0 38550.2 2.8X
+Compression 10000 times at level 3 with buffer pool 603
605 2 0.0 60250.7 1.8X
-OpenJDK 64-Bit Server VM 11.0.18+10 on Linux 5.15.0-1035-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+OpenJDK 64-Bit Server VM 11.0.18+10 on Linux 5.15.0-1036-azure
+Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 722
722 1 0.0 72153.7 1.0X
-Decompression 10000 times from level 2 without buffer pool 723
724 2 0.0 72298.9 1.0X
-Decompression 10000 times from level 3 without buffer pool 722
722 1 0.0 72190.6 1.0X
-Decompression 10000 times from level 1 with buffer pool 530
531 1 0.0 53047.3 1.4X
-Decompression 10000 times from level 2 with buffer pool 529
530 0 0.0 52938.3 1.4X
-Decompression 10000 times from level 3 with buffer pool 530
531 1 0.0 52954.7 1.4X
+Decompression 10000 times from level 1 without buffer pool 1321
1330 14 0.0 132085.8 1.0X
+Decompression 10000 times from level 2 without buffer pool 1291
1308 25 0.0 129084.5 1.0X
+Decompression 10000 times from level 3 without buffer pool 1291
1291 0 0.0 129085.2 1.0X
+Decompression 10000 times from level 1 with buffer pool 966
966 0 0.0 96569.8 1.4X
+Decompression 10000 times from level 2 with buffer pool 970
984 13 0.0 96992.1 1.4X
+Decompression 10000 times from level 3 with buffer pool 965
1007 37 0.0 96530.0 1.4X
diff --git a/core/benchmarks/ZStandardBenchmark-jdk17-results.txt
b/core/benchmarks/ZStandardBenchmark-jdk17-results.txt
index 0aed86ae3ec..98e5dd24ce1 100644
--- a/core/benchmarks/ZStandardBenchmark-jdk17-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-jdk17-results.txt
@@ -2,26 +2,26 @@
Benchmark ZStandardCompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.6+10 on Linux 5.15.0-1035-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+OpenJDK 64-Bit Server VM 17.0.6+10 on Linux 5.15.0-1036-azure
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 2786
2787 1 0.0 278556.4 1.0X
-Compression 10000 times at level 2 without buffer pool 2826
2832 8 0.0 282635.1 1.0X
-Compression 10000 times at level 3 without buffer pool 2960
2961 1 0.0 296039.0 0.9X
-Compression 10000 times at level 1 with buffer pool 2648
2648 0 0.0 264758.0 1.1X
-Compression 10000 times at level 2 with buffer pool 2683
2684 0 0.0 268340.8 1.0X
-Compression 10000 times at level 3 with buffer pool 2806
2806 1 0.0 280608.1 1.0X
+Compression 10000 times at level 1 without buffer pool 2285
2286 1 0.0 228515.4 1.0X
+Compression 10000 times at level 2 without buffer pool 2206
2540 473 0.0 220612.4 1.0X
+Compression 10000 times at level 3 without buffer pool 2396
2402 8 0.0 239645.1 1.0X
+Compression 10000 times at level 1 with buffer pool 2054
2056 2 0.0 205401.6 1.1X
+Compression 10000 times at level 2 with buffer pool 2104
2107 5 0.0 210352.6 1.1X
+Compression 10000 times at level 3 with buffer pool 2302
2306 6 0.0 230178.4 1.0X
-OpenJDK 64-Bit Server VM 17.0.6+10 on Linux 5.15.0-1035-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+OpenJDK 64-Bit Server VM 17.0.6+10 on Linux 5.15.0-1036-azure
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 2742
2744 3 0.0 274226.6 1.0X
-Decompression 10000 times from level 2 without buffer pool 2739
2742 4 0.0 273917.2 1.0X
-Decompression 10000 times from level 3 without buffer pool 2745
2746 2 0.0 274476.4 1.0X
-Decompression 10000 times from level 1 with buffer pool 2612
2613 1 0.0 261204.1 1.0X
-Decompression 10000 times from level 2 with buffer pool 2616
2620 5 0.0 261623.1 1.0X
-Decompression 10000 times from level 3 with buffer pool 2614
2621 11 0.0 261357.8 1.0X
+Decompression 10000 times from level 1 without buffer pool 2120
2124 5 0.0 212043.2 1.0X
+Decompression 10000 times from level 2 without buffer pool 2126
2130 6 0.0 212560.3 1.0X
+Decompression 10000 times from level 3 without buffer pool 2122
2124 3 0.0 212227.1 1.0X
+Decompression 10000 times from level 1 with buffer pool 1958
1959 2 0.0 195798.8 1.1X
+Decompression 10000 times from level 2 with buffer pool 1959
1960 2 0.0 195918.1 1.1X
+Decompression 10000 times from level 3 with buffer pool 1965
1968 3 0.0 196533.7 1.1X
diff --git a/core/benchmarks/ZStandardBenchmark-results.txt
b/core/benchmarks/ZStandardBenchmark-results.txt
index c69fed84f77..d9e6059e027 100644
--- a/core/benchmarks/ZStandardBenchmark-results.txt
+++ b/core/benchmarks/ZStandardBenchmark-results.txt
@@ -2,26 +2,26 @@
Benchmark ZStandardCompressionCodec
================================================================================================
-OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1036-azure
+Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Benchmark ZStandardCompressionCodec: Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------
-Compression 10000 times at level 1 without buffer pool 394
527 151 0.0 39420.6 1.0X
-Compression 10000 times at level 2 without buffer pool 453
455 3 0.0 45251.2 0.9X
-Compression 10000 times at level 3 without buffer pool 640
645 3 0.0 64045.8 0.6X
-Compression 10000 times at level 1 with buffer pool 193
197 3 0.1 19278.3 2.0X
-Compression 10000 times at level 2 with buffer pool 244
251 3 0.0 24416.2 1.6X
-Compression 10000 times at level 3 with buffer pool 420
436 10 0.0 42020.0 0.9X
+Compression 10000 times at level 1 without buffer pool 815
854 39 0.0 81531.3 1.0X
+Compression 10000 times at level 2 without buffer pool 744
750 5 0.0 74444.2 1.1X
+Compression 10000 times at level 3 without buffer pool 1005
1009 5 0.0 100524.4 0.8X
+Compression 10000 times at level 1 with buffer pool 384
388 3 0.0 38388.7 2.1X
+Compression 10000 times at level 2 with buffer pool 480
494 12 0.0 48017.6 1.7X
+Compression 10000 times at level 3 with buffer pool 732
753 26 0.0 73192.6 1.1X
-OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1036-azure
+Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
Benchmark ZStandardCompressionCodec: Best Time(ms)
Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------
-Decompression 10000 times from level 1 without buffer pool 612
615 3 0.0 61241.3 1.0X
-Decompression 10000 times from level 2 without buffer pool 611
615 3 0.0 61124.5 1.0X
-Decompression 10000 times from level 3 without buffer pool 614
617 2 0.0 61402.7 1.0X
-Decompression 10000 times from level 1 with buffer pool 423
424 1 0.0 42257.6 1.4X
-Decompression 10000 times from level 2 with buffer pool 423
424 1 0.0 42305.5 1.4X
-Decompression 10000 times from level 3 with buffer pool 423
424 2 0.0 42259.3 1.4X
+Decompression 10000 times from level 1 without buffer pool 800
808 8 0.0 79950.8 1.0X
+Decompression 10000 times from level 2 without buffer pool 808
821 18 0.0 80750.5 1.0X
+Decompression 10000 times from level 3 without buffer pool 820
831 11 0.0 82019.0 1.0X
+Decompression 10000 times from level 1 with buffer pool 550
561 9 0.0 54984.8 1.5X
+Decompression 10000 times from level 2 with buffer pool 554
560 5 0.0 55370.0 1.4X
+Decompression 10000 times from level 3 with buffer pool 553
569 14 0.0 55272.1 1.4X
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 5174b6b0335..539876ac75f 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -254,4 +254,4 @@ xz/1.9//xz-1.9.jar
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
zookeeper-jute/3.6.3//zookeeper-jute-3.6.3.jar
zookeeper/3.6.3//zookeeper-3.6.3.jar
-zstd-jni/1.5.5-1//zstd-jni-1.5.5-1.jar
+zstd-jni/1.5.5-2//zstd-jni-1.5.5-2.jar
diff --git a/pom.xml b/pom.xml
index 04260b51c60..8e3a77033ed 100644
--- a/pom.xml
+++ b/pom.xml
@@ -788,7 +788,7 @@
<dependency>
<groupId>com.github.luben</groupId>
<artifactId>zstd-jni</artifactId>
- <version>1.5.5-1</version>
+ <version>1.5.5-2</version>
</dependency>
<dependency>
<groupId>com.clearspring.analytics</groupId>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
