This is an automated email from the ASF dual-hosted git repository. dongjoon pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push: new 354a5374208 [SPARK-46035][BUILD] Upgrade zstd-jni to 1.5.5-10 354a5374208 is described below commit 354a53742086c39f7de7b97ec1d2d5e58f6d7b3e Author: yangjie01 <yangji...@baidu.com> AuthorDate: Tue Nov 21 21:36:35 2023 -0800 [SPARK-46035][BUILD] Upgrade zstd-jni to 1.5.5-10 ### What changes were proposed in this pull request? The pr aims to upgrade zstd-jni from 1.5.5-7 to 1.5.5-10. ### Why are the changes needed? This version solved some issues related to OSGi, such as https://github.com/luben/zstd-jni/commit/d5bc6a093ca909dd978fac47d35412d70b870da2, https://github.com/luben/zstd-jni/commit/39c069776992812751402c7cf87cb3e89545b12d and https://github.com/luben/zstd-jni/commit/89c7dcb6b2c21c288e76b31dd567f0c71296facf. The version compare as follows: - https://github.com/luben/zstd-jni/compare/v1.5.5-7...v1.5.5-9 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GitHub Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes #43937 from LuciferYang/SPARK-46035. Authored-by: yangjie01 <yangji...@baidu.com> Signed-off-by: Dongjoon Hyun <dh...@apple.com> --- .../ZStandardBenchmark-jdk21-results.txt | 32 +++++++++++----------- core/benchmarks/ZStandardBenchmark-results.txt | 32 +++++++++++----------- dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +- pom.xml | 2 +- 4 files changed, 34 insertions(+), 34 deletions(-) diff --git a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt index 19bd2faa9a5..9f06fffc508 100644 --- a/core/benchmarks/ZStandardBenchmark-jdk21-results.txt +++ b/core/benchmarks/ZStandardBenchmark-jdk21-results.txt @@ -2,26 +2,26 @@ Benchmark ZStandardCompressionCodec ================================================================================================ -OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1050-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1051-azure +AMD EPYC 7763 64-Core Processor Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative -------------------------------------------------------------------------------------------------------------------------------------- -Compression 10000 times at level 1 without buffer pool 1817 1819 3 0.0 181709.6 1.0X -Compression 10000 times at level 2 without buffer pool 2081 2083 3 0.0 208053.7 0.9X -Compression 10000 times at level 3 without buffer pool 2288 2290 3 0.0 228795.4 0.8X -Compression 10000 times at level 1 with buffer pool 1997 1998 1 0.0 199686.9 0.9X -Compression 10000 times at level 2 with buffer pool 2062 2063 1 0.0 206209.3 0.9X -Compression 10000 times at level 3 with buffer pool 2243 2243 1 0.0 224271.8 0.8X +Compression 10000 times at level 1 without buffer pool 675 925 281 0.0 67464.0 1.0X +Compression 10000 times at level 2 without buffer pool 901 902 3 0.0 90079.7 0.7X +Compression 10000 times at level 3 without buffer pool 994 998 6 0.0 99413.5 0.7X +Compression 10000 times at level 1 with buffer pool 813 813 0 0.0 81311.2 0.8X +Compression 10000 times at level 2 with buffer pool 873 874 0 0.0 87335.4 0.8X +Compression 10000 times at level 3 with buffer pool 987 987 1 0.0 98671.2 0.7X -OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1050-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 21.0.1+12-LTS on Linux 5.15.0-1051-azure +AMD EPYC 7763 64-Core Processor Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------ -Decompression 10000 times from level 1 without buffer pool 1980 1981 1 0.0 197970.8 1.0X -Decompression 10000 times from level 2 without buffer pool 1978 1979 1 0.0 197813.4 1.0X -Decompression 10000 times from level 3 without buffer pool 1981 1983 2 0.0 198141.7 1.0X -Decompression 10000 times from level 1 with buffer pool 1825 1827 3 0.0 182475.2 1.1X -Decompression 10000 times from level 2 with buffer pool 1827 1827 0 0.0 182667.1 1.1X -Decompression 10000 times from level 3 with buffer pool 1826 1826 0 0.0 182579.8 1.1X +Decompression 10000 times from level 1 without buffer pool 826 829 3 0.0 82582.8 1.0X +Decompression 10000 times from level 2 without buffer pool 827 829 2 0.0 82728.9 1.0X +Decompression 10000 times from level 3 without buffer pool 821 823 2 0.0 82099.0 1.0X +Decompression 10000 times from level 1 with buffer pool 755 755 1 0.0 75470.0 1.1X +Decompression 10000 times from level 2 with buffer pool 756 758 3 0.0 75579.0 1.1X +Decompression 10000 times from level 3 with buffer pool 755 755 0 0.0 75526.6 1.1X diff --git a/core/benchmarks/ZStandardBenchmark-results.txt b/core/benchmarks/ZStandardBenchmark-results.txt index 6f7efdd6c94..2c5e894ca78 100644 --- a/core/benchmarks/ZStandardBenchmark-results.txt +++ b/core/benchmarks/ZStandardBenchmark-results.txt @@ -2,26 +2,26 @@ Benchmark ZStandardCompressionCodec ================================================================================================ -OpenJDK 64-Bit Server VM 17.0.9+8-LTS on Linux 5.15.0-1050-azure -Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz +OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1051-azure +AMD EPYC 7763 64-Core Processor Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative -------------------------------------------------------------------------------------------------------------------------------------- -Compression 10000 times at level 1 without buffer pool 2786 2787 2 0.0 278560.1 1.0X -Compression 10000 times at level 2 without buffer pool 2831 2833 3 0.0 283091.0 1.0X -Compression 10000 times at level 3 without buffer pool 2958 2959 2 0.0 295806.3 0.9X -Compression 10000 times at level 1 with buffer pool 211 214 4 0.0 21145.3 13.2X -Compression 10000 times at level 2 with buffer pool 253 255 1 0.0 25328.1 11.0X -Compression 10000 times at level 3 with buffer pool 370 371 1 0.0 37046.1 7.5X +Compression 10000 times at level 1 without buffer pool 657 659 2 0.0 65716.9 1.0X +Compression 10000 times at level 2 without buffer pool 706 707 1 0.0 70617.1 0.9X +Compression 10000 times at level 3 without buffer pool 788 788 0 0.0 78755.6 0.8X +Compression 10000 times at level 1 with buffer pool 585 586 1 0.0 58455.1 1.1X +Compression 10000 times at level 2 with buffer pool 614 616 1 0.0 61437.2 1.1X +Compression 10000 times at level 3 with buffer pool 717 717 0 0.0 71705.1 0.9X -OpenJDK 64-Bit Server VM 17.0.9+8-LTS on Linux 5.15.0-1050-azure -Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz +OpenJDK 64-Bit Server VM 17.0.9+9-LTS on Linux 5.15.0-1051-azure +AMD EPYC 7763 64-Core Processor Benchmark ZStandardCompressionCodec: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative ------------------------------------------------------------------------------------------------------------------------------------------ -Decompression 10000 times from level 1 without buffer pool 2745 2748 5 0.0 274454.0 1.0X -Decompression 10000 times from level 2 without buffer pool 2744 2745 1 0.0 274438.1 1.0X -Decompression 10000 times from level 3 without buffer pool 2746 2746 1 0.0 274586.0 1.0X -Decompression 10000 times from level 1 with buffer pool 2587 2588 1 0.0 258707.4 1.1X -Decompression 10000 times from level 2 with buffer pool 2586 2586 1 0.0 258566.8 1.1X -Decompression 10000 times from level 3 with buffer pool 2589 2589 0 0.0 258870.6 1.1X +Decompression 10000 times from level 1 without buffer pool 595 596 0 0.0 59535.3 1.0X +Decompression 10000 times from level 2 without buffer pool 594 595 1 0.0 59432.7 1.0X +Decompression 10000 times from level 3 without buffer pool 595 600 10 0.0 59485.8 1.0X +Decompression 10000 times from level 1 with buffer pool 549 549 1 0.0 54859.6 1.1X +Decompression 10000 times from level 2 with buffer pool 549 550 0 0.0 54948.0 1.1X +Decompression 10000 times from level 3 with buffer pool 549 549 0 0.0 54917.9 1.1X diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3 b/dev/deps/spark-deps-hadoop-3-hive-2.3 index afde3307c62..df712acdca5 100644 --- a/dev/deps/spark-deps-hadoop-3-hive-2.3 +++ b/dev/deps/spark-deps-hadoop-3-hive-2.3 @@ -265,4 +265,4 @@ xz/1.9//xz-1.9.jar zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar zookeeper-jute/3.9.1//zookeeper-jute-3.9.1.jar zookeeper/3.9.1//zookeeper-3.9.1.jar -zstd-jni/1.5.5-7//zstd-jni-1.5.5-7.jar +zstd-jni/1.5.5-10//zstd-jni-1.5.5-10.jar diff --git a/pom.xml b/pom.xml index 23cd04c205f..8b5d4eff78f 100644 --- a/pom.xml +++ b/pom.xml @@ -800,7 +800,7 @@ <dependency> <groupId>com.github.luben</groupId> <artifactId>zstd-jni</artifactId> - <version>1.5.5-7</version> + <version>1.5.5-10</version> </dependency> <dependency> <groupId>com.clearspring.analytics</groupId> --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org