This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new cde8e4a82e20 [SPARK-49707][BUILD] Upgrade rocksdbjni to 9.7.3
cde8e4a82e20 is described below
commit cde8e4a82e20a363861f451ebd5138efb3194ab8
Author: panbingkun <panbing...@baidu.com>
AuthorDate: Mon Oct 28 20:05:54 2024 +0800
[SPARK-49707][BUILD] Upgrade rocksdbjni to 9.7.3
### What changes were proposed in this pull request?
The pr aims to upgrade rocksdbjni from `9.5.2` to `9.7.3`.
### Why are the changes needed?
The full release notes as follows:
https://github.com/facebook/rocksdb/releases/tag/v9.7.3
https://github.com/facebook/rocksdb/releases/tag/v9.6.1
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #48155 from panbingkun/SPARK-49707.
Authored-by: panbingkun <panbing...@baidu.com>
Signed-off-by: yangjie01 <yangji...@baidu.com>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
...StoreBasicOperationsBenchmark-jdk21-results.txt | 118 ++++++++++----------
.../StateStoreBasicOperationsBenchmark-results.txt | 120 ++++++++++-----------
4 files changed, 121 insertions(+), 121 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index af335cb89cfb..252a2ebfbd9f 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -250,7 +250,7 @@ parquet-jackson/1.14.3//parquet-jackson-1.14.3.jar
pickle/1.5//pickle-1.5.jar
py4j/0.10.9.7//py4j-0.10.9.7.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
-rocksdbjni/9.5.2//rocksdbjni-9.5.2.jar
+rocksdbjni/9.7.3//rocksdbjni-9.7.3.jar
scala-collection-compat_2.13/2.7.0//scala-collection-compat_2.13-2.7.0.jar
scala-compiler/2.13.15//scala-compiler-2.13.15.jar
scala-library/2.13.15//scala-library-2.13.15.jar
diff --git a/pom.xml b/pom.xml
index 04e07197fa4b..b3171310c705 100644
--- a/pom.xml
+++ b/pom.xml
@@ -726,7 +726,7 @@
<dependency>
<groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId>
- <version>9.5.2</version>
+ <version>9.7.3</version>
</dependency>
<dependency>
<groupId>${leveldbjni.group}</groupId>
diff --git
a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
index caca60875a8f..4ab5f6d0061c 100644
--- a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
+++ b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk21-results.txt
@@ -2,143 +2,143 @@
put rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (10000 rows to overwrite - rate 100): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
11 1 1.1 943.6 1.0X
-RocksDB (trackTotalNumberOfRows: true) 43
45 2 0.2 4332.8 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
17 1 0.6 1583.2 0.6X
+In-memory 10
14 1 1.0 1006.5 1.0X
+RocksDB (trackTotalNumberOfRows: true) 43
45 2 0.2 4345.4 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
17 1 0.6 1547.6 0.7X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
11 1 1.1 938.2 1.0X
-RocksDB (trackTotalNumberOfRows: true) 45
46 1 0.2 4452.3 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
17 1 0.6 1586.1 0.6X
+In-memory 10
12 1 1.0 1011.1 1.0X
+RocksDB (trackTotalNumberOfRows: true) 44
46 1 0.2 4441.2 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1521.7 0.7X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 920.7 1.0X
-RocksDB (trackTotalNumberOfRows: true) 45
46 1 0.2 4478.9 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
17 1 0.6 1581.1 0.6X
+In-memory 9
10 1 1.1 940.8 1.0X
+RocksDB (trackTotalNumberOfRows: true) 44
46 1 0.2 4425.1 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1515.2 0.6X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (0 rows to overwrite - rate 0): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 912.4 1.0X
-RocksDB (trackTotalNumberOfRows: true) 44
46 1 0.2 4445.8 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
17 0 0.6 1587.8 0.6X
+In-memory 9
11 2 1.1 932.2 1.0X
+RocksDB (trackTotalNumberOfRows: true) 44
46 1 0.2 4400.3 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
17 1 0.7 1506.0 0.6X
================================================================================================
merge rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (10000 rows to overwrite - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
537 551 7 0.0 53664.8 1.0X
-RocksDB (trackTotalNumberOfRows: false)
173 178 4 0.1 17277.2 3.1X
+RocksDB (trackTotalNumberOfRows: true)
532 547 8 0.0 53154.1 1.0X
+RocksDB (trackTotalNumberOfRows: false)
174 180 3 0.1 17410.5 3.1X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (5000 rows to overwrite - rate 50):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
474 486 5 0.0 47389.2 1.0X
-RocksDB (trackTotalNumberOfRows: false)
172 177 2 0.1 17189.8 2.8X
+RocksDB (trackTotalNumberOfRows: true)
472 484 5 0.0 47228.8 1.0X
+RocksDB (trackTotalNumberOfRows: false)
174 179 3 0.1 17433.5 2.7X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (1000 rows to overwrite - rate 10):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
417 430 6 0.0 41696.3 1.0X
-RocksDB (trackTotalNumberOfRows: false)
175 180 3 0.1 17458.6 2.4X
+RocksDB (trackTotalNumberOfRows: true)
422 434 5 0.0 42226.0 1.0X
+RocksDB (trackTotalNumberOfRows: false)
172 179 3 0.1 17235.9 2.4X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (0 rows to overwrite - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
400 412 5 0.0 39958.9 1.0X
-RocksDB (trackTotalNumberOfRows: false)
170 175 4 0.1 16952.8 2.4X
+RocksDB (trackTotalNumberOfRows: true)
406 419 7 0.0 40646.7 1.0X
+RocksDB (trackTotalNumberOfRows: false)
173 179 3 0.1 17265.8 2.4X
================================================================================================
delete rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(10000 rows are non-existing - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
In-memory
0 1 0 27.0 37.0
1.0X
-RocksDB (trackTotalNumberOfRows: true)
43 44 1 0.2 4315.2
0.0X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1489.0
0.0X
+RocksDB (trackTotalNumberOfRows: true)
44 46 1 0.2 4447.0
0.0X
+RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1453.0
0.0X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(5000 rows are non-existing - rate
50): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 8 0 1.3 781.5
1.0X
-RocksDB (trackTotalNumberOfRows: true)
43 44 1 0.2 4323.9
0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1500.1
0.5X
+In-memory
8 9 1 1.3 796.5
1.0X
+RocksDB (trackTotalNumberOfRows: true)
44 45 1 0.2 4384.0
0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1463.5
0.5X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(1000 rows are non-existing - rate
10): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 1 1.2 829.7
1.0X
-RocksDB (trackTotalNumberOfRows: true)
42 43 1 0.2 4234.1
0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1491.0
0.6X
+In-memory
9 9 1 1.2 853.3
1.0X
+RocksDB (trackTotalNumberOfRows: true)
43 44 1 0.2 4278.0
0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 15 1 0.7 1460.7
0.6X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(0 rows are non-existing - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.2 838.7 1.0X
-RocksDB (trackTotalNumberOfRows: true)
42 43 1 0.2 4185.5 0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1485.0 0.6X
+In-memory
9 10 2 1.2 854.0 1.0X
+RocksDB (trackTotalNumberOfRows: true)
42 44 1 0.2 4183.1 0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1457.0 0.6X
================================================================================================
evict rows
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 10000 rows (maxTimestampToEvictInMillis: 9999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.2 832.1 1.0X
-RocksDB (trackTotalNumberOfRows: true)
41 42 1 0.2 4142.6 0.2X
-RocksDB (trackTotalNumberOfRows: false)
16 16 1 0.6 1621.2 0.5X
+In-memory
8 9 0 1.2 837.4 1.0X
+RocksDB (trackTotalNumberOfRows: true)
41 42 1 0.2 4146.8 0.2X
+RocksDB (trackTotalNumberOfRows: false)
16 17 1 0.6 1623.1 0.5X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 5000 rows (maxTimestampToEvictInMillis: 4999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 8 0 1.3 783.9 1.0X
-RocksDB (trackTotalNumberOfRows: true)
22 23 1 0.4 2226.5 0.4X
-RocksDB (trackTotalNumberOfRows: false)
10 10 0 1.0 969.3 0.8X
+In-memory
8 9 1 1.3 798.1 1.0X
+RocksDB (trackTotalNumberOfRows: true)
22 23 1 0.5 2201.4 0.4X
+RocksDB (trackTotalNumberOfRows: false)
10 10 1 1.0 956.5 0.8X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 1000 rows (maxTimestampToEvictInMillis: 999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 0 1.4 714.1 1.0X
-RocksDB (trackTotalNumberOfRows: true)
7 7 1 1.4 725.5 1.0X
-RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.1 476.0 1.5X
+In-memory
7 8 1 1.4 724.1 1.0X
+RocksDB (trackTotalNumberOfRows: true)
7 7 0 1.4 698.4 1.0X
+RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.2 450.9 1.6X
-OpenJDK 64-Bit Server VM 21.0.4+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 21.0.5+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 0 rows (maxTimestampToEvictInMillis: -1) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 0 0 24.1 41.5 1.0X
-RocksDB (trackTotalNumberOfRows: true)
3 4 0 2.9 343.6 0.1X
-RocksDB (trackTotalNumberOfRows: false)
3 4 0 2.9 343.8 0.1X
+In-memory
0 0 0 24.0 41.6 1.0X
+RocksDB (trackTotalNumberOfRows: true)
3 3 1 3.2 317.3 0.1X
+RocksDB (trackTotalNumberOfRows: false)
3 3 0 3.2 317.2 0.1X
diff --git a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
index 378cecf0271d..856985b5d071 100644
--- a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
+++ b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
@@ -2,143 +2,143 @@
put rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (10000 rows to overwrite - rate 100): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 948.8 1.0X
-RocksDB (trackTotalNumberOfRows: true) 43
44 2 0.2 4286.1 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
16 1 0.6 1556.1 0.6X
+In-memory 10
10 1 1.0 953.1 1.0X
+RocksDB (trackTotalNumberOfRows: true) 43
44 2 0.2 4269.8 0.2X
+RocksDB (trackTotalNumberOfRows: false) 16
16 1 0.6 1550.5 0.6X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 0 1.1 937.8 1.0X
-RocksDB (trackTotalNumberOfRows: true) 44
45 1 0.2 4368.4 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
16 1 0.6 1559.7 0.6X
+In-memory 9
10 0 1.1 930.0 1.0X
+RocksDB (trackTotalNumberOfRows: true) 44
45 1 0.2 4387.9 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1521.4 0.6X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 0 1.1 921.3 1.0X
-RocksDB (trackTotalNumberOfRows: true) 44
45 1 0.2 4406.5 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
16 1 0.6 1555.2 0.6X
+In-memory 9
10 0 1.1 918.1 1.0X
+RocksDB (trackTotalNumberOfRows: true) 44
45 1 0.2 4441.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 1 0.7 1521.7 0.6X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
putting 10000 rows (0 rows to overwrite - rate 0): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 0 1.1 918.1 1.0X
-RocksDB (trackTotalNumberOfRows: true) 44
45 1 0.2 4372.7 0.2X
-RocksDB (trackTotalNumberOfRows: false) 16
16 0 0.6 1559.8 0.6X
+In-memory 9
10 0 1.1 916.9 1.0X
+RocksDB (trackTotalNumberOfRows: true) 44
45 1 0.2 4413.7 0.2X
+RocksDB (trackTotalNumberOfRows: false) 15
16 0 0.7 1522.0 0.6X
================================================================================================
merge rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (10000 rows to overwrite - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
542 555 7 0.0 54234.9 1.0X
-RocksDB (trackTotalNumberOfRows: false)
179 185 3 0.1 17909.4 3.0X
+RocksDB (trackTotalNumberOfRows: true)
542 553 6 0.0 54222.4 1.0X
+RocksDB (trackTotalNumberOfRows: false)
174 179 3 0.1 17391.9 3.1X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (5000 rows to overwrite - rate 50):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
474 487 5 0.0 47434.9 1.0X
-RocksDB (trackTotalNumberOfRows: false)
180 184 3 0.1 17961.1 2.6X
+RocksDB (trackTotalNumberOfRows: true)
479 490 5 0.0 47921.1 1.0X
+RocksDB (trackTotalNumberOfRows: false)
174 179 3 0.1 17446.2 2.7X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (1000 rows to overwrite - rate 10):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
419 428 4 0.0 41901.9 1.0X
-RocksDB (trackTotalNumberOfRows: false)
175 181 2 0.1 17545.5 2.4X
+RocksDB (trackTotalNumberOfRows: true)
423 433 5 0.0 42311.4 1.0X
+RocksDB (trackTotalNumberOfRows: false)
173 178 3 0.1 17309.1 2.4X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
merging 10000 rows with 10 values per key (0 rows to overwrite - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------------------------------------
-RocksDB (trackTotalNumberOfRows: true)
400 410 5 0.0 39961.3 1.0X
-RocksDB (trackTotalNumberOfRows: false)
175 182 3 0.1 17527.9 2.3X
+RocksDB (trackTotalNumberOfRows: true)
408 419 5 0.0 40762.3 1.0X
+RocksDB (trackTotalNumberOfRows: false)
174 183 3 0.1 17377.7 2.3X
================================================================================================
delete rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(10000 rows are non-existing - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 1 0 25.7 38.9
1.0X
-RocksDB (trackTotalNumberOfRows: true)
43 45 1 0.2 4347.8
0.0X
-RocksDB (trackTotalNumberOfRows: false)
15 16 0 0.7 1495.4
0.0X
+In-memory
0 0 0 26.1 38.3
1.0X
+RocksDB (trackTotalNumberOfRows: true)
44 46 1 0.2 4444.2
0.0X
+RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1489.6
0.0X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(5000 rows are non-existing - rate
50): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.3 789.7
1.0X
-RocksDB (trackTotalNumberOfRows: true)
44 45 1 0.2 4360.4
0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1502.6
0.5X
+In-memory
8 8 0 1.3 788.8
1.0X
+RocksDB (trackTotalNumberOfRows: true)
44 45 1 0.2 4425.4
0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1499.2
0.5X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(1000 rows are non-existing - rate
10): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.2 833.1
1.0X
-RocksDB (trackTotalNumberOfRows: true)
43 44 1 0.2 4274.2
0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1499.0
0.6X
+In-memory
8 9 0 1.2 841.3
1.0X
+RocksDB (trackTotalNumberOfRows: true)
43 44 1 0.2 4336.9
0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 16 1 0.7 1493.6
0.6X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
trying to delete 10000 rows from 10000 rows(0 rows are non-existing - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.2 845.5 1.0X
-RocksDB (trackTotalNumberOfRows: true)
42 43 1 0.2 4220.8 0.2X
-RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1479.2 0.6X
+In-memory
8 9 0 1.2 848.9 1.0X
+RocksDB (trackTotalNumberOfRows: true)
42 43 1 0.2 4216.8 0.2X
+RocksDB (trackTotalNumberOfRows: false)
15 15 0 0.7 1467.4 0.6X
================================================================================================
evict rows
================================================================================================
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 10000 rows (maxTimestampToEvictInMillis: 9999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 9 0 1.2 848.5 1.0X
-RocksDB (trackTotalNumberOfRows: true)
42 43 0 0.2 4184.8 0.2X
-RocksDB (trackTotalNumberOfRows: false)
16 16 0 0.6 1592.8 0.5X
+In-memory
8 9 0 1.2 836.6 1.0X
+RocksDB (trackTotalNumberOfRows: true)
42 43 2 0.2 4182.0 0.2X
+RocksDB (trackTotalNumberOfRows: false)
16 17 0 0.6 1645.0 0.5X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 5000 rows (maxTimestampToEvictInMillis: 4999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
8 8 0 1.3 792.3 1.0X
-RocksDB (trackTotalNumberOfRows: true)
23 23 1 0.4 2267.5 0.3X
-RocksDB (trackTotalNumberOfRows: false)
10 10 0 1.0 983.7 0.8X
+In-memory
8 8 0 1.3 785.1 1.0X
+RocksDB (trackTotalNumberOfRows: true)
23 23 1 0.4 2258.3 0.3X
+RocksDB (trackTotalNumberOfRows: false)
10 10 0 1.0 999.7 0.8X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 1000 rows (maxTimestampToEvictInMillis: 999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 0 1.4 737.0 1.0X
-RocksDB (trackTotalNumberOfRows: true)
7 8 0 1.3 742.5 1.0X
-RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.1 486.5 1.5X
+In-memory
7 8 0 1.4 726.0 1.0X
+RocksDB (trackTotalNumberOfRows: true)
7 8 0 1.4 736.8 1.0X
+RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.1 487.0 1.5X
-OpenJDK 64-Bit Server VM 17.0.12+7-LTS on Linux 6.5.0-1025-azure
+OpenJDK 64-Bit Server VM 17.0.13+11-LTS on Linux 6.5.0-1025-azure
AMD EPYC 7763 64-Core Processor
evicting 0 rows (maxTimestampToEvictInMillis: -1) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 0 0 21.4 46.6 1.0X
-RocksDB (trackTotalNumberOfRows: true)
4 4 0 2.8 354.6 0.1X
-RocksDB (trackTotalNumberOfRows: false)
4 4 0 2.8 353.8 0.1X
+In-memory
0 0 0 22.8 43.9 1.0X
+RocksDB (trackTotalNumberOfRows: true)
4 4 0 2.8 354.8 0.1X
+RocksDB (trackTotalNumberOfRows: false)
4 4 0 2.8 353.1 0.1X
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
