This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e8c347b5499 [SPARK-43436][BUILD] Upgrade rocksdbjni to 8.1.1.1
e8c347b5499 is described below
commit e8c347b549901bec5c0a1f5d5d3e5430adbe69b5
Author: YangJie <yangji...@baidu.com>
AuthorDate: Wed May 17 13:30:08 2023 -0700
[SPARK-43436][BUILD] Upgrade rocksdbjni to 8.1.1.1
### What changes were proposed in this pull request?
This pr aims to upgrade rocksdbjni from 8.0.0 to 8.1.1.1.
### Why are the changes needed?
The full release notes as follows:
- https://github.com/facebook/rocksdb/releases/tag/v8.1.1
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- Manual test `RocksDBBenchmark`:
**8.0.0**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max
95th
dbClose 4 0.339 0.256
0.532 0.532
dbCreation 4 69.637 4.128
268.371 268.371
naturalIndexCreateIterator 1024 0.005 0.002
1.581 0.006
naturalIndexDescendingCreateIterator 1024 0.005 0.005
0.067 0.007
naturalIndexDescendingIteration 1024 0.006 0.004
0.029 0.008
naturalIndexIteration 1024 0.006 0.004
0.069 0.009
randomDeleteIndexed 1024 0.026 0.018
0.200 0.035
randomDeletesNoIndex 1024 0.015 0.012
0.050 0.018
randomUpdatesIndexed 1024 0.088 0.033
33.234 0.096
randomUpdatesNoIndex 1024 0.035 0.032
0.497 0.039
randomWritesIndexed 1024 0.129 0.035
56.362 0.121
randomWritesNoIndex 1024 0.041 0.034
1.415 0.047
refIndexCreateIterator 1024 0.005 0.004
0.029 0.006
refIndexDescendingCreateIterator 1024 0.003 0.003
0.028 0.004
refIndexDescendingIteration 1024 0.007 0.005
0.044 0.008
refIndexIteration 1024 0.008 0.005
0.367 0.010
sequentialDeleteIndexed 1024 0.023 0.017
1.716 0.025
sequentialDeleteNoIndex 1024 0.015 0.012
0.043 0.017
sequentialUpdatesIndexed 1024 0.045 0.037
0.943 0.054
sequentialUpdatesNoIndex 1024 0.041 0.029
0.838 0.053
sequentialWritesIndexed 1024 0.051 0.043
2.945 0.056
sequentialWritesNoIndex 1024 0.037 0.030
2.828 0.042
```
**8.1.1.1**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max
95th
dbClose 4 0.319 0.233
0.514 0.514
dbCreation 4 69.826 3.767
268.894 268.894
naturalIndexCreateIterator 1024 0.005 0.002
1.409 0.007
naturalIndexDescendingCreateIterator 1024 0.005 0.005
0.062 0.007
naturalIndexDescendingIteration 1024 0.006 0.004
0.026 0.008
naturalIndexIteration 1024 0.007 0.004
0.182 0.011
randomDeleteIndexed 1024 0.027 0.020
0.259 0.037
randomDeletesNoIndex 1024 0.016 0.013
0.041 0.019
randomUpdatesIndexed 1024 0.087 0.033
29.514 0.102
randomUpdatesNoIndex 1024 0.036 0.033
0.495 0.040
randomWritesIndexed 1024 0.120 0.033
53.330 0.126
randomWritesNoIndex 1024 0.041 0.035
1.403 0.047
refIndexCreateIterator 1024 0.006 0.005
0.019 0.007
refIndexDescendingCreateIterator 1024 0.003 0.003
0.028 0.005
refIndexDescendingIteration 1024 0.007 0.005
0.053 0.008
refIndexIteration 1024 0.008 0.006
0.254 0.010
sequentialDeleteIndexed 1024 0.023 0.018
1.237 0.027
sequentialDeleteNoIndex 1024 0.015 0.013
0.043 0.018
sequentialUpdatesIndexed 1024 0.046 0.038
0.816 0.057
sequentialUpdatesNoIndex 1024 0.042 0.030
0.719 0.052
sequentialWritesIndexed 1024 0.050 0.042
2.089 0.057
sequentialWritesNoIndex 1024 0.036 0.030
2.030 0.042
```
- Checked core module UTs with rocksdb live ui
```
export LIVE_UI_LOCAL_STORE_DIR=/${basedir}/spark-ui
build/mvn clean install -pl core -am
-Dtest.exclude.tags=org.apache.spark.tags.ExtendedLevelDBTest -fn
```
all test passed
Closes #41122 from LuciferYang/SPARK-43436.
Lead-authored-by: YangJie <yangji...@baidu.com>
Co-authored-by: yangjie01 <yangji...@baidu.com>
Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 2 +-
pom.xml | 2 +-
...StoreBasicOperationsBenchmark-jdk11-results.txt | 97 +++++++++++-----------
...StoreBasicOperationsBenchmark-jdk17-results.txt | 73 ++++++++--------
.../StateStoreBasicOperationsBenchmark-results.txt | 97 +++++++++++-----------
5 files changed, 137 insertions(+), 134 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index d3f3c990997..e02fca34737 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -223,7 +223,7 @@ parquet-jackson/1.13.0//parquet-jackson-1.13.0.jar
pickle/1.3//pickle-1.3.jar
py4j/0.10.9.7//py4j-0.10.9.7.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
-rocksdbjni/8.0.0//rocksdbjni-8.0.0.jar
+rocksdbjni/8.1.1.1//rocksdbjni-8.1.1.1.jar
scala-collection-compat_2.12/2.7.0//scala-collection-compat_2.12-2.7.0.jar
scala-compiler/2.12.17//scala-compiler-2.12.17.jar
scala-library/2.12.17//scala-library-2.12.17.jar
diff --git a/pom.xml b/pom.xml
index a81e6df057d..25df19f7f2d 100644
--- a/pom.xml
+++ b/pom.xml
@@ -667,7 +667,7 @@
<dependency>
<groupId>org.rocksdb</groupId>
<artifactId>rocksdbjni</artifactId>
- <version>8.0.0</version>
+ <version>8.1.1.1</version>
</dependency>
<dependency>
<groupId>${leveldbjni.group}</groupId>
diff --git
a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk11-results.txt
b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk11-results.txt
index 5ba8d824b71..ca82adcdcde 100644
--- a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk11-results.txt
+++ b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk11-results.txt
@@ -3,36 +3,36 @@ put rows
================================================================================================
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
putting 10000 rows (10000 rows to overwrite - rate 100): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 894.3 1.0X
-RocksDB (trackTotalNumberOfRows: true) 70
75 3 0.1 6964.6 0.1X
-RocksDB (trackTotalNumberOfRows: false) 23
26 1 0.4 2342.7 0.4X
+In-memory 8
8 1 1.3 751.0 1.0X
+RocksDB (trackTotalNumberOfRows: true) 59
61 1 0.2 5930.2 0.1X
+RocksDB (trackTotalNumberOfRows: false) 21
21 0 0.5 2057.7 0.4X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
putting 10000 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.1 882.4 1.0X
-RocksDB (trackTotalNumberOfRows: true) 57
63 3 0.2 5712.7 0.2X
-RocksDB (trackTotalNumberOfRows: false) 23
26 2 0.4 2299.9 0.4X
+In-memory 8
8 1 1.3 757.4 1.0X
+RocksDB (trackTotalNumberOfRows: true) 50
51 1 0.2 4970.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 20
22 0 0.5 2041.7 0.4X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
putting 10000 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 8
10 1 1.2 840.5 1.0X
-RocksDB (trackTotalNumberOfRows: true) 46
50 3 0.2 4554.6 0.2X
-RocksDB (trackTotalNumberOfRows: false) 22
25 2 0.4 2223.1 0.4X
+In-memory 7
8 1 1.4 728.3 1.0X
+RocksDB (trackTotalNumberOfRows: true) 41
42 0 0.2 4060.1 0.2X
+RocksDB (trackTotalNumberOfRows: false) 21
22 0 0.5 2053.6 0.4X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
putting 10000 rows (0 rows to overwrite - rate 0): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
-In-memory 8
10 1 1.2 821.2 1.0X
-RocksDB (trackTotalNumberOfRows: true) 42
47 2 0.2 4208.4 0.2X
-RocksDB (trackTotalNumberOfRows: false) 22
25 1 0.4 2229.2 0.4X
+In-memory 7
8 0 1.4 722.5 1.0X
+RocksDB (trackTotalNumberOfRows: true) 38
40 1 0.3 3815.7 0.2X
+RocksDB (trackTotalNumberOfRows: false) 20
22 0 0.5 2048.5 0.4X
================================================================================================
@@ -40,36 +40,36 @@ delete rows
================================================================================================
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
trying to delete 10000 rows from 10000 rows(10000 rows are non-existing - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
1 1 0 16.0 62.6
1.0X
-RocksDB (trackTotalNumberOfRows: true)
42 46 2 0.2 4168.6
0.0X
-RocksDB (trackTotalNumberOfRows: false)
22 24 2 0.5 2203.7
0.0X
+In-memory
1 1 0 16.8 59.5
1.0X
+RocksDB (trackTotalNumberOfRows: true)
38 39 1 0.3 3801.1
0.0X
+RocksDB (trackTotalNumberOfRows: false)
20 21 0 0.5 2038.7
0.0X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
trying to delete 10000 rows from 10000 rows(5000 rows are non-existing - rate
50): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 1 1.6 616.9
1.0X
-RocksDB (trackTotalNumberOfRows: true)
55 59 2 0.2 5479.9
0.1X
-RocksDB (trackTotalNumberOfRows: false)
22 24 1 0.5 2165.7
0.3X
+In-memory
5 6 0 1.8 547.3
1.0X
+RocksDB (trackTotalNumberOfRows: true)
49 50 1 0.2 4874.6
0.1X
+RocksDB (trackTotalNumberOfRows: false)
20 21 0 0.5 2021.8
0.3X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
trying to delete 10000 rows from 10000 rows(1000 rows are non-existing - rate
10): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 1 1.4 703.1
1.0X
-RocksDB (trackTotalNumberOfRows: true)
65 70 3 0.2 6455.5
0.1X
-RocksDB (trackTotalNumberOfRows: false)
22 24 1 0.5 2183.6
0.3X
+In-memory
6 7 1 1.6 619.6
1.0X
+RocksDB (trackTotalNumberOfRows: true)
57 58 1 0.2 5695.0
0.1X
+RocksDB (trackTotalNumberOfRows: false)
20 21 0 0.5 2036.2
0.3X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
trying to delete 10000 rows from 10000 rows(0 rows are non-existing - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 1 1.4 725.6 1.0X
-RocksDB (trackTotalNumberOfRows: true)
66 73 3 0.2 6631.7 0.1X
-RocksDB (trackTotalNumberOfRows: false)
22 24 1 0.5 2154.2 0.3X
+In-memory
6 7 0 1.6 629.0 1.0X
+RocksDB (trackTotalNumberOfRows: true)
59 60 1 0.2 5910.3 0.1X
+RocksDB (trackTotalNumberOfRows: false)
20 21 0 0.5 2028.0 0.3X
================================================================================================
@@ -77,34 +77,35 @@ evict rows
================================================================================================
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
evicting 10000 rows (maxTimestampToEvictInMillis: 9999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 1 1.4 717.1 1.0X
-RocksDB (trackTotalNumberOfRows: true)
68 74 3 0.1 6806.7 0.1X
-RocksDB (trackTotalNumberOfRows: false)
23 26 2 0.4 2298.4 0.3X
+In-memory
7 7 0 1.5 660.9 1.0X
+RocksDB (trackTotalNumberOfRows: true)
57 59 1 0.2 5725.3 0.1X
+RocksDB (trackTotalNumberOfRows: false)
23 23 0 0.4 2257.0 0.3X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
evicting 5000 rows (maxTimestampToEvictInMillis: 4999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 1 1.7 605.3 1.0X
-RocksDB (trackTotalNumberOfRows: true)
36 40 2 0.3 3591.4 0.2X
-RocksDB (trackTotalNumberOfRows: false)
13 15 1 0.7 1335.0 0.5X
+In-memory
6 6 0 1.8 554.4 1.0X
+RocksDB (trackTotalNumberOfRows: true)
31 32 1 0.3 3122.3 0.2X
+RocksDB (trackTotalNumberOfRows: false)
14 14 0 0.7 1369.2 0.4X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
evicting 1000 rows (maxTimestampToEvictInMillis: 999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
5 6 1 1.9 513.0 1.0X
-RocksDB (trackTotalNumberOfRows: true)
11 13 1 0.9 1100.6 0.5X
-RocksDB (trackTotalNumberOfRows: false)
7 8 1 1.5 668.5 0.8X
+In-memory
5 5 0 2.2 462.3 1.0X
+RocksDB (trackTotalNumberOfRows: true)
11 11 0 0.9 1057.6 0.4X
+RocksDB (trackTotalNumberOfRows: false)
7 7 0 1.4 704.0 0.7X
OpenJDK 64-Bit Server VM 11.0.19+7 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
evicting 0 rows (maxTimestampToEvictInMillis: -1) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
1 1 0 13.9 71.8 1.0X
-RocksDB (trackTotalNumberOfRows: true)
5 6 1 2.1 482.4 0.1X
-RocksDB (trackTotalNumberOfRows: false)
5 6 1 2.1 474.3 0.2X
+In-memory
1 1 0 15.6 64.0 1.0X
+RocksDB (trackTotalNumberOfRows: true)
5 5 0 1.9 525.6 0.1X
+RocksDB (trackTotalNumberOfRows: false)
5 5 0 1.9 514.3 0.1X
+
diff --git
a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk17-results.txt
b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk17-results.txt
index 266ef207bd9..cf7fae08f47 100644
--- a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk17-results.txt
+++ b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-jdk17-results.txt
@@ -6,33 +6,33 @@ OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (10000 rows to overwrite - rate 100): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
10 1 1.2 862.1 1.0X
-RocksDB (trackTotalNumberOfRows: true) 60
63 1 0.2 6048.9 0.1X
-RocksDB (trackTotalNumberOfRows: false) 20
21 0 0.5 1993.6 0.4X
+In-memory 9
10 1 1.2 864.7 1.0X
+RocksDB (trackTotalNumberOfRows: true) 59
61 1 0.2 5924.5 0.1X
+RocksDB (trackTotalNumberOfRows: false) 20
21 0 0.5 2031.8 0.4X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 9
9 1 1.1 873.8 1.0X
-RocksDB (trackTotalNumberOfRows: true) 50
52 1 0.2 5041.4 0.2X
-RocksDB (trackTotalNumberOfRows: false) 20
21 0 0.5 2041.0 0.4X
+In-memory 9
10 1 1.1 883.6 1.0X
+RocksDB (trackTotalNumberOfRows: true) 49
50 1 0.2 4867.5 0.2X
+RocksDB (trackTotalNumberOfRows: false) 20
22 0 0.5 2017.7 0.4X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 8
9 1 1.2 832.2 1.0X
-RocksDB (trackTotalNumberOfRows: true) 41
42 1 0.2 4069.5 0.2X
-RocksDB (trackTotalNumberOfRows: false) 20
21 0 0.5 1984.3 0.4X
+In-memory 8
9 1 1.2 825.4 1.0X
+RocksDB (trackTotalNumberOfRows: true) 38
40 1 0.3 3848.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 20
21 1 0.5 1989.0 0.4X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (0 rows to overwrite - rate 0): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
-In-memory 8
9 1 1.2 826.6 1.0X
-RocksDB (trackTotalNumberOfRows: true) 38
39 1 0.3 3804.5 0.2X
-RocksDB (trackTotalNumberOfRows: false) 20
21 0 0.5 1972.5 0.4X
+In-memory 8
9 1 1.2 817.1 1.0X
+RocksDB (trackTotalNumberOfRows: true) 36
37 1 0.3 3563.6 0.2X
+RocksDB (trackTotalNumberOfRows: false) 20
21 0 0.5 1985.2 0.4X
================================================================================================
@@ -43,33 +43,33 @@ OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(10000 rows are non-existing - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
1 1 0 17.6 56.8
1.0X
-RocksDB (trackTotalNumberOfRows: true)
38 39 0 0.3 3782.8
0.0X
-RocksDB (trackTotalNumberOfRows: false)
20 20 0 0.5 1986.1
0.0X
+In-memory
1 1 0 16.8 59.4
1.0X
+RocksDB (trackTotalNumberOfRows: true)
35 37 1 0.3 3548.2
0.0X
+RocksDB (trackTotalNumberOfRows: false)
20 20 0 0.5 1951.4
0.0X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(5000 rows are non-existing - rate
50): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 0 1.6 641.8
1.0X
-RocksDB (trackTotalNumberOfRows: true)
50 51 0 0.2 4968.6
0.1X
-RocksDB (trackTotalNumberOfRows: false)
20 20 0 0.5 1968.3
0.3X
+In-memory
6 7 1 1.6 639.5
1.0X
+RocksDB (trackTotalNumberOfRows: true)
47 48 1 0.2 4680.8
0.1X
+RocksDB (trackTotalNumberOfRows: false)
19 20 0 0.5 1944.8
0.3X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(1000 rows are non-existing - rate
10): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 0 1.4 713.9
1.0X
-RocksDB (trackTotalNumberOfRows: true)
59 60 1 0.2 5868.8
0.1X
-RocksDB (trackTotalNumberOfRows: false)
20 20 0 0.5 1971.1
0.4X
+In-memory
7 8 0 1.4 699.0
1.0X
+RocksDB (trackTotalNumberOfRows: true)
56 57 1 0.2 5552.7
0.1X
+RocksDB (trackTotalNumberOfRows: false)
19 20 0 0.5 1941.1
0.4X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(0 rows are non-existing - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 0 1.4 716.9 1.0X
-RocksDB (trackTotalNumberOfRows: true)
61 62 1 0.2 6053.0 0.1X
-RocksDB (trackTotalNumberOfRows: false)
20 20 0 0.5 1954.3 0.4X
+In-memory
7 8 0 1.4 708.8 1.0X
+RocksDB (trackTotalNumberOfRows: true)
57 59 2 0.2 5734.3 0.1X
+RocksDB (trackTotalNumberOfRows: false)
19 20 0 0.5 1934.9 0.4X
================================================================================================
@@ -80,31 +80,32 @@ OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 10000 rows (maxTimestampToEvictInMillis: 9999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
7 8 0 1.4 712.5 1.0X
-RocksDB (trackTotalNumberOfRows: true)
60 62 1 0.2 6016.9 0.1X
-RocksDB (trackTotalNumberOfRows: false)
20 20 0 0.5 2003.4 0.4X
+In-memory
7 8 0 1.4 695.6 1.0X
+RocksDB (trackTotalNumberOfRows: true)
59 60 1 0.2 5870.6 0.1X
+RocksDB (trackTotalNumberOfRows: false)
21 22 0 0.5 2096.3 0.3X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 5000 rows (maxTimestampToEvictInMillis: 4999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 0 1.5 646.0 1.0X
-RocksDB (trackTotalNumberOfRows: true)
32 33 1 0.3 3152.2 0.2X
-RocksDB (trackTotalNumberOfRows: false)
12 12 0 0.9 1171.4 0.6X
+In-memory
6 7 0 1.5 646.1 1.0X
+RocksDB (trackTotalNumberOfRows: true)
31 32 0 0.3 3146.2 0.2X
+RocksDB (trackTotalNumberOfRows: false)
13 13 0 0.8 1257.7 0.5X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 1000 rows (maxTimestampToEvictInMillis: 999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 6 0 1.8 558.3 1.0X
-RocksDB (trackTotalNumberOfRows: true)
10 10 0 1.0 963.9 0.6X
-RocksDB (trackTotalNumberOfRows: false)
6 6 0 1.8 568.1 1.0X
+In-memory
6 6 0 1.8 556.0 1.0X
+RocksDB (trackTotalNumberOfRows: true)
10 10 0 1.0 971.6 0.6X
+RocksDB (trackTotalNumberOfRows: false)
6 6 0 1.7 588.4 0.9X
OpenJDK 64-Bit Server VM 17.0.7+7 on Linux 5.15.0-1037-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 0 rows (maxTimestampToEvictInMillis: -1) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
1 1 0 14.8 67.5 1.0X
-RocksDB (trackTotalNumberOfRows: true)
4 4 0 2.5 403.0 0.2X
-RocksDB (trackTotalNumberOfRows: false)
4 4 0 2.5 401.5 0.2X
+In-memory
1 1 0 14.9 67.2 1.0X
+RocksDB (trackTotalNumberOfRows: true)
4 4 0 2.4 412.3 0.2X
+RocksDB (trackTotalNumberOfRows: false)
4 4 0 2.5 406.9 0.2X
+
diff --git a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
index 202e25805a7..d1fbeb3cd8b 100644
--- a/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
+++ b/sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt
@@ -3,36 +3,36 @@ put rows
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (10000 rows to overwrite - rate 100): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------------
-In-memory 8
9 1 1.3 788.9 1.0X
-RocksDB (trackTotalNumberOfRows: true) 60
63 3 0.2 6043.3 0.1X
-RocksDB (trackTotalNumberOfRows: false) 21
22 1 0.5 2105.1 0.4X
+In-memory 9
11 2 1.1 879.8 1.0X
+RocksDB (trackTotalNumberOfRows: true) 56
60 4 0.2 5649.3 0.2X
+RocksDB (trackTotalNumberOfRows: false) 19
21 2 0.5 1912.6 0.5X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 8
10 1 1.2 823.9 1.0X
-RocksDB (trackTotalNumberOfRows: true) 52
54 2 0.2 5182.9 0.2X
-RocksDB (trackTotalNumberOfRows: false) 21
23 1 0.5 2138.7 0.4X
+In-memory 9
11 2 1.2 859.4 1.0X
+RocksDB (trackTotalNumberOfRows: true) 48
51 3 0.2 4777.0 0.2X
+RocksDB (trackTotalNumberOfRows: false) 20
22 1 0.5 2034.6 0.4X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-In-memory 7
8 1 1.4 722.7 1.0X
-RocksDB (trackTotalNumberOfRows: true) 42
43 1 0.2 4188.7 0.2X
-RocksDB (trackTotalNumberOfRows: false) 21
22 1 0.5 2103.1 0.3X
+In-memory 8
10 2 1.3 765.4 1.0X
+RocksDB (trackTotalNumberOfRows: true) 38
41 2 0.3 3827.4 0.2X
+RocksDB (trackTotalNumberOfRows: false) 20
21 1 0.5 1969.3 0.4X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
putting 10000 rows (0 rows to overwrite - rate 0): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------------
-In-memory 7
8 1 1.4 720.2 1.0X
-RocksDB (trackTotalNumberOfRows: true) 40
41 1 0.3 3958.8 0.2X
-RocksDB (trackTotalNumberOfRows: false) 21
23 2 0.5 2109.9 0.3X
+In-memory 7
9 1 1.3 744.1 1.0X
+RocksDB (trackTotalNumberOfRows: true) 35
38 2 0.3 3544.0 0.2X
+RocksDB (trackTotalNumberOfRows: false) 19
21 1 0.5 1938.0 0.4X
================================================================================================
@@ -40,36 +40,36 @@ delete rows
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(10000 rows are non-existing - rate
100): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
0 1 0 21.3 47.0
1.0X
-RocksDB (trackTotalNumberOfRows: true)
40 42 2 0.3 3989.6
0.0X
-RocksDB (trackTotalNumberOfRows: false)
20 21 1 0.5 2045.9
0.0X
+In-memory
1 1 0 19.1 52.5
1.0X
+RocksDB (trackTotalNumberOfRows: true)
36 38 2 0.3 3550.6
0.0X
+RocksDB (trackTotalNumberOfRows: false)
19 20 1 0.5 1920.1
0.0X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(5000 rows are non-existing - rate
50): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 1 1.7 596.9
1.0X
-RocksDB (trackTotalNumberOfRows: true)
51 52 1 0.2 5127.4
0.1X
-RocksDB (trackTotalNumberOfRows: false)
21 22 1 0.5 2123.1
0.3X
+In-memory
6 7 1 1.8 556.2
1.0X
+RocksDB (trackTotalNumberOfRows: true)
47 50 3 0.2 4683.6
0.1X
+RocksDB (trackTotalNumberOfRows: false)
20 21 1 0.5 1954.1
0.3X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(1000 rows are non-existing - rate
10): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns)
Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 8 1 1.6 631.9
1.0X
-RocksDB (trackTotalNumberOfRows: true)
59 61 2 0.2 5876.7
0.1X
-RocksDB (trackTotalNumberOfRows: false)
21 22 1 0.5 2089.2
0.3X
+In-memory
7 9 1 1.5 660.5
1.0X
+RocksDB (trackTotalNumberOfRows: true)
56 59 3 0.2 5577.0
0.1X
+RocksDB (trackTotalNumberOfRows: false)
20 21 1 0.5 1973.1
0.3X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
trying to delete 10000 rows from 10000 rows(0 rows are non-existing - rate 0):
Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 1 1.5 647.1 1.0X
-RocksDB (trackTotalNumberOfRows: true)
60 62 1 0.2 6034.9 0.1X
-RocksDB (trackTotalNumberOfRows: false)
21 22 1 0.5 2063.4 0.3X
+In-memory
7 8 1 1.5 670.0 1.0X
+RocksDB (trackTotalNumberOfRows: true)
58 61 3 0.2 5800.5 0.1X
+RocksDB (trackTotalNumberOfRows: false)
20 21 1 0.5 1980.6 0.3X
================================================================================================
@@ -77,34 +77,35 @@ evict rows
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 10000 rows (maxTimestampToEvictInMillis: 9999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 0 1.6 625.7 1.0X
-RocksDB (trackTotalNumberOfRows: true)
60 61 2 0.2 5950.1 0.1X
-RocksDB (trackTotalNumberOfRows: false)
22 23 1 0.5 2194.1 0.3X
+In-memory
7 8 1 1.5 661.3 1.0X
+RocksDB (trackTotalNumberOfRows: true)
56 59 3 0.2 5648.5 0.1X
+RocksDB (trackTotalNumberOfRows: false)
20 21 1 0.5 1953.6 0.3X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 5000 rows (maxTimestampToEvictInMillis: 4999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
6 7 1 1.7 586.7 1.0X
-RocksDB (trackTotalNumberOfRows: true)
31 33 1 0.3 3146.3 0.2X
-RocksDB (trackTotalNumberOfRows: false)
13 13 0 0.8 1260.5 0.5X
+In-memory
6 7 1 1.8 570.4 1.0X
+RocksDB (trackTotalNumberOfRows: true)
30 32 2 0.3 3018.1 0.2X
+RocksDB (trackTotalNumberOfRows: false)
11 12 1 0.9 1129.8 0.5X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 1000 rows (maxTimestampToEvictInMillis: 999) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
5 5 0 2.2 463.0 1.0X
-RocksDB (trackTotalNumberOfRows: true)
9 9 0 1.1 925.8 0.5X
-RocksDB (trackTotalNumberOfRows: false)
6 6 0 1.8 552.6 0.8X
+In-memory
5 6 1 2.1 477.2 1.0X
+RocksDB (trackTotalNumberOfRows: true)
9 9 1 1.1 882.8 0.5X
+RocksDB (trackTotalNumberOfRows: false)
5 5 0 2.0 512.0 0.9X
OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure
-Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
+Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
evicting 0 rows (maxTimestampToEvictInMillis: -1) from 10000 rows: Best
Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------------------
-In-memory
1 1 0 18.4 54.4 1.0X
-RocksDB (trackTotalNumberOfRows: true)
4 4 0 2.8 351.8 0.2X
-RocksDB (trackTotalNumberOfRows: false)
3 4 0 2.9 342.9 0.2X
+In-memory
1 1 0 14.8 67.3 1.0X
+RocksDB (trackTotalNumberOfRows: true)
3 4 0 3.0 332.8 0.2X
+RocksDB (trackTotalNumberOfRows: false)
3 4 0 3.0 333.5 0.2X
+
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
