[GitHub] [spark] anishshri-db commented on a diff in pull request #41175: [SPARK-43512][SS] Update StateStoreOperationsBenchmark to reflect updates to RocksDB usage as state store provider
anishshri-db commented on code in PR #41175: URL: https://github.com/apache/spark/pull/41175#discussion_r1194656873 ## sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt: ## @@ -2,182 +2,109 @@ put rows -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure +Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz putting 1 rows (1 rows to overwrite - rate 100): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative --- -In-memory8 9 1 1.3 776.2 1.0X -RocksDB (trackTotalNumberOfRows: true) 50 52 1 0.25024.8 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 14 0 0.81290.8 0.6X +In-memory8 9 1 1.3 788.9 1.0X +RocksDB (trackTotalNumberOfRows: true) 60 63 3 0.26043.3 0.1X +RocksDB (trackTotalNumberOfRows: false) 21 22 1 0.52105.1 0.4X -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz -putting 1 rows (7500 rows to overwrite - rate 75): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -- -In-memory 8 9 1 1.2 800.0 1.0X -RocksDB (trackTotalNumberOfRows: true)47 48 1 0.24667.5 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 15 0 0.81316.2 0.6X - -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure +Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz putting 1 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative - -In-memory 8 9 1 1.3 786.1 1.0X -RocksDB (trackTotalNumberOfRows: true)42 43 1 0.24157.9 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 15 0 0.71345.1 0.6X - -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz -putting 1 rows (2500 rows to overwrite - rate 25): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -- -In-memory 8 9 1 1.3 755.5 1.0X -RocksDB (trackTotalNumberOfRows: true)36 38 1 0.33638.3 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 14 0 0.81308.7 0.6X +In-memory 8 10 1 1.2 823.9 1.0X +RocksDB (trackTotalNumberOfRows: true)52 54 2 0.25182.9 0.2X +RocksDB (trackTotalNumberOfRows: false) 21 23 1 0.52138.7 0.4X -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure +Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz putting 1 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative
[GitHub] [spark] anishshri-db commented on a diff in pull request #41175: [SPARK-43512][SS] Update StateStoreOperationsBenchmark to reflect updates to RocksDB usage as state store provider
anishshri-db commented on code in PR #41175: URL: https://github.com/apache/spark/pull/41175#discussion_r1194656873 ## sql/core/benchmarks/StateStoreBasicOperationsBenchmark-results.txt: ## @@ -2,182 +2,109 @@ put rows -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure +Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz putting 1 rows (1 rows to overwrite - rate 100): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative --- -In-memory8 9 1 1.3 776.2 1.0X -RocksDB (trackTotalNumberOfRows: true) 50 52 1 0.25024.8 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 14 0 0.81290.8 0.6X +In-memory8 9 1 1.3 788.9 1.0X +RocksDB (trackTotalNumberOfRows: true) 60 63 3 0.26043.3 0.1X +RocksDB (trackTotalNumberOfRows: false) 21 22 1 0.52105.1 0.4X -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz -putting 1 rows (7500 rows to overwrite - rate 75): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -- -In-memory 8 9 1 1.2 800.0 1.0X -RocksDB (trackTotalNumberOfRows: true)47 48 1 0.24667.5 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 15 0 0.81316.2 0.6X - -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure +Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz putting 1 rows (5000 rows to overwrite - rate 50): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative - -In-memory 8 9 1 1.3 786.1 1.0X -RocksDB (trackTotalNumberOfRows: true)42 43 1 0.24157.9 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 15 0 0.71345.1 0.6X - -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz -putting 1 rows (2500 rows to overwrite - rate 25): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative -- -In-memory 8 9 1 1.3 755.5 1.0X -RocksDB (trackTotalNumberOfRows: true)36 38 1 0.33638.3 0.2X -RocksDB (trackTotalNumberOfRows: false) 13 14 0 0.81308.7 0.6X +In-memory 8 10 1 1.2 823.9 1.0X +RocksDB (trackTotalNumberOfRows: true)52 54 2 0.25182.9 0.2X +RocksDB (trackTotalNumberOfRows: false) 21 23 1 0.52138.7 0.4X -OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1035-azure -Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz +OpenJDK 64-Bit Server VM 1.8.0_362-b09 on Linux 5.15.0-1037-azure +Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz putting 1 rows (1000 rows to overwrite - rate 10): Best Time(ms) Avg Time(ms) Stdev(ms)Rate(M/s) Per Row(ns) Relative
