dongjoon-hyun commented on code in PR #45654:
URL: https://github.com/apache/spark/pull/45654#discussion_r1535000904
##########
sql/core/benchmarks/FilterPushdownBenchmark-jdk21-results.txt:
##########
@@ -2,733 +2,733 @@
Pushdown for many distinct value case
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 0 string row (value IS NULL): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6598 6767
128 2.4 419.5 1.0X
-Parquet Vectorized (Pushdown) 292 304
14 53.8 18.6 22.6X
-Native ORC Vectorized 7341 7411
88 2.1 466.7 0.9X
-Native ORC Vectorized (Pushdown) 282 291
10 55.7 18.0 23.4X
+Parquet Vectorized 6365 6420
67 2.5 404.7 1.0X
+Parquet Vectorized (Pushdown) 315 341
24 49.9 20.0 20.2X
+Native ORC Vectorized 4984 5073
69 3.2 316.9 1.3X
+Native ORC Vectorized (Pushdown) 311 321
11 50.5 19.8 20.5X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 0 string row ('7864320' < value < '7864320'): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6631
6653 20 2.4 421.6 1.0X
-Parquet Vectorized (Pushdown) 288
299 15 54.7 18.3 23.1X
-Native ORC Vectorized 7453
7468 19 2.1 473.8 0.9X
-Native ORC Vectorized (Pushdown) 280
287 9 56.2 17.8 23.7X
+Parquet Vectorized 6261
6292 25 2.5 398.1 1.0X
+Parquet Vectorized (Pushdown) 281
298 12 56.1 17.8 22.3X
+Native ORC Vectorized 5053
5078 22 3.1 321.3 1.2X
+Native ORC Vectorized (Pushdown) 300
328 35 52.4 19.1 20.9X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 1 string row (value = '7864320'): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6635 6654
17 2.4 421.9 1.0X
-Parquet Vectorized (Pushdown) 271 282
9 58.1 17.2 24.5X
-Native ORC Vectorized 7482 7512
35 2.1 475.7 0.9X
-Native ORC Vectorized (Pushdown) 268 277
4 58.6 17.1 24.7X
+Parquet Vectorized 6294 6356
90 2.5 400.1 1.0X
+Parquet Vectorized (Pushdown) 270 283
14 58.3 17.1 23.3X
+Native ORC Vectorized 5057 5086
20 3.1 321.5 1.2X
+Native ORC Vectorized (Pushdown) 289 298
8 54.4 18.4 21.8X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 1 string row (value <=> '7864320'): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6622 6633
10 2.4 421.0 1.0X
-Parquet Vectorized (Pushdown) 262 270
9 60.1 16.6 25.3X
-Native ORC Vectorized 7413 7429
15 2.1 471.3 0.9X
-Native ORC Vectorized (Pushdown) 261 268
6 60.3 16.6 25.4X
+Parquet Vectorized 6252 6274
17 2.5 397.5 1.0X
+Parquet Vectorized (Pushdown) 256 272
15 61.4 16.3 24.4X
+Native ORC Vectorized 5036 5054
12 3.1 320.2 1.2X
+Native ORC Vectorized (Pushdown) 275 291
8 57.1 17.5 22.7X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 1 string row ('7864320' <= value <= '7864320'): Best Time(ms) Avg
Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6597
6632 34 2.4 419.4 1.0X
-Parquet Vectorized (Pushdown) 260
268 7 60.5 16.5 25.4X
-Native ORC Vectorized 7453
7467 11 2.1 473.9 0.9X
-Native ORC Vectorized (Pushdown) 261
270 7 60.3 16.6 25.3X
+Parquet Vectorized 6241
6259 11 2.5 396.8 1.0X
+Parquet Vectorized (Pushdown) 257
266 12 61.3 16.3 24.3X
+Native ORC Vectorized 5038
5055 20 3.1 320.3 1.2X
+Native ORC Vectorized (Pushdown) 277
290 10 56.8 17.6 22.5X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select all string rows (value IS NOT NULL): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 12805 12872
92 1.2 814.1 1.0X
-Parquet Vectorized (Pushdown) 12833 12851
14 1.2 815.9 1.0X
-Native ORC Vectorized 13594 13615
16 1.2 864.3 0.9X
-Native ORC Vectorized (Pushdown) 13626 13637
11 1.2 866.3 0.9X
+Parquet Vectorized 12335 12564
207 1.3 784.2 1.0X
+Parquet Vectorized (Pushdown) 12561 12587
20 1.3 798.6 1.0X
+Native ORC Vectorized 11278 11295
12 1.4 717.0 1.1X
+Native ORC Vectorized (Pushdown) 11398 11468
98 1.4 724.7 1.1X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 0 int row (value IS NULL): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6401 6574
131 2.5 407.0 1.0X
-Parquet Vectorized (Pushdown) 240 248
7 65.6 15.2 26.7X
-Native ORC Vectorized 6666 6703
34 2.4 423.8 1.0X
-Native ORC Vectorized (Pushdown) 262 269
10 60.0 16.7 24.4X
+Parquet Vectorized 6150 6175
26 2.6 391.0 1.0X
+Parquet Vectorized (Pushdown) 252 276
25 62.4 16.0 24.4X
+Native ORC Vectorized 4484 4503
12 3.5 285.1 1.4X
+Native ORC Vectorized (Pushdown) 276 290
10 57.0 17.5 22.3X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 0 int row (7864320 < value < 7864320): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6299 6330
35 2.5 400.5 1.0X
-Parquet Vectorized (Pushdown) 238 251
9 66.0 15.2 26.4X
-Native ORC Vectorized 6662 6673
13 2.4 423.6 0.9X
-Native ORC Vectorized (Pushdown) 256 268
9 61.3 16.3 24.6X
+Parquet Vectorized 5983 5997
10 2.6 380.4 1.0X
+Parquet Vectorized (Pushdown) 258 274
18 61.1 16.4 23.2X
+Native ORC Vectorized 4470 4481
9 3.5 284.2 1.3X
+Native ORC Vectorized (Pushdown) 284 289
6 55.3 18.1 21.1X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 1 int row (value = 7864320): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6374 6395
20 2.5 405.3 1.0X
-Parquet Vectorized (Pushdown) 239 244
4 65.7 15.2 26.6X
-Native ORC Vectorized 6699 6745
40 2.3 425.9 1.0X
-Native ORC Vectorized (Pushdown) 255 262
7 61.6 16.2 25.0X
+Parquet Vectorized 6050 6074
27 2.6 384.6 1.0X
+Parquet Vectorized (Pushdown) 251 262
15 62.7 16.0 24.1X
+Native ORC Vectorized 4539 4568
29 3.5 288.6 1.3X
+Native ORC Vectorized (Pushdown) 270 286
11 58.4 17.1 22.4X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 1 int row (value <=> 7864320): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6363 6385
26 2.5 404.6 1.0X
-Parquet Vectorized (Pushdown) 234 241
4 67.1 14.9 27.1X
-Native ORC Vectorized 6725 6750
22 2.3 427.5 0.9X
-Native ORC Vectorized (Pushdown) 265 273
14 59.3 16.9 24.0X
+Parquet Vectorized 5983 6015
26 2.6 380.4 1.0X
+Parquet Vectorized (Pushdown) 242 247
4 65.0 15.4 24.7X
+Native ORC Vectorized 4502 4529
34 3.5 286.2 1.3X
+Native ORC Vectorized (Pushdown) 267 276
6 58.8 17.0 22.4X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 1 int row (7864320 <= value <= 7864320): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6363 6399
37 2.5 404.5 1.0X
-Parquet Vectorized (Pushdown) 236 241
5 66.7 15.0 27.0X
-Native ORC Vectorized 6762 6771
14 2.3 429.9 0.9X
-Native ORC Vectorized (Pushdown) 246 256
4 63.9 15.7 25.8X
+Parquet Vectorized 5978 6000
14 2.6 380.1 1.0X
+Parquet Vectorized (Pushdown) 243 254
12 64.8 15.4 24.6X
+Native ORC Vectorized 4520 4532
12 3.5 287.4 1.3X
+Native ORC Vectorized (Pushdown) 267 280
11 58.9 17.0 22.4X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 1 int row (7864319 < value < 7864321): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6365 6401
38 2.5 404.7 1.0X
-Parquet Vectorized (Pushdown) 236 240
5 66.8 15.0 27.0X
-Native ORC Vectorized 6734 6775
46 2.3 428.2 0.9X
-Native ORC Vectorized (Pushdown) 250 255
4 62.8 15.9 25.4X
+Parquet Vectorized 5992 6010
17 2.6 381.0 1.0X
+Parquet Vectorized (Pushdown) 250 257
9 63.0 15.9 24.0X
+Native ORC Vectorized 4525 4536
12 3.5 287.7 1.3X
+Native ORC Vectorized (Pushdown) 267 276
7 58.9 17.0 22.4X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 10% int rows (value < 1572864): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 6968 6990
14 2.3 443.0 1.0X
-Parquet Vectorized (Pushdown) 1456 1487
42 10.8 92.6 4.8X
-Native ORC Vectorized 7284 7290
5 2.2 463.1 1.0X
-Native ORC Vectorized (Pushdown) 1488 1497
8 10.6 94.6 4.7X
+Parquet Vectorized 6624 6642
20 2.4 421.1 1.0X
+Parquet Vectorized (Pushdown) 1438 1450
11 10.9 91.4 4.6X
+Native ORC Vectorized 5086 5101
17 3.1 323.4 1.3X
+Native ORC Vectorized (Pushdown) 1293 1303
8 12.2 82.2 5.1X
-OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 5.15.0-1053-azure
+OpenJDK 64-Bit Server VM 21.0.2+13-LTS on Linux 6.5.0-1016-azure
AMD EPYC 7763 64-Core Processor
Select 50% int rows (value < 7864320): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Parquet Vectorized 9072 9082
11 1.7 576.8 1.0X
-Parquet Vectorized (Pushdown) 6042 6065
21 2.6 384.1 1.5X
-Native ORC Vectorized 9554 9580
17 1.6 607.4 0.9X
-Native ORC Vectorized (Pushdown) 6387 6400
9 2.5 406.1 1.4X
+Parquet Vectorized 8735 8748
8 1.8 555.3 1.0X
+Parquet Vectorized (Pushdown) 5861 5871
12 2.7 372.7 1.5X
+Native ORC Vectorized 7281 7331
29 2.2 462.9 1.2X
+Native ORC Vectorized (Pushdown) 5242 5250
12 3.0 333.3 1.7X
Review Comment:
Th ratio is slightly reversed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
