This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new aa7e54fc072 [SPARK-41862][SQL][TESTS][FOLLOWUP] Update
OrcReadBenchmark result
aa7e54fc072 is described below
commit aa7e54fc0728af99cc8c55f3bd88b4ade4aaab05
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Jan 3 22:05:50 2023 -0800
[SPARK-41862][SQL][TESTS][FOLLOWUP] Update OrcReadBenchmark result
### What changes were proposed in this pull request?
This PR is a follow-up of https://github.com/apache/spark/pull/39370 .
### Why are the changes needed?
To sync the patch with the recovered perf result.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual review.
- Java 8: https://github.com/dongjoon-hyun/spark/actions/runs/3834890434
- Java 11: https://github.com/dongjoon-hyun/spark/actions/runs/3834892478
- Java 17: https://github.com/dongjoon-hyun/spark/actions/runs/3834893844
Closes #39380 from dongjoon-hyun/SPARK-41862.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../benchmarks/OrcReadBenchmark-jdk11-results.txt | 188 ++++++++++-----------
.../benchmarks/OrcReadBenchmark-jdk17-results.txt | 188 ++++++++++-----------
sql/hive/benchmarks/OrcReadBenchmark-results.txt | 188 ++++++++++-----------
3 files changed, 282 insertions(+), 282 deletions(-)
diff --git a/sql/hive/benchmarks/OrcReadBenchmark-jdk11-results.txt
b/sql/hive/benchmarks/OrcReadBenchmark-jdk11-results.txt
index 5c44741b591..7d6db9ae30d 100644
--- a/sql/hive/benchmarks/OrcReadBenchmark-jdk11-results.txt
+++ b/sql/hive/benchmarks/OrcReadBenchmark-jdk11-results.txt
@@ -3,52 +3,52 @@ SQL Single Numeric Column Scan
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1088 1102
18 14.5 69.2 1.0X
-Native ORC MR 905 971
90 17.4 57.5 1.2X
-Native ORC Vectorized 137 206
69 114.4 8.7 7.9X
+Hive built-in ORC 1087 1119
45 14.5 69.1 1.0X
+Native ORC MR 882 936
50 17.8 56.1 1.2X
+Native ORC Vectorized 164 213
31 96.0 10.4 6.6X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1265 1279
20 12.4 80.4 1.0X
-Native ORC MR 1022 1102
113 15.4 65.0 1.2X
-Native ORC Vectorized 135 201
63 116.8 8.6 9.4X
+Hive built-in ORC 1282 1289
10 12.3 81.5 1.0X
+Native ORC MR 916 962
65 17.2 58.2 1.4X
+Native ORC Vectorized 151 212
47 104.1 9.6 8.5X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single INT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1196 1258
88 13.1 76.0 1.0X
-Native ORC MR 995 1014
27 15.8 63.3 1.2X
-Native ORC Vectorized 188 257
52 83.8 11.9 6.4X
+Hive built-in ORC 1245 1278
46 12.6 79.1 1.0X
+Native ORC MR 932 948
22 16.9 59.2 1.3X
+Native ORC Vectorized 163 253
42 96.3 10.4 7.6X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single BIGINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1341 1417
108 11.7 85.2 1.0X
-Native ORC MR 1166 1175
12 13.5 74.1 1.1X
-Native ORC Vectorized 330 368
63 47.7 21.0 4.1X
+Hive built-in ORC 1268 1295
37 12.4 80.6 1.0X
+Native ORC MR 969 1001
49 16.2 61.6 1.3X
+Native ORC Vectorized 243 286
56 64.6 15.5 5.2X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1409 1410
1 11.2 89.6 1.0X
-Native ORC MR 1245 1255
15 12.6 79.1 1.1X
-Native ORC Vectorized 227 360
76 69.4 14.4 6.2X
+Hive built-in ORC 1225 1293
95 12.8 77.9 1.0X
+Native ORC MR 921 969
44 17.1 58.6 1.3X
+Native ORC Vectorized 278 313
19 56.5 17.7 4.4X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1421 1427
8 11.1 90.4 1.0X
-Native ORC MR 1226 1268
59 12.8 78.0 1.2X
-Native ORC Vectorized 291 376
51 54.1 18.5 4.9X
+Hive built-in ORC 1273 1334
86 12.4 80.9 1.0X
+Native ORC MR 963 1000
38 16.3 61.2 1.3X
+Native ORC Vectorized 248 312
35 63.4 15.8 5.1X
================================================================================================
@@ -56,12 +56,12 @@ Int and String Scan
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Int and String Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2419 2476
81 4.3 230.7 1.0X
-Native ORC MR 2313 2326
18 4.5 220.6 1.0X
-Native ORC Vectorized 1294 1324
43 8.1 123.4 1.9X
+Hive built-in ORC 2757 2891
189 3.8 263.0 1.0X
+Native ORC MR 2117 2124
10 5.0 201.9 1.3X
+Native ORC Vectorized 1342 1351
13 7.8 127.9 2.1X
================================================================================================
@@ -69,18 +69,18 @@ Partitioned Table Scan
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Partitioned Table: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Data column - Hive built-in ORC 1534 1647
160 10.3 97.5 1.0X
-Data column - Native ORC MR 1448 1637
267 10.9 92.1 1.1X
-Data column - Native ORC Vectorized 374 401
30 42.0 23.8 4.1X
-Partition column - Hive built-in ORC 1073 1088
21 14.7 68.2 1.4X
-Partition column - Native ORC MR 949 968
24 16.6 60.3 1.6X
-Partition column - Native ORC Vectorized 41 63
19 383.7 2.6 37.4X
-Both columns - Hive built-in ORC 1732 1801
98 9.1 110.1 0.9X
-Both columns - Native ORC MR 1657 1713
78 9.5 105.4 0.9X
-Both columns - Native ORC Vectorized 392 440
53 40.1 24.9 3.9X
+Data column - Hive built-in ORC 1514 1559
63 10.4 96.3 1.0X
+Data column - Native ORC MR 1163 1228
92 13.5 73.9 1.3X
+Data column - Native ORC Vectorized 314 328
18 50.1 20.0 4.8X
+Partition column - Hive built-in ORC 895 910
13 17.6 56.9 1.7X
+Partition column - Native ORC MR 807 850
48 19.5 51.3 1.9X
+Partition column - Native ORC Vectorized 53 71
19 295.0 3.4 28.4X
+Both columns - Hive built-in ORC 1597 1604
11 9.9 101.5 0.9X
+Both columns - Native ORC MR 1174 1240
93 13.4 74.7 1.3X
+Both columns - Native ORC Vectorized 318 360
35 49.4 20.2 4.8X
================================================================================================
@@ -88,12 +88,12 @@ Repeated String Scan
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Repeated String: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1229 1243
20 8.5 117.2 1.0X
-Native ORC MR 1006 1018
18 10.4 95.9 1.2X
-Native ORC Vectorized 200 251
83 52.3 19.1 6.1X
+Hive built-in ORC 1231 1272
59 8.5 117.4 1.0X
+Native ORC MR 806 818
16 13.0 76.9 1.5X
+Native ORC Vectorized 191 234
44 54.9 18.2 6.4X
================================================================================================
@@ -101,28 +101,28 @@ String with Nulls Scan
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (0.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2149 2312
231 4.9 204.9 1.0X
-Native ORC MR 1806 1815
12 5.8 172.2 1.2X
-Native ORC Vectorized 548 671
106 19.1 52.3 3.9X
+Hive built-in ORC 2518 2563
64 4.2 240.2 1.0X
+Native ORC MR 1678 1694
24 6.3 160.0 1.5X
+Native ORC Vectorized 667 678
10 15.7 63.6 3.8X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (50.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2026 2112
122 5.2 193.2 1.0X
-Native ORC MR 1652 1680
40 6.3 157.5 1.2X
-Native ORC Vectorized 789 854
100 13.3 75.2 2.6X
+Hive built-in ORC 2197 2323
177 4.8 209.6 1.0X
+Native ORC MR 1521 1561
55 6.9 145.1 1.4X
+Native ORC Vectorized 818 826
8 12.8 78.0 2.7X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (95.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1119 1129
13 9.4 106.7 1.0X
-Native ORC MR 902 929
35 11.6 86.0 1.2X
-Native ORC Vectorized 231 294
80 45.3 22.1 4.8X
+Hive built-in ORC 1264 1282
26 8.3 120.5 1.0X
+Native ORC MR 837 855
16 12.5 79.8 1.5X
+Native ORC Vectorized 239 293
51 43.8 22.8 5.3X
================================================================================================
@@ -130,28 +130,28 @@ Single Column Scan From Wide Columns
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 100 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 956 975
17 1.1 912.0 1.0X
-Native ORC MR 185 229
45 5.7 176.9 5.2X
-Native ORC Vectorized 73 105
31 14.3 69.7 13.1X
+Hive built-in ORC 664 672
12 1.6 633.4 1.0X
+Native ORC MR 149 196
37 7.0 142.5 4.4X
+Native ORC Vectorized 75 115
31 14.0 71.6 8.9X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 200 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1708 1826
167 0.6 1629.0 1.0X
-Native ORC MR 173 212
51 6.1 164.9 9.9X
-Native ORC Vectorized 124 182
62 8.5 118.3 13.8X
+Hive built-in ORC 1159 1170
15 0.9 1105.6 1.0X
+Native ORC MR 187 249
39 5.6 178.8 6.2X
+Native ORC Vectorized 127 182
45 8.3 120.9 9.1X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 300 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2529 2600
99 0.4 2412.3 1.0X
-Native ORC MR 241 329
61 4.3 230.0 10.5X
-Native ORC Vectorized 175 260
59 6.0 166.4 14.5X
+Hive built-in ORC 1719 1734
21 0.6 1639.2 1.0X
+Native ORC MR 259 350
51 4.0 247.0 6.6X
+Native ORC Vectorized 200 275
51 5.2 190.8 8.6X
================================================================================================
@@ -159,36 +159,36 @@ Struct scan
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1118 1153
50 0.9 1066.4 1.0X
-Native ORC MR 1734 1799
92 0.6 1653.9 0.6X
-Native ORC Vectorized 307 326
19 3.4 292.9 3.6X
+Hive built-in ORC 562 582
16 1.9 535.6 1.0X
+Native ORC MR 317 392
38 3.3 302.1 1.8X
+Native ORC Vectorized 237 259
14 4.4 226.2 2.4X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 100 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 7102 7251
210 0.1 6772.8 1.0X
-Native ORC MR 11195 11213
25 0.1 10676.4 0.6X
-Native ORC Vectorized 2558 2591
47 0.4 2439.7 2.8X
+Hive built-in ORC 3510 3571
87 0.3 3347.0 1.0X
+Native ORC MR 3357 3519
229 0.3 3201.4 1.0X
+Native ORC Vectorized 1972 1974
3 0.5 1880.3 1.8X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 300 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 22048 22363
445 0.0 21026.8 1.0X
-Native ORC MR 35831 35982
214 0.0 34171.0 0.6X
-Native ORC Vectorized 35836 36163
463 0.0 34175.9 0.6X
+Hive built-in ORC 10917 11045
181 0.1 10411.2 1.0X
+Native ORC MR 10418 10518
142 0.1 9935.5 1.0X
+Native ORC Vectorized 10004 10315
441 0.1 9540.4 1.1X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 600 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 47258 47526
379 0.0 45068.9 1.0X
-Native ORC MR 72743 72953
297 0.0 69373.0 0.6X
-Native ORC Vectorized 72600 72679
111 0.0 69236.9 0.7X
+Hive built-in ORC 20536 20555
26 0.1 19584.8 1.0X
+Native ORC MR 21028 21295
378 0.0 20054.0 1.0X
+Native ORC Vectorized 20436 21379
1333 0.1 19489.2 1.0X
================================================================================================
@@ -196,27 +196,27 @@ Nested Struct scan
================================================================================================
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 10 Elements, 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 8836 8838
3 0.1 8426.7 1.0X
-Native ORC MR 4196 4359
231 0.2 4001.2 2.1X
-Native ORC Vectorized 1156 1161
7 0.9 1102.4 7.6X
+Hive built-in ORC 4156 4177
29 0.3 3963.4 1.0X
+Native ORC MR 4200 4225
35 0.2 4005.5 1.0X
+Native ORC Vectorized 959 985
27 1.1 914.5 4.3X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 30 Elements, 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 24332 24448
164 0.0 23204.9 1.0X
-Native ORC MR 8741 8906
234 0.1 8336.3 2.8X
-Native ORC Vectorized 2673 2738
91 0.4 2549.6 9.1X
+Hive built-in ORC 10916 10936
29 0.1 10410.1 1.0X
+Native ORC MR 9396 9414
26 0.1 8960.7 1.2X
+Native ORC Vectorized 2485 2527
60 0.4 2370.0 4.4X
OpenJDK 64-Bit Server VM 11.0.17+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 10 Elements, 30 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 20054 20057
4 0.1 19124.9 1.0X
-Native ORC MR 10183 10207
35 0.1 9710.8 2.0X
-Native ORC Vectorized 4668 4790
172 0.2 4452.1 4.3X
+Hive built-in ORC 8233 8251
26 0.1 7851.3 1.0X
+Native ORC MR 10245 10269
34 0.1 9770.1 0.8X
+Native ORC Vectorized 2940 3004
90 0.4 2804.1 2.8X
diff --git a/sql/hive/benchmarks/OrcReadBenchmark-jdk17-results.txt
b/sql/hive/benchmarks/OrcReadBenchmark-jdk17-results.txt
index 1840f144a07..909b56f829a 100644
--- a/sql/hive/benchmarks/OrcReadBenchmark-jdk17-results.txt
+++ b/sql/hive/benchmarks/OrcReadBenchmark-jdk17-results.txt
@@ -3,52 +3,52 @@ SQL Single Numeric Column Scan
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1063 1099
52 14.8 67.6 1.0X
-Native ORC MR 842 870
42 18.7 53.5 1.3X
-Native ORC Vectorized 136 189
47 115.5 8.7 7.8X
+Hive built-in ORC 978 990
14 16.1 62.2 1.0X
+Native ORC MR 786 834
49 20.0 50.0 1.2X
+Native ORC Vectorized 135 172
21 116.4 8.6 7.2X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1345 1414
99 11.7 85.5 1.0X
-Native ORC MR 914 958
64 17.2 58.1 1.5X
-Native ORC Vectorized 129 172
55 122.0 8.2 10.4X
+Hive built-in ORC 987 1001
19 15.9 62.8 1.0X
+Native ORC MR 767 788
30 20.5 48.8 1.3X
+Native ORC Vectorized 120 137
14 131.4 7.6 8.2X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single INT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1286 1303
24 12.2 81.8 1.0X
-Native ORC MR 967 980
14 16.3 61.5 1.3X
-Native ORC Vectorized 164 184
26 95.7 10.4 7.8X
+Hive built-in ORC 1043 1053
14 15.1 66.3 1.0X
+Native ORC MR 790 798
11 19.9 50.2 1.3X
+Native ORC Vectorized 133 151
15 118.6 8.4 7.9X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single BIGINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1410 1418
11 11.2 89.7 1.0X
-Native ORC MR 1116 1124
11 14.1 70.9 1.3X
-Native ORC Vectorized 230 246
12 68.5 14.6 6.1X
+Hive built-in ORC 1090 1102
17 14.4 69.3 1.0X
+Native ORC MR 832 867
42 18.9 52.9 1.3X
+Native ORC Vectorized 164 195
34 96.2 10.4 6.7X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1459 1485
38 10.8 92.8 1.0X
-Native ORC MR 1149 1181
45 13.7 73.1 1.3X
-Native ORC Vectorized 231 308
56 67.9 14.7 6.3X
+Hive built-in ORC 1133 1140
11 13.9 72.0 1.0X
+Native ORC MR 856 861
5 18.4 54.4 1.3X
+Native ORC Vectorized 198 219
19 79.4 12.6 5.7X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1493 1522
42 10.5 94.9 1.0X
-Native ORC MR 1389 1467
110 11.3 88.3 1.1X
-Native ORC Vectorized 273 315
41 57.6 17.4 5.5X
+Hive built-in ORC 1218 1236
24 12.9 77.5 1.0X
+Native ORC MR 878 920
60 17.9 55.8 1.4X
+Native ORC Vectorized 224 245
18 70.1 14.3 5.4X
================================================================================================
@@ -56,12 +56,12 @@ Int and String Scan
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Int and String Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2609 2623
20 4.0 248.8 1.0X
-Native ORC MR 2176 2253
109 4.8 207.5 1.2X
-Native ORC Vectorized 1339 1342
3 7.8 127.7 1.9X
+Hive built-in ORC 2341 2394
74 4.5 223.3 1.0X
+Native ORC MR 1889 1903
20 5.6 180.2 1.2X
+Native ORC Vectorized 1144 1165
30 9.2 109.1 2.0X
================================================================================================
@@ -69,18 +69,18 @@ Partitioned Table Scan
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Partitioned Table: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Data column - Hive built-in ORC 1951 1954
3 8.1 124.1 1.0X
-Data column - Native ORC MR 2496 2522
37 6.3 158.7 0.8X
-Data column - Native ORC Vectorized 389 449
60 40.5 24.7 5.0X
-Partition column - Hive built-in ORC 1185 1205
27 13.3 75.4 1.6X
-Partition column - Native ORC MR 966 1011
44 16.3 61.4 2.0X
-Partition column - Native ORC Vectorized 42 51
9 371.9 2.7 46.1X
-Both columns - Hive built-in ORC 1916 1973
80 8.2 121.8 1.0X
-Both columns - Native ORC MR 1504 1512
10 10.5 95.6 1.3X
-Both columns - Native ORC Vectorized 286 293
6 55.1 18.2 6.8X
+Data column - Hive built-in ORC 1250 1262
16 12.6 79.5 1.0X
+Data column - Native ORC MR 1031 1049
26 15.3 65.5 1.2X
+Data column - Native ORC Vectorized 168 185
20 93.7 10.7 7.4X
+Partition column - Hive built-in ORC 976 988
10 16.1 62.1 1.3X
+Partition column - Native ORC MR 710 742
54 22.2 45.1 1.8X
+Partition column - Native ORC Vectorized 41 49
6 382.1 2.6 30.4X
+Both columns - Hive built-in ORC 1301 1343
60 12.1 82.7 1.0X
+Both columns - Native ORC MR 1096 1194
139 14.4 69.7 1.1X
+Both columns - Native ORC Vectorized 193 221
26 81.5 12.3 6.5X
================================================================================================
@@ -88,12 +88,12 @@ Repeated String Scan
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Repeated String: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1274 1299
36 8.2 121.5 1.0X
-Native ORC MR 999 1002
4 10.5 95.2 1.3X
-Native ORC Vectorized 218 229
12 48.0 20.8 5.8X
+Hive built-in ORC 1099 1124
34 9.5 104.9 1.0X
+Native ORC MR 767 791
32 13.7 73.1 1.4X
+Native ORC Vectorized 179 191
10 58.4 17.1 6.1X
================================================================================================
@@ -101,28 +101,28 @@ String with Nulls Scan
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (0.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2631 2632
1 4.0 250.9 1.0X
-Native ORC MR 2051 2054
4 5.1 195.6 1.3X
-Native ORC Vectorized 642 682
37 16.3 61.2 4.1X
+Hive built-in ORC 2082 2095
18 5.0 198.6 1.0X
+Native ORC MR 1450 1455
7 7.2 138.2 1.4X
+Native ORC Vectorized 538 552
14 19.5 51.3 3.9X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (50.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2283 2305
32 4.6 217.7 1.0X
-Native ORC MR 1915 1922
9 5.5 182.6 1.2X
-Native ORC Vectorized 750 752
2 14.0 71.5 3.0X
+Hive built-in ORC 2034 2039
7 5.2 194.0 1.0X
+Native ORC MR 1493 1494
1 7.0 142.4 1.4X
+Native ORC Vectorized 712 719
10 14.7 67.9 2.9X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (95.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1194 1196
3 8.8 113.9 1.0X
-Native ORC MR 936 972
59 11.2 89.3 1.3X
-Native ORC Vectorized 234 338
87 44.8 22.3 5.1X
+Hive built-in ORC 1111 1120
13 9.4 105.9 1.0X
+Native ORC MR 782 801
20 13.4 74.6 1.4X
+Native ORC Vectorized 223 247
19 47.0 21.3 5.0X
================================================================================================
@@ -130,28 +130,28 @@ Single Column Scan From Wide Columns
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 100 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1031 1089
82 1.0 983.2 1.0X
-Native ORC MR 139 155
12 7.5 132.5 7.4X
-Native ORC Vectorized 81 88
7 12.9 77.6 12.7X
+Hive built-in ORC 897 936
59 1.2 855.5 1.0X
+Native ORC MR 111 126
12 9.5 105.8 8.1X
+Native ORC Vectorized 63 75
9 16.6 60.2 14.2X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 200 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1898 1913
21 0.6 1810.2 1.0X
-Native ORC MR 189 201
11 5.6 180.1 10.1X
-Native ORC Vectorized 132 142
10 8.0 125.6 14.4X
+Hive built-in ORC 1624 1705
115 0.6 1548.7 1.0X
+Native ORC MR 141 164
17 7.5 134.2 11.5X
+Native ORC Vectorized 88 106
13 12.0 83.7 18.5X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 300 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2616 2642
38 0.4 2494.6 1.0X
-Native ORC MR 256 266
7 4.1 244.6 10.2X
-Native ORC Vectorized 215 272
50 4.9 205.1 12.2X
+Hive built-in ORC 2362 2425
89 0.4 2252.6 1.0X
+Native ORC MR 170 192
24 6.2 161.7 13.9X
+Native ORC Vectorized 121 135
13 8.7 115.1 19.6X
================================================================================================
@@ -159,36 +159,36 @@ Struct scan
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 527 654
95 2.0 502.5 1.0X
-Native ORC MR 1389 1405
23 0.8 1324.6 0.4X
-Native ORC Vectorized 297 317
16 3.5 283.3 1.8X
+Hive built-in ORC 526 554
19 2.0 502.1 1.0X
+Native ORC MR 346 380
21 3.0 330.3 1.5X
+Native ORC Vectorized 226 245
11 4.6 215.2 2.3X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 100 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 3297 3312
21 0.3 3144.3 1.0X
-Native ORC MR 10926 10968
60 0.1 10419.7 0.3X
-Native ORC Vectorized 2059 2496
618 0.5 1963.6 1.6X
+Hive built-in ORC 2793 2848
77 0.4 2663.5 1.0X
+Native ORC MR 2329 2334
8 0.5 2220.9 1.2X
+Native ORC Vectorized 1254 1330
107 0.8 1196.0 2.2X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 300 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 10303 10722
592 0.1 9825.7 1.0X
-Native ORC MR 34028 34075
67 0.0 32451.3 0.3X
-Native ORC Vectorized 33861 33926
92 0.0 32291.9 0.3X
+Hive built-in ORC 10687 10736
70 0.1 10191.8 1.0X
+Native ORC MR 7443 7453
14 0.1 7098.5 1.4X
+Native ORC Vectorized 7435 7879
628 0.1 7090.1 1.4X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 600 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 21817 22263
630 0.0 20806.1 1.0X
-Native ORC MR 73318 73597
396 0.0 69921.0 0.3X
-Native ORC Vectorized 69794 70959
1648 0.0 66560.9 0.3X
+Hive built-in ORC 21409 24272
NaN 0.0 20417.6 1.0X
+Native ORC MR 15825 15873
68 0.1 15092.0 1.4X
+Native ORC Vectorized 18672 21934
1687 0.1 17806.7 1.1X
================================================================================================
@@ -196,27 +196,27 @@ Nested Struct scan
================================================================================================
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 10 Elements, 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 3696 3701
7 0.3 3524.6 1.0X
-Native ORC MR 3491 3497
9 0.3 3328.9 1.1X
-Native ORC Vectorized 1231 1234
5 0.9 1173.9 3.0X
+Hive built-in ORC 3331 3369
54 0.3 3176.7 1.0X
+Native ORC MR 3324 3336
17 0.3 3170.3 1.0X
+Native ORC Vectorized 986 1001
13 1.1 940.4 3.4X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 30 Elements, 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 11867 11980
160 0.1 11317.2 1.0X
-Native ORC MR 8249 8293
63 0.1 7866.5 1.4X
-Native ORC Vectorized 3172 3234
88 0.3 3025.3 3.7X
+Hive built-in ORC 9077 9617
765 0.1 8656.1 1.0X
+Native ORC MR 8431 8466
50 0.1 8040.2 1.1X
+Native ORC Vectorized 2401 2498
137 0.4 2290.2 3.8X
OpenJDK 64-Bit Server VM 17.0.5+8 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 10 Elements, 30 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 9749 9854
149 0.1 9296.9 1.0X
-Native ORC MR 9966 9971
6 0.1 9504.8 1.0X
-Native ORC Vectorized 3318 3518
283 0.3 3164.4 2.9X
+Hive built-in ORC 8454 8567
159 0.1 8062.7 1.0X
+Native ORC MR 8255 8294
55 0.1 7872.9 1.0X
+Native ORC Vectorized 2579 2585
9 0.4 2459.6 3.3X
diff --git a/sql/hive/benchmarks/OrcReadBenchmark-results.txt
b/sql/hive/benchmarks/OrcReadBenchmark-results.txt
index 137303c15bf..7f18b124e4e 100644
--- a/sql/hive/benchmarks/OrcReadBenchmark-results.txt
+++ b/sql/hive/benchmarks/OrcReadBenchmark-results.txt
@@ -3,52 +3,52 @@ SQL Single Numeric Column Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1137 1146
14 13.8 72.3 1.0X
-Native ORC MR 1034 1048
20 15.2 65.7 1.1X
-Native ORC Vectorized 92 117
23 170.9 5.8 12.4X
+Hive built-in ORC 933 956
20 16.9 59.3 1.0X
+Native ORC MR 908 1013
149 17.3 57.7 1.0X
+Native ORC Vectorized 131 166
28 120.2 8.3 7.1X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1269 1319
71 12.4 80.7 1.0X
-Native ORC MR 1087 1088
1 14.5 69.1 1.2X
-Native ORC Vectorized 155 182
24 101.3 9.9 8.2X
+Hive built-in ORC 1051 1087
51 15.0 66.8 1.0X
+Native ORC MR 750 773
26 21.0 47.7 1.4X
+Native ORC Vectorized 116 135
17 136.1 7.3 9.1X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single INT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1369 1466
137 11.5 87.0 1.0X
-Native ORC MR 1103 1277
247 14.3 70.1 1.2X
-Native ORC Vectorized 175 192
26 90.1 11.1 7.8X
+Hive built-in ORC 963 1018
70 16.3 61.2 1.0X
+Native ORC MR 814 822
11 19.3 51.8 1.2X
+Native ORC Vectorized 138 172
21 113.8 8.8 7.0X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single BIGINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1469 1539
98 10.7 93.4 1.0X
-Native ORC MR 1214 1239
36 13.0 77.2 1.2X
-Native ORC Vectorized 256 274
26 61.4 16.3 5.7X
+Hive built-in ORC 1020 1041
29 15.4 64.9 1.0X
+Native ORC MR 828 899
117 19.0 52.6 1.2X
+Native ORC Vectorized 165 191
17 95.4 10.5 6.2X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1467 1480
18 10.7 93.3 1.0X
-Native ORC MR 1239 1303
90 12.7 78.8 1.2X
-Native ORC Vectorized 217 224
8 72.6 13.8 6.8X
+Hive built-in ORC 1018 1022
6 15.5 64.7 1.0X
+Native ORC MR 812 828
17 19.4 51.6 1.3X
+Native ORC Vectorized 175 195
21 90.0 11.1 5.8X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
SQL Single DOUBLE Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1432 1456
33 11.0 91.1 1.0X
-Native ORC MR 1230 1244
20 12.8 78.2 1.2X
-Native ORC Vectorized 243 263
24 64.7 15.5 5.9X
+Hive built-in ORC 1063 1076
19 14.8 67.6 1.0X
+Native ORC MR 864 890
23 18.2 54.9 1.2X
+Native ORC Vectorized 212 235
25 74.1 13.5 5.0X
================================================================================================
@@ -56,12 +56,12 @@ Int and String Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Int and String Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2601 2642
57 4.0 248.1 1.0X
-Native ORC MR 2371 2376
7 4.4 226.1 1.1X
-Native ORC Vectorized 1270 1294
33 8.3 121.2 2.0X
+Hive built-in ORC 2233 2247
19 4.7 213.0 1.0X
+Native ORC MR 1941 1948
10 5.4 185.1 1.2X
+Native ORC Vectorized 1086 1104
26 9.7 103.6 2.1X
================================================================================================
@@ -69,18 +69,18 @@ Partitioned Table Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Partitioned Table: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Data column - Hive built-in ORC 1584 1607
33 9.9 100.7 1.0X
-Data column - Native ORC MR 1502 1537
49 10.5 95.5 1.1X
-Data column - Native ORC Vectorized 268 280
12 58.7 17.0 5.9X
-Partition column - Hive built-in ORC 1209 1212
5 13.0 76.8 1.3X
-Partition column - Native ORC MR 1010 1018
12 15.6 64.2 1.6X
-Partition column - Native ORC Vectorized 52 59
12 301.1 3.3 30.3X
-Both columns - Hive built-in ORC 1712 1735
33 9.2 108.8 0.9X
-Both columns - Native ORC MR 1608 1704
136 9.8 102.2 1.0X
-Both columns - Native ORC Vectorized 303 314
14 51.9 19.3 5.2X
+Data column - Hive built-in ORC 1090 1115
35 14.4 69.3 1.0X
+Data column - Native ORC MR 1025 1048
32 15.3 65.2 1.1X
+Data column - Native ORC Vectorized 170 193
25 92.5 10.8 6.4X
+Partition column - Hive built-in ORC 803 842
40 19.6 51.0 1.4X
+Partition column - Native ORC MR 629 686
50 25.0 40.0 1.7X
+Partition column - Native ORC Vectorized 47 53
10 335.2 3.0 23.2X
+Both columns - Hive built-in ORC 1167 1180
18 13.5 74.2 0.9X
+Both columns - Native ORC MR 1028 1037
12 15.3 65.4 1.1X
+Both columns - Native ORC Vectorized 181 209
46 87.1 11.5 6.0X
================================================================================================
@@ -88,12 +88,12 @@ Repeated String Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Repeated String: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1390 1396
9 7.5 132.5 1.0X
-Native ORC MR 1144 1155
16 9.2 109.1 1.2X
-Native ORC Vectorized 222 240
25 47.2 21.2 6.3X
+Hive built-in ORC 1071 1123
74 9.8 102.1 1.0X
+Native ORC MR 840 871
28 12.5 80.1 1.3X
+Native ORC Vectorized 163 175
9 64.3 15.6 6.6X
================================================================================================
@@ -101,28 +101,28 @@ String with Nulls Scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (0.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2446 2516
100 4.3 233.2 1.0X
-Native ORC MR 2004 2028
34 5.2 191.1 1.2X
-Native ORC Vectorized 556 577
30 18.9 53.0 4.4X
+Hive built-in ORC 2025 2059
48 5.2 193.1 1.0X
+Native ORC MR 1608 1614
7 6.5 153.4 1.3X
+Native ORC Vectorized 500 530
25 21.0 47.7 4.1X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (50.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2207 2267
85 4.8 210.4 1.0X
-Native ORC MR 1942 1946
6 5.4 185.2 1.1X
-Native ORC Vectorized 733 747
14 14.3 69.9 3.0X
+Hive built-in ORC 1853 1862
12 5.7 176.7 1.0X
+Native ORC MR 1537 1563
37 6.8 146.6 1.2X
+Native ORC Vectorized 623 636
15 16.8 59.4 3.0X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
String with Nulls Scan (95.0%): Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1300 1317
24 8.1 124.0 1.0X
-Native ORC MR 1046 1049
3 10.0 99.8 1.2X
-Native ORC Vectorized 274 282
8 38.2 26.2 4.7X
+Hive built-in ORC 958 1005
48 10.9 91.3 1.0X
+Native ORC MR 753 773
27 13.9 71.8 1.3X
+Native ORC Vectorized 196 215
34 53.5 18.7 4.9X
================================================================================================
@@ -130,28 +130,28 @@ Single Column Scan From Wide Columns
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 100 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 1098 1300
285 1.0 1047.4 1.0X
-Native ORC MR 155 162
7 6.7 148.2 7.1X
-Native ORC Vectorized 89 96
9 11.8 84.7 12.4X
+Hive built-in ORC 847 924
85 1.2 807.6 1.0X
+Native ORC MR 115 134
23 9.1 109.5 7.4X
+Native ORC Vectorized 63 74
12 16.7 59.9 13.5X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 200 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 2004 2027
33 0.5 1910.7 1.0X
-Native ORC MR 202 235
24 5.2 192.4 9.9X
-Native ORC Vectorized 128 138
16 8.2 121.7 15.7X
+Hive built-in ORC 1625 1678
76 0.6 1549.4 1.0X
+Native ORC MR 144 162
18 7.3 137.0 11.3X
+Native ORC Vectorized 92 101
13 11.4 87.4 17.7X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Column Scan from 300 columns: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 3016 3041
36 0.3 2875.9 1.0X
-Native ORC MR 247 262
20 4.2 235.5 12.2X
-Native ORC Vectorized 181 201
24 5.8 172.2 16.7X
+Hive built-in ORC 2334 2353
26 0.4 2225.9 1.0X
+Native ORC MR 176 198
33 6.0 167.7 13.3X
+Native ORC Vectorized 128 146
17 8.2 122.2 18.2X
================================================================================================
@@ -159,36 +159,36 @@ Struct scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 523 575
68 2.0 498.5 1.0X
-Native ORC MR 1477 1477
0 0.7 1408.2 0.4X
-Native ORC Vectorized 210 225
14 5.0 199.8 2.5X
+Hive built-in ORC 782 795
19 1.3 746.2 1.0X
+Native ORC MR 288 296
8 3.6 274.5 2.7X
+Native ORC Vectorized 154 163
11 6.8 146.8 5.1X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 100 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 3677 4179
709 0.3 3507.0 1.0X
-Native ORC MR 12584 12615
43 0.1 12001.3 0.3X
-Native ORC Vectorized 1867 1875
11 0.6 1780.9 2.0X
+Hive built-in ORC 5146 5164
25 0.2 4907.5 1.0X
+Native ORC MR 2403 2410
9 0.4 2292.1 2.1X
+Native ORC Vectorized 1333 1339
8 0.8 1271.3 3.9X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 300 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 14307 15087
1103 0.1 13644.5 1.0X
-Native ORC MR 39693 41121
2020 0.0 37853.7 0.4X
-Native ORC Vectorized 39081 39252
243 0.0 37270.5 0.4X
+Hive built-in ORC 15968 17060
1545 0.1 15228.3 1.0X
+Native ORC MR 8321 8459
195 0.1 7935.3 1.9X
+Native ORC Vectorized 8055 8145
127 0.1 7682.2 2.0X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Single Struct Column Scan with 600 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 32291 36141
NaN 0.0 30795.2 1.0X
-Native ORC MR 94939 95045
149 0.0 90541.2 0.3X
-Native ORC Vectorized 93062 93335
386 0.0 88750.4 0.3X
+Hive built-in ORC 36374 37652
1808 0.0 34688.9 1.0X
+Native ORC MR 30738 31431
980 0.0 29313.9 1.2X
+Native ORC Vectorized 27877 28169
413 0.0 26585.4 1.3X
================================================================================================
@@ -196,27 +196,27 @@ Nested Struct scan
================================================================================================
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 10 Elements, 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 4224 5811
2245 0.2 4027.9 1.0X
-Native ORC MR 4213 4759
773 0.2 4017.6 1.0X
-Native ORC Vectorized 1081 1135
75 1.0 1031.3 3.9X
+Hive built-in ORC 4210 4306
135 0.2 4014.8 1.0X
+Native ORC MR 3444 3448
6 0.3 3284.4 1.2X
+Native ORC Vectorized 871 898
25 1.2 830.5 4.8X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 30 Elements, 10 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 11392 11420
40 0.1 10864.3 1.0X
-Native ORC MR 9739 9768
41 0.1 9288.1 1.2X
-Native ORC Vectorized 2735 2823
125 0.4 2608.1 4.2X
+Hive built-in ORC 8711 8751
56 0.1 8307.9 1.0X
+Native ORC MR 7917 7929
17 0.1 7550.6 1.1X
+Native ORC Vectorized 2291 2312
31 0.5 2184.6 3.8X
OpenJDK 64-Bit Server VM 1.8.0_352-b08 on Linux 5.15.0-1023-azure
-Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
+Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
Nested Struct Scan with 10 Elements, 30 Fields: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------------
-Hive built-in ORC 9674 12873
1421 0.1 9225.5 1.0X
-Native ORC MR 12151 16892
2838 0.1 11588.4 0.8X
-Native ORC Vectorized 3333 3448
163 0.3 3178.4 2.9X
+Hive built-in ORC 7205 8030
1166 0.1 6871.5 1.0X
+Native ORC MR 8944 8961
25 0.1 8529.8 0.8X
+Native ORC Vectorized 2341 2372
44 0.4 2232.4 3.1X
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
