dongjoon-hyun commented on code in PR #54313:
URL: https://github.com/apache/spark/pull/54313#discussion_r2805638665
##########
sql/core/benchmarks/DataSourceReadBenchmark-jdk21-results.txt:
##########
@@ -2,437 +2,437 @@
SQL Single Numeric Column Scan
================================================================================================
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
SQL Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-SQL CSV 10326 10410
118 1.5 656.5 1.0X
-SQL Json 7489 7659
241 2.1 476.1 1.4X
-SQL Json with UnsafeRow 8554 8559
6 1.8 543.9 1.2X
-SQL Parquet Vectorized: DataPageV1 83 95
10 190.2 5.3 124.9X
-SQL Parquet Vectorized: DataPageV2 91 101
9 173.0 5.8 113.6X
-SQL Parquet MR: DataPageV1 1922 1931
13 8.2 122.2 5.4X
-SQL Parquet MR: DataPageV2 1740 1743
4 9.0 110.7 5.9X
-SQL ORC Vectorized 139 147
8 113.4 8.8 74.4X
-SQL ORC MR 1483 1496
19 10.6 94.3 7.0X
-
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+SQL CSV 10371 10431
85 1.5 659.4 1.0X
+SQL Json 7575 7729
218 2.1 481.6 1.4X
+SQL Json with UnsafeRow 8696 8712
24 1.8 552.9 1.2X
+SQL Parquet Vectorized: DataPageV1 88 95
7 178.6 5.6 117.7X
+SQL Parquet Vectorized: DataPageV2 93 102
8 168.3 5.9 111.0X
+SQL Parquet MR: DataPageV1 1759 1761
3 8.9 111.9 5.9X
+SQL Parquet MR: DataPageV2 1636 1644
11 9.6 104.0 6.3X
+SQL ORC Vectorized 139 145
7 113.4 8.8 74.8X
+SQL ORC MR 1561 1575
19 10.1 99.3 6.6X
+
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
Parquet Reader Single BOOLEAN Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
-ParquetReader Vectorized: DataPageV1 93 96
1 168.9 5.9 1.0X
-ParquetReader Vectorized: DataPageV2 108 109
2 146.1 6.8 0.9X
-ParquetReader Vectorized -> Row: DataPageV1 75 76
1 209.1 4.8 1.2X
-ParquetReader Vectorized -> Row: DataPageV2 89 90
1 176.9 5.7 1.0X
+ParquetReader Vectorized: DataPageV1 91 92
1 172.6 5.8 1.0X
+ParquetReader Vectorized: DataPageV2 105 107
2 149.2 6.7 0.9X
+ParquetReader Vectorized -> Row: DataPageV1 72 74
2 217.6 4.6 1.3X
+ParquetReader Vectorized -> Row: DataPageV2 88 89
1 178.8 5.6 1.0X
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
SQL Single TINYINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-SQL CSV 9609 9629
27 1.6 611.0 1.0X
-SQL Json 9153 9169
22 1.7 581.9 1.0X
-SQL Json with UnsafeRow 9804 9805
2 1.6 623.3 1.0X
-SQL Parquet Vectorized: DataPageV1 92 103
8 170.8 5.9 104.4X
-SQL Parquet Vectorized: DataPageV2 94 107
8 167.8 6.0 102.5X
-SQL Parquet MR: DataPageV1 1949 1971
31 8.1 123.9 4.9X
-SQL Parquet MR: DataPageV2 1889 1896
9 8.3 120.1 5.1X
-SQL ORC Vectorized 138 142
6 113.9 8.8 69.6X
-SQL ORC MR 1643 1667
34 9.6 104.5 5.8X
-
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+SQL CSV 9487 9506
27 1.7 603.2 1.0X
+SQL Json 8686 8698
17 1.8 552.2 1.1X
+SQL Json with UnsafeRow 9338 9341
5 1.7 593.7 1.0X
+SQL Parquet Vectorized: DataPageV1 93 103
7 168.5 5.9 101.6X
+SQL Parquet Vectorized: DataPageV2 91 105
9 171.9 5.8 103.7X
+SQL Parquet MR: DataPageV1 1947 1957
14 8.1 123.8 4.9X
+SQL Parquet MR: DataPageV2 1883 1892
12 8.4 119.7 5.0X
+SQL ORC Vectorized 111 125
9 141.5 7.1 85.3X
+SQL ORC MR 1788 1796
11 8.8 113.7 5.3X
+
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
Parquet Reader Single TINYINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
-ParquetReader Vectorized: DataPageV1 86 87
1 182.6 5.5 1.0X
-ParquetReader Vectorized: DataPageV2 86 88
1 182.8 5.5 1.0X
-ParquetReader Vectorized -> Row: DataPageV1 51 53
2 310.3 3.2 1.7X
-ParquetReader Vectorized -> Row: DataPageV2 51 52
1 311.4 3.2 1.7X
+ParquetReader Vectorized: DataPageV1 77 78
1 203.7 4.9 1.0X
+ParquetReader Vectorized: DataPageV2 78 78
1 202.4 4.9 1.0X
+ParquetReader Vectorized -> Row: DataPageV1 49 50
1 322.6 3.1 1.6X
+ParquetReader Vectorized -> Row: DataPageV2 49 51
2 320.9 3.1 1.6X
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
SQL Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-SQL CSV 10101 10121
28 1.6 642.2 1.0X
-SQL Json 9160 9164
6 1.7 582.4 1.1X
-SQL Json with UnsafeRow 9855 9856
1 1.6 626.6 1.0X
-SQL Parquet Vectorized: DataPageV1 137 143
4 114.7 8.7 73.6X
-SQL Parquet Vectorized: DataPageV2 145 153
7 108.7 9.2 69.8X
-SQL Parquet MR: DataPageV1 2068 2086
26 7.6 131.5 4.9X
-SQL Parquet MR: DataPageV2 2040 2069
41 7.7 129.7 5.0X
-SQL ORC Vectorized 140 151
8 112.3 8.9 72.1X
-SQL ORC MR 1726 1727
2 9.1 109.7 5.9X
-
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+SQL CSV 9786 9788
2 1.6 622.2 1.0X
+SQL Json 9669 9673
6 1.6 614.7 1.0X
+SQL Json with UnsafeRow 10439 10446
11 1.5 663.7 0.9X
+SQL Parquet Vectorized: DataPageV1 134 140
4 117.7 8.5 73.2X
+SQL Parquet Vectorized: DataPageV2 152 155
3 103.4 9.7 64.4X
+SQL Parquet MR: DataPageV1 2016 2052
50 7.8 128.2 4.9X
+SQL Parquet MR: DataPageV2 2110 2127
25 7.5 134.1 4.6X
+SQL ORC Vectorized 161 163
2 98.0 10.2 61.0X
+SQL ORC MR 1722 1738
23 9.1 109.5 5.7X
+
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
Parquet Reader Single SMALLINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
-ParquetReader Vectorized: DataPageV1 166 168
2 94.6 10.6 1.0X
-ParquetReader Vectorized: DataPageV2 182 184
2 86.5 11.6 0.9X
-ParquetReader Vectorized -> Row: DataPageV1 158 162
2 99.4 10.1 1.1X
-ParquetReader Vectorized -> Row: DataPageV2 173 177
3 91.0 11.0 1.0X
+ParquetReader Vectorized: DataPageV1 167 171
4 94.0 10.6 1.0X
+ParquetReader Vectorized: DataPageV2 193 197
2 81.5 12.3 0.9X
+ParquetReader Vectorized -> Row: DataPageV1 171 179
8 92.1 10.9 1.0X
+ParquetReader Vectorized -> Row: DataPageV2 203 204
2 77.6 12.9 0.8X
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
SQL Single INT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-SQL CSV 11163 11189
37 1.4 709.7 1.0X
-SQL Json 9646 9654
11 1.6 613.3 1.2X
-SQL Json with UnsafeRow 10375 10381
8 1.5 659.6 1.1X
-SQL Parquet Vectorized: DataPageV1 115 128
7 136.3 7.3 96.7X
-SQL Parquet Vectorized: DataPageV2 200 209
6 78.7 12.7 55.9X
-SQL Parquet MR: DataPageV1 2026 2060
49 7.8 128.8 5.5X
-SQL Parquet MR: DataPageV2 2047 2057
14 7.7 130.2 5.5X
-SQL ORC Vectorized 155 164
6 101.2 9.9 71.8X
-SQL ORC MR 1742 1748
9 9.0 110.7 6.4X
-
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+SQL CSV 11008 11010
3 1.4 699.9 1.0X
+SQL Json 10119 10131
18 1.6 643.3 1.1X
+SQL Json with UnsafeRow 10856 10869
19 1.4 690.2 1.0X
+SQL Parquet Vectorized: DataPageV1 109 119
7 144.2 6.9 100.9X
+SQL Parquet Vectorized: DataPageV2 194 203
5 80.9 12.4 56.6X
+SQL Parquet MR: DataPageV1 1960 1964
6 8.0 124.6 5.6X
+SQL Parquet MR: DataPageV2 2081 2094
19 7.6 132.3 5.3X
+SQL ORC Vectorized 175 183
7 89.9 11.1 62.9X
+SQL ORC MR 1778 1787
13 8.8 113.0 6.2X
+
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
Parquet Reader Single INT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
-ParquetReader Vectorized: DataPageV1 153 155
2 102.8 9.7 1.0X
-ParquetReader Vectorized: DataPageV2 240 241
1 65.5 15.3 0.6X
-ParquetReader Vectorized -> Row: DataPageV1 148 149
1 106.6 9.4 1.0X
-ParquetReader Vectorized -> Row: DataPageV2 231 232
1 68.1 14.7 0.7X
+ParquetReader Vectorized: DataPageV1 152 154
2 103.4 9.7 1.0X
+ParquetReader Vectorized: DataPageV2 240 243
2 65.4 15.3 0.6X
+ParquetReader Vectorized -> Row: DataPageV1 155 161
3 101.4 9.9 1.0X
+ParquetReader Vectorized -> Row: DataPageV2 241 245
3 65.1 15.4 0.6X
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
SQL Single BIGINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-SQL CSV 10817 10835
26 1.5 687.7 1.0X
-SQL Json 9477 9480
4 1.7 602.5 1.1X
-SQL Json with UnsafeRow 10140 10150
14 1.6 644.7 1.1X
-SQL Parquet Vectorized: DataPageV1 299 306
5 52.6 19.0 36.2X
-SQL Parquet Vectorized: DataPageV2 237 248
7 66.3 15.1 45.6X
-SQL Parquet MR: DataPageV1 2530 2534
7 6.2 160.8 4.3X
-SQL Parquet MR: DataPageV2 2014 2036
31 7.8 128.1 5.4X
-SQL ORC Vectorized 159 162
2 98.7 10.1 67.9X
-SQL ORC MR 1767 1775
11 8.9 112.4 6.1X
-
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+SQL CSV 10731 10742
15 1.5 682.2 1.0X
+SQL Json 10105 10114
13 1.6 642.5 1.1X
+SQL Json with UnsafeRow 10865 10868
6 1.4 690.7 1.0X
+SQL Parquet Vectorized: DataPageV1 302 315
10 52.1 19.2 35.6X
+SQL Parquet Vectorized: DataPageV2 257 270
7 61.3 16.3 41.8X
+SQL Parquet MR: DataPageV1 2430 2434
5 6.5 154.5 4.4X
+SQL Parquet MR: DataPageV2 1911 1930
27 8.2 121.5 5.6X
+SQL ORC Vectorized 180 201
8 87.2 11.5 59.5X
+SQL ORC MR 1761 1788
38 8.9 111.9 6.1X
+
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
Parquet Reader Single BIGINT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
-ParquetReader Vectorized: DataPageV1 323 325
1 48.6 20.6 1.0X
-ParquetReader Vectorized: DataPageV2 273 275
2 57.6 17.4 1.2X
-ParquetReader Vectorized -> Row: DataPageV1 329 334
7 47.8 20.9 1.0X
-ParquetReader Vectorized -> Row: DataPageV2 272 285
10 57.9 17.3 1.2X
+ParquetReader Vectorized: DataPageV1 335 338
3 46.9 21.3 1.0X
+ParquetReader Vectorized: DataPageV2 299 303
4 52.7 19.0 1.1X
+ParquetReader Vectorized -> Row: DataPageV1 343 344
1 45.9 21.8 1.0X
+ParquetReader Vectorized -> Row: DataPageV2 284 288
4 55.4 18.0 1.2X
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
SQL Single FLOAT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-SQL CSV 11124 11128
5 1.4 707.2 1.0X
-SQL Json 11288 11317
40 1.4 717.7 1.0X
-SQL Json with UnsafeRow 12153 12166
19 1.3 772.7 0.9X
-SQL Parquet Vectorized: DataPageV1 105 109
3 150.1 6.7 106.1X
-SQL Parquet Vectorized: DataPageV2 96 108
6 163.1 6.1 115.3X
-SQL Parquet MR: DataPageV1 1997 2014
25 7.9 126.9 5.6X
-SQL Parquet MR: DataPageV2 1956 1960
5 8.0 124.4 5.7X
-SQL ORC Vectorized 237 244
8 66.3 15.1 46.9X
-SQL ORC MR 1793 1811
26 8.8 114.0 6.2X
-
-OpenJDK 64-Bit Server VM 21.0.8+9-LTS on Linux 6.11.0-1018-azure
+SQL CSV 10942 10942
1 1.4 695.6 1.0X
+SQL Json 11987 12003
22 1.3 762.1 0.9X
+SQL Json with UnsafeRow 12629 12643
19 1.2 802.9 0.9X
+SQL Parquet Vectorized: DataPageV1 105 110
3 149.2 6.7 103.8X
+SQL Parquet Vectorized: DataPageV2 105 110
5 149.4 6.7 103.9X
+SQL Parquet MR: DataPageV1 2017 2064
66 7.8 128.2 5.4X
+SQL Parquet MR: DataPageV2 1936 1966
42 8.1 123.1 5.7X
+SQL ORC Vectorized 240 258
13 65.6 15.2 45.7X
+SQL ORC MR 1906 1915
12 8.3 121.2 5.7X
+
+OpenJDK 64-Bit Server VM 21.0.10+7-LTS on Linux 6.14.0-1017-azure
AMD EPYC 7763 64-Core Processor
Parquet Reader Single FLOAT Column Scan: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
-ParquetReader Vectorized: DataPageV1 181 183
1 86.9 11.5 1.0X
-ParquetReader Vectorized: DataPageV2 185 187
2 85.1 11.7 1.0X
-ParquetReader Vectorized -> Row: DataPageV1 147 148
1 107.0 9.3 1.2X
-ParquetReader Vectorized -> Row: DataPageV2 150 152
4 105.1 9.5 1.2X
+ParquetReader Vectorized: DataPageV1 151 153
1 103.9 9.6 1.0X
+ParquetReader Vectorized: DataPageV2 156 158
2 100.7 9.9 1.0X
+ParquetReader Vectorized -> Row: DataPageV1 162 163
1 97.2 10.3 0.9X
+ParquetReader Vectorized -> Row: DataPageV2 162 163
2 97.1 10.3 0.9X
Review Comment:
The ratio is reverted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
