MaxGekk commented on a change in pull request #30118:
URL: https://github.com/apache/spark/pull/30118#discussion_r509171281
##########
File path: sql/core/benchmarks/DateTimeRebaseBenchmark-jdk11-results.txt
##########
@@ -2,153 +2,153 @@
Rebasing dates/timestamps in Parquet datasource
================================================================================================
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on Linux
5.3.0-1034-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save DATE to parquet: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-after 1582, noop 20023 20023
0 5.0 200.2 1.0X
-before 1582, noop 10729 10729
0 9.3 107.3 1.9X
-after 1582, rebase EXCEPTION 31834 31834
0 3.1 318.3 0.6X
-after 1582, rebase LEGACY 31997 31997
0 3.1 320.0 0.6X
-after 1582, rebase CORRECTED 31712 31712
0 3.2 317.1 0.6X
-before 1582, rebase LEGACY 23663 23663
0 4.2 236.6 0.8X
-before 1582, rebase CORRECTED 22749 22749
0 4.4 227.5 0.9X
-
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+after 1582, noop 21041 21041
0 4.8 210.4 1.0X
+before 1582, noop 11202 11202
0 8.9 112.0 1.9X
+after 1582, rebase EXCEPTION 32810 32810
0 3.0 328.1 0.6X
+after 1582, rebase LEGACY 32530 32530
0 3.1 325.3 0.6X
+after 1582, rebase CORRECTED 32849 32849
0 3.0 328.5 0.6X
+before 1582, rebase LEGACY 23537 23537
0 4.2 235.4 0.9X
+before 1582, rebase CORRECTED 22870 22870
0 4.4 228.7 0.9X
+
+OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on Linux
5.3.0-1034-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load DATE from parquet: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-after 1582, vec off, rebase EXCEPTION 12984 13262
257 7.7 129.8 1.0X
-after 1582, vec off, rebase LEGACY 13278 13330
50 7.5 132.8 1.0X
-after 1582, vec off, rebase CORRECTED 13202 13255
50 7.6 132.0 1.0X
-after 1582, vec on, rebase EXCEPTION 3823 3853
40 26.2 38.2 3.4X
-after 1582, vec on, rebase LEGACY 3846 3876
27 26.0 38.5 3.4X
-after 1582, vec on, rebase CORRECTED 3775 3838
62 26.5 37.7 3.4X
-before 1582, vec off, rebase LEGACY 13671 13692
26 7.3 136.7 0.9X
-before 1582, vec off, rebase CORRECTED 13387 13476
106 7.5 133.9 1.0X
-before 1582, vec on, rebase LEGACY 4477 4484
7 22.3 44.8 2.9X
-before 1582, vec on, rebase CORRECTED 3729 3773
50 26.8 37.3 3.5X
-
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+after 1582, vec off, rebase EXCEPTION 13114 13225
104 7.6 131.1 1.0X
+after 1582, vec off, rebase LEGACY 13175 13189
15 7.6 131.8 1.0X
+after 1582, vec off, rebase CORRECTED 13080 13115
34 7.6 130.8 1.0X
+after 1582, vec on, rebase EXCEPTION 3698 3726
29 27.0 37.0 3.5X
+after 1582, vec on, rebase LEGACY 3730 3745
17 26.8 37.3 3.5X
+after 1582, vec on, rebase CORRECTED 3714 3758
75 26.9 37.1 3.5X
+before 1582, vec off, rebase LEGACY 13519 13575
63 7.4 135.2 1.0X
+before 1582, vec off, rebase CORRECTED 13210 13309
108 7.6 132.1 1.0X
+before 1582, vec on, rebase LEGACY 4459 4488
44 22.4 44.6 2.9X
+before 1582, vec on, rebase CORRECTED 3661 3718
88 27.3 36.6 3.6X
+
+OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on Linux
5.3.0-1034-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save TIMESTAMP_INT96 to parquet: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-after 1900, noop 3020 3020
0 33.1 30.2 1.0X
-before 1900, noop 3013 3013
0 33.2 30.1 1.0X
-after 1900, rebase EXCEPTION 28796 28796
0 3.5 288.0 0.1X
-after 1900, rebase LEGACY 28869 28869
0 3.5 288.7 0.1X
-after 1900, rebase CORRECTED 28522 28522
0 3.5 285.2 0.1X
-before 1900, rebase LEGACY 30594 30594
0 3.3 305.9 0.1X
-before 1900, rebase CORRECTED 30743 30743
0 3.3 307.4 0.1X
-
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+after 1900, noop 2900 2900
0 34.5 29.0 1.0X
+before 1900, noop 2848 2848
0 35.1 28.5 1.0X
+after 1900, rebase EXCEPTION 27623 27623
0 3.6 276.2 0.1X
+after 1900, rebase LEGACY 27305 27305
0 3.7 273.0 0.1X
+after 1900, rebase CORRECTED 27715 27715
0 3.6 277.2 0.1X
+before 1900, rebase LEGACY 30911 30911
0 3.2 309.1 0.1X
+before 1900, rebase CORRECTED 27944 27944
0 3.6 279.4 0.1X
+
+OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on Linux
5.3.0-1034-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load TIMESTAMP_INT96 from parquet: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-after 1900, vec off, rebase EXCEPTION 19325 19468
135 5.2 193.3 1.0X
-after 1900, vec off, rebase LEGACY 19568 19602
30 5.1 195.7 1.0X
-after 1900, vec off, rebase CORRECTED 19532 19538
6 5.1 195.3 1.0X
-after 1900, vec on, rebase EXCEPTION 9884 9990
94 10.1 98.8 2.0X
-after 1900, vec on, rebase LEGACY 9933 9985
49 10.1 99.3 1.9X
-after 1900, vec on, rebase CORRECTED 9967 10043
76 10.0 99.7 1.9X
-before 1900, vec off, rebase LEGACY 24162 24198
37 4.1 241.6 0.8X
-before 1900, vec off, rebase CORRECTED 24034 24056
20 4.2 240.3 0.8X
-before 1900, vec on, rebase LEGACY 12548 12625
72 8.0 125.5 1.5X
-before 1900, vec on, rebase CORRECTED 12580 12660
115 7.9 125.8 1.5X
-
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+after 1900, vec off, rebase EXCEPTION 16853 16885
41 5.9 168.5 1.0X
+after 1900, vec off, rebase LEGACY 16804 16816
21 6.0 168.0 1.0X
+after 1900, vec off, rebase CORRECTED 16985 17020
58 5.9 169.9 1.0X
+after 1900, vec on, rebase EXCEPTION 7044 7063
19 14.2 70.4 2.4X
+after 1900, vec on, rebase LEGACY 7183 7255
94 13.9 71.8 2.3X
+after 1900, vec on, rebase CORRECTED 7047 7137
86 14.2 70.5 2.4X
+before 1900, vec off, rebase LEGACY 20371 20458
81 4.9 203.7 0.8X
+before 1900, vec off, rebase CORRECTED 17484 17541
54 5.7 174.8 1.0X
+before 1900, vec on, rebase LEGACY 10284 10327
45 9.7 102.8 1.6X
+before 1900, vec on, rebase CORRECTED 7044 7073
37 14.2 70.4 2.4X
Review comment:
Vectorized Reader speed up: ~30%
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
