MaxGekk commented on a change in pull request #30118:
URL: https://github.com/apache/spark/pull/30118#discussion_r509170757
##########
File path: sql/core/benchmarks/DateTimeRebaseBenchmark-jdk11-results.txt
##########
@@ -2,153 +2,153 @@
Rebasing dates/timestamps in Parquet datasource
================================================================================================
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on Linux
5.3.0-1034-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save DATE to parquet: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-after 1582, noop 20023 20023
0 5.0 200.2 1.0X
-before 1582, noop 10729 10729
0 9.3 107.3 1.9X
-after 1582, rebase EXCEPTION 31834 31834
0 3.1 318.3 0.6X
-after 1582, rebase LEGACY 31997 31997
0 3.1 320.0 0.6X
-after 1582, rebase CORRECTED 31712 31712
0 3.2 317.1 0.6X
-before 1582, rebase LEGACY 23663 23663
0 4.2 236.6 0.8X
-before 1582, rebase CORRECTED 22749 22749
0 4.4 227.5 0.9X
-
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+after 1582, noop 21041 21041
0 4.8 210.4 1.0X
+before 1582, noop 11202 11202
0 8.9 112.0 1.9X
+after 1582, rebase EXCEPTION 32810 32810
0 3.0 328.1 0.6X
+after 1582, rebase LEGACY 32530 32530
0 3.1 325.3 0.6X
+after 1582, rebase CORRECTED 32849 32849
0 3.0 328.5 0.6X
+before 1582, rebase LEGACY 23537 23537
0 4.2 235.4 0.9X
+before 1582, rebase CORRECTED 22870 22870
0 4.4 228.7 0.9X
+
+OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on Linux
5.3.0-1034-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load DATE from parquet: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-after 1582, vec off, rebase EXCEPTION 12984 13262
257 7.7 129.8 1.0X
-after 1582, vec off, rebase LEGACY 13278 13330
50 7.5 132.8 1.0X
-after 1582, vec off, rebase CORRECTED 13202 13255
50 7.6 132.0 1.0X
-after 1582, vec on, rebase EXCEPTION 3823 3853
40 26.2 38.2 3.4X
-after 1582, vec on, rebase LEGACY 3846 3876
27 26.0 38.5 3.4X
-after 1582, vec on, rebase CORRECTED 3775 3838
62 26.5 37.7 3.4X
-before 1582, vec off, rebase LEGACY 13671 13692
26 7.3 136.7 0.9X
-before 1582, vec off, rebase CORRECTED 13387 13476
106 7.5 133.9 1.0X
-before 1582, vec on, rebase LEGACY 4477 4484
7 22.3 44.8 2.9X
-before 1582, vec on, rebase CORRECTED 3729 3773
50 26.8 37.3 3.5X
-
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+after 1582, vec off, rebase EXCEPTION 13114 13225
104 7.6 131.1 1.0X
+after 1582, vec off, rebase LEGACY 13175 13189
15 7.6 131.8 1.0X
+after 1582, vec off, rebase CORRECTED 13080 13115
34 7.6 130.8 1.0X
+after 1582, vec on, rebase EXCEPTION 3698 3726
29 27.0 37.0 3.5X
+after 1582, vec on, rebase LEGACY 3730 3745
17 26.8 37.3 3.5X
+after 1582, vec on, rebase CORRECTED 3714 3758
75 26.9 37.1 3.5X
+before 1582, vec off, rebase LEGACY 13519 13575
63 7.4 135.2 1.0X
+before 1582, vec off, rebase CORRECTED 13210 13309
108 7.6 132.1 1.0X
+before 1582, vec on, rebase LEGACY 4459 4488
44 22.4 44.6 2.9X
+before 1582, vec on, rebase CORRECTED 3661 3718
88 27.3 36.6 3.6X
+
+OpenJDK 64-Bit Server VM 11.0.8+10-post-Ubuntu-0ubuntu118.04.1 on Linux
5.3.0-1034-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save TIMESTAMP_INT96 to parquet: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-after 1900, noop 3020 3020
0 33.1 30.2 1.0X
-before 1900, noop 3013 3013
0 33.2 30.1 1.0X
-after 1900, rebase EXCEPTION 28796 28796
0 3.5 288.0 0.1X
-after 1900, rebase LEGACY 28869 28869
0 3.5 288.7 0.1X
-after 1900, rebase CORRECTED 28522 28522
0 3.5 285.2 0.1X
-before 1900, rebase LEGACY 30594 30594
0 3.3 305.9 0.1X
-before 1900, rebase CORRECTED 30743 30743
0 3.3 307.4 0.1X
-
-OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux
4.15.0-1063-aws
+after 1900, noop 2900 2900
0 34.5 29.0 1.0X
+before 1900, noop 2848 2848
0 35.1 28.5 1.0X
+after 1900, rebase EXCEPTION 27623 27623
0 3.6 276.2 0.1X
+after 1900, rebase LEGACY 27305 27305
0 3.7 273.0 0.1X
+after 1900, rebase CORRECTED 27715 27715
0 3.6 277.2 0.1X
+before 1900, rebase LEGACY 30911 30911
0 3.2 309.1 0.1X
+before 1900, rebase CORRECTED 27944 27944
0 3.6 279.4 0.1X
Review comment:
Parquet writer without rebasing is ~10% faster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
