This is an automated email from the ASF dual-hosted git repository. wenchen pushed a commit to branch branch-3.0 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 76874cf [SPARK-31111][SQL][TESTS] Fix interval output issue in
ExtractBenchmark
76874cf is described below
commit 76874cfa4a9f6ada746f9c80a3a5a4d569f1a5a8
Author: Kent Yao <[email protected]>
AuthorDate: Wed Mar 11 20:13:59 2020 +0800
[SPARK-31111][SQL][TESTS] Fix interval output issue in ExtractBenchmark
### What changes were proposed in this pull request?
fix the error caused by interval output in ExtractBenchmark
### Why are the changes needed?
fix a bug in the test
```scala
[info] Running case: cast to interval
[error] Exception in thread "main" org.apache.spark.sql.AnalysisException:
Cannot use interval type in the table schema.;;
[error] OverwriteByExpression RelationV2[] noop-table, true, true
[error] +- Project [(subtractdates(cast(cast(id#0L as timestamp) as date),
-719162) + subtracttimestamps(cast(id#0L as timestamp), -30610249419876544)) AS
((CAST(CAST(id AS TIMESTAMP) AS DATE) - DATE '0001-01-01') + (CAST(id AS
TIMESTAMP) - TIMESTAMP '1000-01-01 01:02:03.123456'))#2]
[error] +- Range (1262304000, 1272304000, step=1, splits=Some(1))
[error]
[error] at
org.apache.spark.sql.catalyst.util.TypeUtils$.failWithIntervalType(TypeUtils.scala:106)
[error] at
org.apache.spark.sql.catalyst.analysis.CheckAnalysis.$anonfun$checkAnalysis$25(CheckAnalysis.scala:389)
[error] at org.a
```
### Does this PR introduce any user-facing change?
no
### How was this patch tested?
re-run benchmark
Closes #27867 from yaooqinn/SPARK-31111.
Authored-by: Kent Yao <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 2b46662bd0335320a668d2247a9c62552da07336)
Signed-off-by: Wenchen Fan <[email protected]>
---
.../benchmarks/ExtractBenchmark-jdk11-results.txt | 208 ++++++++++-----------
sql/core/benchmarks/ExtractBenchmark-results.txt | 208 ++++++++++-----------
.../sql/execution/benchmark/ExtractBenchmark.scala | 4 +-
3 files changed, 211 insertions(+), 209 deletions(-)
diff --git a/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
b/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
index 0572cc2..0873215 100644
--- a/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
+++ b/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
@@ -1,119 +1,119 @@
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for timestamp: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 380 456
67 26.3 38.0 1.0X
-MILLENNIUM of timestamp 1274 1361
138 7.9 127.4 0.3X
-CENTURY of timestamp 1119 1132
19 8.9 111.9 0.3X
-DECADE of timestamp 1076 1083
6 9.3 107.6 0.4X
-YEAR of timestamp 1066 1098
43 9.4 106.6 0.4X
-ISOYEAR of timestamp 1190 1194
4 8.4 119.0 0.3X
-QUARTER of timestamp 1269 1273
4 7.9 126.9 0.3X
-MONTH of timestamp 1060 1075
22 9.4 106.0 0.4X
-WEEK of timestamp 1560 1565
8 6.4 156.0 0.2X
-DAY of timestamp 1039 1046
8 9.6 103.9 0.4X
-DAYOFWEEK of timestamp 1248 1274
24 8.0 124.8 0.3X
-DOW of timestamp 1252 1273
25 8.0 125.2 0.3X
-ISODOW of timestamp 1195 1204
9 8.4 119.5 0.3X
-DOY of timestamp 1081 1086
6 9.3 108.1 0.4X
-HOUR of timestamp 778 781
5 12.9 77.8 0.5X
-MINUTE of timestamp 779 780
1 12.8 77.9 0.5X
-SECOND of timestamp 597 611
20 16.7 59.7 0.6X
-MILLISECONDS of timestamp 636 642
6 15.7 63.6 0.6X
-MICROSECONDS of timestamp 498 504
5 20.1 49.8 0.8X
-EPOCH of timestamp 946 956
9 10.6 94.6 0.4X
+cast to timestamp 343 348
5 29.1 34.3 1.0X
+MILLENNIUM of timestamp 888 899
14 11.3 88.8 0.4X
+CENTURY of timestamp 835 846
15 12.0 83.5 0.4X
+DECADE of timestamp 824 834
9 12.1 82.4 0.4X
+YEAR of timestamp 818 824
10 12.2 81.8 0.4X
+ISOYEAR of timestamp 879 890
17 11.4 87.9 0.4X
+QUARTER of timestamp 965 1073
96 10.4 96.5 0.4X
+MONTH of timestamp 787 793
8 12.7 78.7 0.4X
+WEEK of timestamp 1277 1341
57 7.8 127.7 0.3X
+DAY of timestamp 780 785
8 12.8 78.0 0.4X
+DAYOFWEEK of timestamp 956 962
7 10.5 95.6 0.4X
+DOW of timestamp 995 1034
36 10.1 99.5 0.3X
+ISODOW of timestamp 885 898
11 11.3 88.5 0.4X
+DOY of timestamp 808 810
3 12.4 80.8 0.4X
+HOUR of timestamp 622 627
7 16.1 62.2 0.6X
+MINUTE of timestamp 629 636
8 15.9 62.9 0.5X
+SECOND of timestamp 800 819
30 12.5 80.0 0.4X
+MILLISECONDS of timestamp 743 755
21 13.5 74.3 0.5X
+MICROSECONDS of timestamp 692 730
53 14.5 69.2 0.5X
+EPOCH of timestamp 773 798
40 12.9 77.3 0.4X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for timestamp: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 356 362
9 28.1 35.6 1.0X
-MILLENNIUM of timestamp 1260 1297
41 7.9 126.0 0.3X
-CENTURY of timestamp 1082 1085
3 9.2 108.2 0.3X
-DECADE of timestamp 1056 1068
19 9.5 105.6 0.3X
-YEAR of timestamp 1045 1053
13 9.6 104.5 0.3X
-ISOYEAR of timestamp 1300 1316
25 7.7 130.0 0.3X
-QUARTER of timestamp 1279 1280
2 7.8 127.9 0.3X
-MONTH of timestamp 1037 1046
11 9.6 103.7 0.3X
-WEEK of timestamp 1539 1557
28 6.5 153.9 0.2X
-DAY of timestamp 1032 1038
6 9.7 103.2 0.3X
-DAYOFWEEK of timestamp 1241 1244
4 8.1 124.1 0.3X
-DOW of timestamp 1237 1241
7 8.1 123.7 0.3X
-ISODOW of timestamp 1155 1158
3 8.7 115.5 0.3X
-DOY of timestamp 1075 1080
4 9.3 107.5 0.3X
-HOUR of timestamp 766 770
5 13.1 76.6 0.5X
-MINUTE of timestamp 764 769
4 13.1 76.4 0.5X
-SECOND of timestamp 590 592
2 16.9 59.0 0.6X
-MILLISECONDS of timestamp 627 636
10 16.0 62.7 0.6X
-MICROSECONDS of timestamp 493 505
15 20.3 49.3 0.7X
-EPOCH of timestamp 962 966
4 10.4 96.2 0.4X
+cast to timestamp 303 323
24 33.0 30.3 1.0X
+MILLENNIUM of timestamp 841 852
9 11.9 84.1 0.4X
+CENTURY of timestamp 818 821
4 12.2 81.8 0.4X
+DECADE of timestamp 821 836
15 12.2 82.1 0.4X
+YEAR of timestamp 858 928
61 11.7 85.8 0.4X
+ISOYEAR of timestamp 933 939
6 10.7 93.3 0.3X
+QUARTER of timestamp 979 1004
26 10.2 97.9 0.3X
+MONTH of timestamp 755 784
25 13.2 75.5 0.4X
+WEEK of timestamp 1189 1228
64 8.4 118.9 0.3X
+DAY of timestamp 768 770
4 13.0 76.8 0.4X
+DAYOFWEEK of timestamp 918 948
27 10.9 91.8 0.3X
+DOW of timestamp 933 983
53 10.7 93.3 0.3X
+ISODOW of timestamp 884 928
61 11.3 88.4 0.3X
+DOY of timestamp 787 797
12 12.7 78.7 0.4X
+HOUR of timestamp 611 664
58 16.4 61.1 0.5X
+MINUTE of timestamp 616 622
5 16.2 61.6 0.5X
+SECOND of timestamp 732 748
14 13.7 73.2 0.4X
+MILLISECONDS of timestamp 705 716
17 14.2 70.5 0.4X
+MICROSECONDS of timestamp 642 664
33 15.6 64.2 0.5X
+EPOCH of timestamp 760 781
18 13.2 76.0 0.4X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for date: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 886 905
17 11.3 88.6 1.0X
-MILLENNIUM of date 1253 1261
7 8.0 125.3 0.7X
-CENTURY of date 1068 1079
10 9.4 106.8 0.8X
-DECADE of date 1040 1068
25 9.6 104.0 0.9X
-YEAR of date 1032 1043
11 9.7 103.2 0.9X
-ISOYEAR of date 1304 1313
12 7.7 130.4 0.7X
-QUARTER of date 1284 1301
16 7.8 128.4 0.7X
-MONTH of date 1033 1036
4 9.7 103.3 0.9X
-WEEK of date 1535 1545
8 6.5 153.5 0.6X
-DAY of date 1023 1033
11 9.8 102.3 0.9X
-DAYOFWEEK of date 1230 1236
6 8.1 123.0 0.7X
-DOW of date 1238 1247
9 8.1 123.8 0.7X
-ISODOW of date 1159 1169
17 8.6 115.9 0.8X
-DOY of date 1082 1084
3 9.2 108.2 0.8X
-HOUR of date 1879 1891
11 5.3 187.9 0.5X
-MINUTE of date 1881 1905
21 5.3 188.1 0.5X
-SECOND of date 1718 1724
5 5.8 171.8 0.5X
-MILLISECONDS of date 1733 1737
6 5.8 173.3 0.5X
-MICROSECONDS of date 1629 1644
23 6.1 162.9 0.5X
-EPOCH of date 2085 2090
5 4.8 208.5 0.4X
+cast to date 712 786
120 14.0 71.2 1.0X
+MILLENNIUM of date 846 888
73 11.8 84.6 0.8X
+CENTURY of date 781 792
11 12.8 78.1 0.9X
+DECADE of date 770 824
48 13.0 77.0 0.9X
+YEAR of date 804 832
27 12.4 80.4 0.9X
+ISOYEAR of date 909 931
24 11.0 90.9 0.8X
+QUARTER of date 957 975
22 10.4 95.7 0.7X
+MONTH of date 789 794
7 12.7 78.9 0.9X
+WEEK of date 1141 1165
29 8.8 114.1 0.6X
+DAY of date 784 800
22 12.8 78.4 0.9X
+DAYOFWEEK of date 907 916
14 11.0 90.7 0.8X
+DOW of date 931 958
25 10.7 93.1 0.8X
+ISODOW of date 852 857
7 11.7 85.2 0.8X
+DOY of date 831 870
45 12.0 83.1 0.9X
+HOUR of date 1574 1598
26 6.4 157.4 0.5X
+MINUTE of date 1525 1590
58 6.6 152.5 0.5X
+SECOND of date 1728 1739
10 5.8 172.8 0.4X
+MILLISECONDS of date 1715 1727
10 5.8 171.5 0.4X
+MICROSECONDS of date 1559 1669
95 6.4 155.9 0.5X
+EPOCH of date 1766 1774
9 5.7 176.6 0.4X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for date: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 888 891
6 11.3 88.8 1.0X
-MILLENNIUM of date 1250 1260
17 8.0 125.0 0.7X
-CENTURY of date 1070 1076
6 9.3 107.0 0.8X
-DECADE of date 1036 1041
5 9.6 103.6 0.9X
-YEAR of date 1037 1038
1 9.6 103.7 0.9X
-ISOYEAR of date 1300 1307
9 7.7 130.0 0.7X
-QUARTER of date 1267 1277
9 7.9 126.7 0.7X
-MONTH of date 1034 1037
4 9.7 103.4 0.9X
-WEEK of date 1543 1554
10 6.5 154.3 0.6X
-DAY of date 1022 1030
12 9.8 102.2 0.9X
-DAYOFWEEK of date 1230 1232
4 8.1 123.0 0.7X
-DOW of date 1227 1242
15 8.1 122.7 0.7X
-ISODOW of date 1157 1173
20 8.6 115.7 0.8X
-DOY of date 1073 1083
18 9.3 107.3 0.8X
-HOUR of date 1873 1878
7 5.3 187.3 0.5X
-MINUTE of date 1861 1876
14 5.4 186.1 0.5X
-SECOND of date 1717 1724
6 5.8 171.7 0.5X
-MILLISECONDS of date 1729 1736
7 5.8 172.9 0.5X
-MICROSECONDS of date 1622 1627
5 6.2 162.2 0.5X
-EPOCH of date 2066 2079
19 4.8 206.6 0.4X
+cast to date 803 825
23 12.5 80.3 1.0X
+MILLENNIUM of date 918 924
8 10.9 91.8 0.9X
+CENTURY of date 855 879
25 11.7 85.5 0.9X
+DECADE of date 777 795
19 12.9 77.7 1.0X
+YEAR of date 772 779
9 13.0 77.2 1.0X
+ISOYEAR of date 903 927
21 11.1 90.3 0.9X
+QUARTER of date 983 1339
381 10.2 98.3 0.8X
+MONTH of date 764 780
20 13.1 76.4 1.1X
+WEEK of date 1239 1316
112 8.1 123.9 0.6X
+DAY of date 769 857
91 13.0 76.9 1.0X
+DAYOFWEEK of date 900 965
61 11.1 90.0 0.9X
+DOW of date 990 1041
62 10.1 99.0 0.8X
+ISODOW of date 974 977
5 10.3 97.4 0.8X
+DOY of date 862 918
50 11.6 86.2 0.9X
+HOUR of date 1491 1517
25 6.7 149.1 0.5X
+MINUTE of date 1514 1520
9 6.6 151.4 0.5X
+SECOND of date 1657 1677
27 6.0 165.7 0.5X
+MILLISECONDS of date 1646 1669
26 6.1 164.6 0.5X
+MICROSECONDS of date 1538 1556
23 6.5 153.8 0.5X
+EPOCH of date 1699 1718
28 5.9 169.9 0.5X
-OpenJDK 64-Bit Server VM 11.0.5+10-post-Ubuntu-0ubuntu1.118.04 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 11.0.5+10-LTS on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for interval: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to interval 1273 1279
6 7.9 127.3 1.0X
-MILLENNIUM of interval 1323 1352
36 7.6 132.3 1.0X
-CENTURY of interval 1353 1356
4 7.4 135.3 0.9X
-DECADE of interval 1326 1339
11 7.5 132.6 1.0X
-YEAR of interval 1341 1345
3 7.5 134.1 0.9X
-QUARTER of interval 1368 1372
4 7.3 136.8 0.9X
-MONTH of interval 1320 1326
6 7.6 132.0 1.0X
-DAY of interval 1306 1310
4 7.7 130.6 1.0X
-HOUR of interval 1341 1347
8 7.5 134.1 0.9X
-MINUTE of interval 1337 1349
11 7.5 133.7 1.0X
-SECOND of interval 1450 1451
1 6.9 145.0 0.9X
-MILLISECONDS of interval 1476 1490
23 6.8 147.6 0.9X
-MICROSECONDS of interval 1316 1331
25 7.6 131.6 1.0X
-EPOCH of interval 1461 1462
1 6.8 146.1 0.9X
+cast to interval 971 976
5 10.3 97.1 1.0X
+MILLENNIUM of interval 983 1009
23 10.2 98.3 1.0X
+CENTURY of interval 972 984
12 10.3 97.2 1.0X
+DECADE of interval 954 962
10 10.5 95.4 1.0X
+YEAR of interval 954 973
17 10.5 95.4 1.0X
+QUARTER of interval 1009 1020
11 9.9 100.9 1.0X
+MONTH of interval 946 963
16 10.6 94.6 1.0X
+DAY of interval 952 963
12 10.5 95.2 1.0X
+HOUR of interval 948 960
11 10.5 94.8 1.0X
+MINUTE of interval 1035 1040
4 9.7 103.5 0.9X
+SECOND of interval 1085 1105
31 9.2 108.5 0.9X
+MILLISECONDS of interval 1065 1088
24 9.4 106.5 0.9X
+MICROSECONDS of interval 992 1007
17 10.1 99.2 1.0X
+EPOCH of interval 1087 1103
25 9.2 108.7 0.9X
diff --git a/sql/core/benchmarks/ExtractBenchmark-results.txt
b/sql/core/benchmarks/ExtractBenchmark-results.txt
index 317f107..36bdbd0 100644
--- a/sql/core/benchmarks/ExtractBenchmark-results.txt
+++ b/sql/core/benchmarks/ExtractBenchmark-results.txt
@@ -1,119 +1,119 @@
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for timestamp: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 409 457
45 24.5 40.9 1.0X
-MILLENNIUM of timestamp 1240 1295
57 8.1 124.0 0.3X
-CENTURY of timestamp 1186 1240
49 8.4 118.6 0.3X
-DECADE of timestamp 1083 1104
20 9.2 108.3 0.4X
-YEAR of timestamp 1061 1073
15 9.4 106.1 0.4X
-ISOYEAR of timestamp 1198 1213
25 8.3 119.8 0.3X
-QUARTER of timestamp 1304 1322
23 7.7 130.4 0.3X
-MONTH of timestamp 1052 1067
19 9.5 105.2 0.4X
-WEEK of timestamp 1534 1558
25 6.5 153.4 0.3X
-DAY of timestamp 1038 1057
26 9.6 103.8 0.4X
-DAYOFWEEK of timestamp 1226 1239
22 8.2 122.6 0.3X
-DOW of timestamp 1212 1224
13 8.3 121.2 0.3X
-ISODOW of timestamp 1148 1165
24 8.7 114.8 0.4X
-DOY of timestamp 1066 1075
14 9.4 106.6 0.4X
-HOUR of timestamp 358 362
6 27.9 35.8 1.1X
-MINUTE of timestamp 364 369
4 27.4 36.4 1.1X
-SECOND of timestamp 453 471
26 22.1 45.3 0.9X
-MILLISECONDS of timestamp 497 500
2 20.1 49.7 0.8X
-MICROSECONDS of timestamp 360 363
4 27.8 36.0 1.1X
-EPOCH of timestamp 1100 1104
5 9.1 110.0 0.4X
+cast to timestamp 287 308
19 34.8 28.7 1.0X
+MILLENNIUM of timestamp 896 918
20 11.2 89.6 0.3X
+CENTURY of timestamp 849 856
7 11.8 84.9 0.3X
+DECADE of timestamp 765 777
17 13.1 76.5 0.4X
+YEAR of timestamp 754 756
2 13.3 75.4 0.4X
+ISOYEAR of timestamp 843 849
5 11.9 84.3 0.3X
+QUARTER of timestamp 867 873
9 11.5 86.7 0.3X
+MONTH of timestamp 758 762
4 13.2 75.8 0.4X
+WEEK of timestamp 1049 1054
6 9.5 104.9 0.3X
+DAY of timestamp 750 763
11 13.3 75.0 0.4X
+DAYOFWEEK of timestamp 890 918
25 11.2 89.0 0.3X
+DOW of timestamp 879 887
8 11.4 87.9 0.3X
+ISODOW of timestamp 862 869
11 11.6 86.2 0.3X
+DOY of timestamp 811 868
55 12.3 81.1 0.4X
+HOUR of timestamp 627 638
11 16.0 62.7 0.5X
+MINUTE of timestamp 600 606
6 16.7 60.0 0.5X
+SECOND of timestamp 743 799
51 13.5 74.3 0.4X
+MILLISECONDS of timestamp 723 737
22 13.8 72.3 0.4X
+MICROSECONDS of timestamp 648 653
5 15.4 64.8 0.4X
+EPOCH of timestamp 780 800
17 12.8 78.0 0.4X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for timestamp: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to timestamp 315 317
2 31.8 31.5 1.0X
-MILLENNIUM of timestamp 1155 1162
8 8.7 115.5 0.3X
-CENTURY of timestamp 1146 1152
5 8.7 114.6 0.3X
-DECADE of timestamp 1031 1043
11 9.7 103.1 0.3X
-YEAR of timestamp 1033 1041
10 9.7 103.3 0.3X
-ISOYEAR of timestamp 1274 1278
5 7.8 127.4 0.2X
-QUARTER of timestamp 1326 1346
30 7.5 132.6 0.2X
-MONTH of timestamp 1027 1031
7 9.7 102.7 0.3X
-WEEK of timestamp 1529 1535
6 6.5 152.9 0.2X
-DAY of timestamp 1024 1031
9 9.8 102.4 0.3X
-DAYOFWEEK of timestamp 1197 1201
5 8.4 119.7 0.3X
-DOW of timestamp 1201 1218
15 8.3 120.1 0.3X
-ISODOW of timestamp 1143 1149
6 8.8 114.3 0.3X
-DOY of timestamp 1074 1085
9 9.3 107.4 0.3X
-HOUR of timestamp 354 354
0 28.3 35.4 0.9X
-MINUTE of timestamp 358 361
5 27.9 35.8 0.9X
-SECOND of timestamp 445 456
17 22.5 44.5 0.7X
-MILLISECONDS of timestamp 504 514
12 19.8 50.4 0.6X
-MICROSECONDS of timestamp 361 370
8 27.7 36.1 0.9X
-EPOCH of timestamp 1111 1117
9 9.0 111.1 0.3X
+cast to timestamp 238 248
12 42.0 23.8 1.0X
+MILLENNIUM of timestamp 862 875
12 11.6 86.2 0.3X
+CENTURY of timestamp 833 847
22 12.0 83.3 0.3X
+DECADE of timestamp 759 765
7 13.2 75.9 0.3X
+YEAR of timestamp 744 755
15 13.4 74.4 0.3X
+ISOYEAR of timestamp 937 1019
73 10.7 93.7 0.3X
+QUARTER of timestamp 1011 1091
69 9.9 101.1 0.2X
+MONTH of timestamp 846 888
40 11.8 84.6 0.3X
+WEEK of timestamp 1210 1239
41 8.3 121.0 0.2X
+DAY of timestamp 932 979
41 10.7 93.2 0.3X
+DAYOFWEEK of timestamp 1174 1200
42 8.5 117.4 0.2X
+DOW of timestamp 1131 1172
37 8.8 113.1 0.2X
+ISODOW of timestamp 896 903
7 11.2 89.6 0.3X
+DOY of timestamp 805 818
12 12.4 80.5 0.3X
+HOUR of timestamp 596 597
2 16.8 59.6 0.4X
+MINUTE of timestamp 582 597
17 17.2 58.2 0.4X
+SECOND of timestamp 697 709
16 14.4 69.7 0.3X
+MILLISECONDS of timestamp 700 710
10 14.3 70.0 0.3X
+MICROSECONDS of timestamp 612 631
21 16.3 61.2 0.4X
+EPOCH of timestamp 755 760
7 13.2 75.5 0.3X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke extract for date: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 849 851
3 11.8 84.9 1.0X
-MILLENNIUM of date 1129 1139
11 8.9 112.9 0.8X
-CENTURY of date 1136 1143
7 8.8 113.6 0.7X
-DECADE of date 1039 1043
5 9.6 103.9 0.8X
-YEAR of date 1030 1037
10 9.7 103.0 0.8X
-ISOYEAR of date 1269 1278
9 7.9 126.9 0.7X
-QUARTER of date 1323 1330
6 7.6 132.3 0.6X
-MONTH of date 1021 1023
2 9.8 102.1 0.8X
-WEEK of date 1541 1549
8 6.5 154.1 0.6X
-DAY of date 1021 1033
13 9.8 102.1 0.8X
-DAYOFWEEK of date 1196 1209
11 8.4 119.6 0.7X
-DOW of date 1214 1229
13 8.2 121.4 0.7X
-ISODOW of date 1148 1153
7 8.7 114.8 0.7X
-DOY of date 1073 1079
5 9.3 107.3 0.8X
-HOUR of date 1311 1314
4 7.6 131.1 0.6X
-MINUTE of date 1311 1311
1 7.6 131.1 0.6X
-SECOND of date 1420 1434
13 7.0 142.0 0.6X
-MILLISECONDS of date 1426 1442
14 7.0 142.6 0.6X
-MICROSECONDS of date 1312 1318
6 7.6 131.2 0.6X
-EPOCH of date 2034 2050
16 4.9 203.4 0.4X
+cast to date 633 641
7 15.8 63.3 1.0X
+MILLENNIUM of date 824 845
26 12.1 82.4 0.8X
+CENTURY of date 864 878
13 11.6 86.4 0.7X
+DECADE of date 746 763
17 13.4 74.6 0.8X
+YEAR of date 752 785
39 13.3 75.2 0.8X
+ISOYEAR of date 900 905
5 11.1 90.0 0.7X
+QUARTER of date 906 930
23 11.0 90.6 0.7X
+MONTH of date 747 752
6 13.4 74.7 0.8X
+WEEK of date 1089 1100
17 9.2 108.9 0.6X
+DAY of date 795 803
13 12.6 79.5 0.8X
+DAYOFWEEK of date 937 944
10 10.7 93.7 0.7X
+DOW of date 927 1003
66 10.8 92.7 0.7X
+ISODOW of date 977 980
7 10.2 97.7 0.6X
+DOY of date 827 877
45 12.1 82.7 0.8X
+HOUR of date 1525 1547
34 6.6 152.5 0.4X
+MINUTE of date 1473 1499
23 6.8 147.3 0.4X
+SECOND of date 1600 1615
15 6.2 160.0 0.4X
+MILLISECONDS of date 1666 1765
156 6.0 166.6 0.4X
+MICROSECONDS of date 1554 1627
127 6.4 155.4 0.4X
+EPOCH of date 1615 1646
27 6.2 161.5 0.4X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for date: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to date 852 879
42 11.7 85.2 1.0X
-MILLENNIUM of date 1131 1136
7 8.8 113.1 0.8X
-CENTURY of date 1138 1145
6 8.8 113.8 0.7X
-DECADE of date 1030 1043
13 9.7 103.0 0.8X
-YEAR of date 1022 1028
8 9.8 102.2 0.8X
-ISOYEAR of date 1260 1265
6 7.9 126.0 0.7X
-QUARTER of date 1326 1330
7 7.5 132.6 0.6X
-MONTH of date 1014 1034
26 9.9 101.4 0.8X
-WEEK of date 1523 1526
5 6.6 152.3 0.6X
-DAY of date 1022 1023
2 9.8 102.2 0.8X
-DAYOFWEEK of date 1197 1203
9 8.4 119.7 0.7X
-DOW of date 1188 1198
16 8.4 118.8 0.7X
-ISODOW of date 1143 1153
9 8.8 114.3 0.7X
-DOY of date 1052 1058
7 9.5 105.2 0.8X
-HOUR of date 1309 1311
4 7.6 130.9 0.7X
-MINUTE of date 1302 1305
6 7.7 130.2 0.7X
-SECOND of date 1414 1432
16 7.1 141.4 0.6X
-MILLISECONDS of date 1441 1450
11 6.9 144.1 0.6X
-MICROSECONDS of date 1292 1301
8 7.7 129.2 0.7X
-EPOCH of date 2030 2036
8 4.9 203.0 0.4X
+cast to date 676 690
20 14.8 67.6 1.0X
+MILLENNIUM of date 868 882
16 11.5 86.8 0.8X
+CENTURY of date 875 880
8 11.4 87.5 0.8X
+DECADE of date 762 776
16 13.1 76.2 0.9X
+YEAR of date 788 800
11 12.7 78.8 0.9X
+ISOYEAR of date 903 917
12 11.1 90.3 0.7X
+QUARTER of date 983 1018
40 10.2 98.3 0.7X
+MONTH of date 836 857
19 12.0 83.6 0.8X
+WEEK of date 1137 1168
28 8.8 113.7 0.6X
+DAY of date 768 817
82 13.0 76.8 0.9X
+DAYOFWEEK of date 890 926
36 11.2 89.0 0.8X
+DOW of date 1007 1033
39 9.9 100.7 0.7X
+ISODOW of date 962 969
7 10.4 96.2 0.7X
+DOY of date 797 882
80 12.6 79.7 0.8X
+HOUR of date 1449 1482
29 6.9 144.9 0.5X
+MINUTE of date 1536 1610
69 6.5 153.6 0.4X
+SECOND of date 1675 1823
128 6.0 167.5 0.4X
+MILLISECONDS of date 1605 1622
18 6.2 160.5 0.4X
+MICROSECONDS of date 1481 1504
32 6.8 148.1 0.5X
+EPOCH of date 1656 1843
296 6.0 165.6 0.4X
-OpenJDK 64-Bit Server VM 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09 on Linux
4.15.0-1044-aws
-Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
+Java HotSpot(TM) 64-Bit Server VM 1.8.0_231-b11 on Mac OS X 10.15.3
+Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Invoke date_part for interval: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-cast to interval 1249 1254
6 8.0 124.9 1.0X
-MILLENNIUM of interval 1310 1316
9 7.6 131.0 1.0X
-CENTURY of interval 1304 1315
10 7.7 130.4 1.0X
-DECADE of interval 1306 1313
7 7.7 130.6 1.0X
-YEAR of interval 1304 1313
11 7.7 130.4 1.0X
-QUARTER of interval 1310 1317
7 7.6 131.0 1.0X
-MONTH of interval 1311 1319
12 7.6 131.1 1.0X
-DAY of interval 1295 1304
13 7.7 129.5 1.0X
-HOUR of interval 1301 1306
8 7.7 130.1 1.0X
-MINUTE of interval 1316 1319
3 7.6 131.6 0.9X
-SECOND of interval 1437 1440
3 7.0 143.7 0.9X
-MILLISECONDS of interval 1435 1449
16 7.0 143.5 0.9X
-MICROSECONDS of interval 1304 1314
9 7.7 130.4 1.0X
-EPOCH of interval 1440 1453
19 6.9 144.0 0.9X
+cast to interval 919 959
45 10.9 91.9 1.0X
+MILLENNIUM of interval 959 1001
36 10.4 95.9 1.0X
+CENTURY of interval 974 998
21 10.3 97.4 0.9X
+DECADE of interval 975 989
13 10.3 97.5 0.9X
+YEAR of interval 987 1012
24 10.1 98.7 0.9X
+QUARTER of interval 1014 1031
25 9.9 101.4 0.9X
+MONTH of interval 978 1018
45 10.2 97.8 0.9X
+DAY of interval 1012 1095
92 9.9 101.2 0.9X
+HOUR of interval 1129 1143
14 8.9 112.9 0.8X
+MINUTE of interval 1038 1060
25 9.6 103.8 0.9X
+SECOND of interval 1062 1289
356 9.4 106.2 0.9X
+MILLISECONDS of interval 1142 1226
83 8.8 114.2 0.8X
+MICROSECONDS of interval 1038 1129
79 9.6 103.8 0.9X
+EPOCH of interval 1090 1127
34 9.2 109.0 0.8X
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
index de23132..f1315d4 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/ExtractBenchmark.scala
@@ -42,7 +42,9 @@ object ExtractBenchmark extends SqlBasedBenchmark {
spark
.range(sinceSecond, sinceSecond + cardinality, 1, 1)
.selectExpr(exprs: _*)
- .noop()
+ .queryExecution
+ .toRdd
+ .foreach(_ => ())
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
