LuciferYang commented on code in PR #37843:
URL: https://github.com/apache/spark/pull/37843#discussion_r967265692
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/util/V2ExpressionSQLBuilder.java:
##########
@@ -393,4 +384,20 @@ private String joinListToString(
}
return joiner.toString();
}
+
+ private String[] expressionsToStringArray(Expression[] expressions) {
+ String[] result = new String[expressions.length];
+ for (int i = 0; i < expressions.length; i++) {
+ result[i] = build(expressions[i]);
+ }
+ return result;
+ }
+
+ private List<String> expressionsToStringList(Expression[] expressions) {
+ List<String> list = new ArrayList<>(expressions.length);
Review Comment:
Ignore the complexity of build(expressions[i]) compare
```
Arrays.stream(input).mapToObj(String::valueOf).collect(Collectors.toList())
```
vs
```
List<String> list = new ArrayList<>();
for (long l : input) {
String s = String.valueOf(l);
list.add(s);
}
return list;
```
with input size 1, 5, 10, 20, 50, 100, 500, 1000:
Java 8
```
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 1: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 10 12
1 10.4 95.9 1.0X
Use Loop api 4 5
1 23.2 43.1 2.2X
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 5: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 22 25
1 4.5 222.6 1.0X
Use Loop api 13 15
1 7.7 130.0 1.7X
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 10: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 38 41
2 2.7 375.8 1.0X
Use Loop api 24 28
2 4.2 240.4 1.6X
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 20: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 77 83
4 1.3 767.2 1.0X
Use Loop api 55 62
3 1.8 554.4 1.4X
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 50: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 192 201
7 0.5 1924.7 1.0X
Use Loop api 147 155
6 0.7 1474.3 1.3X
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 100: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 367 392
18 0.3 3671.7 1.0X
Use Loop api 297 309
16 0.3 2965.8 1.2X
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 500: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 2018 2030
18 0.0 20175.5 1.0X
Use Loop api 1736 1742
8 0.1 17364.4 1.2X
OpenJDK 64-Bit Server VM 1.8.0_345-b01 on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Test map to String List with input size 1000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 4156 4157
1 0.0 41562.0 1.0X
Use Loop api 3562 3578
24 0.0 35615.5 1.2X
```
Java 11
```
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 1: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 10 10
0 10.0 100.2 1.0X
Use Loop api 3 4
0 28.7 34.9 2.9X
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 5: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 19 19
0 5.3 187.5 1.0X
Use Loop api 8 8
0 12.9 77.7 2.4X
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 10: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 31 32
1 3.2 310.9 1.0X
Use Loop api 13 14
2 7.6 130.8 2.4X
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 20: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 58 59
0 1.7 583.8 1.0X
Use Loop api 28 29
4 3.6 279.4 2.1X
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 50: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 146 148
6 0.7 1460.4 1.0X
Use Loop api 72 73
0 1.4 722.5 2.0X
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 100: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 275 276
0 0.4 2748.2 1.0X
Use Loop api 137 139
1 0.7 1372.4 2.0X
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 500: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 1356 1356
0 0.1 13556.1 1.0X
Use Loop api 793 794
1 0.1 7927.0 1.7X
OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 1000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 2741 2742
2 0.0 27410.6 1.0X
Use Loop api 1653 1665
16 0.1 16534.3 1.7X
```
Java 17
```
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 1: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 14 15
0 7.0 143.0 1.0X
Use Loop api 3 3
0 37.5 26.6 5.4X
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 5: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 23 23
0 4.4 228.1 1.0X
Use Loop api 7 7
0 14.9 67.1 3.4X
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 10: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 34 34
1 2.9 339.5 1.0X
Use Loop api 12 12
0 8.5 117.3 2.9X
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 20: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 59 59
0 1.7 587.6 1.0X
Use Loop api 26 27
1 3.8 261.3 2.2X
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 50: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 139 139
0 0.7 1389.1 1.0X
Use Loop api 70 71
0 1.4 702.0 2.0X
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 100: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 255 255
1 0.4 2545.7 1.0X
Use Loop api 132 133
1 0.8 1322.0 1.9X
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 500: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 1282 1282
0 0.1 12822.7 1.0X
Use Loop api 750 754
4 0.1 7497.5 1.7X
OpenJDK 64-Bit Server VM 17.0.4+8-LTS on Linux 5.15.0-1019-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
Test map to String List with input size 1000: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
Use Arrays.steam api 2592 2596
5 0.0 25924.0 1.0X
Use Loop api 1553 1554
1 0.1 15529.3 1.7X
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
