LuciferYang opened a new pull request, #36942:
URL: https://github.com/apache/spark/pull/36942
### What changes were proposed in this pull request?
`ExpressionSet ++` method in the master branch a little slower than the
branch-3.3 with Scala-2.13, so this pr override `concat` method for
`ExpressionSet` in Scala 2.13.
### Why are the changes needed?
Improve the performance
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GA
- Manual test 1:
microbench as follows and run with Scala 2.13:
```scala
val valuesPerIteration = 100000
val benchmark = new Benchmark("Test ExpressionSet ++ ",
valuesPerIteration, output = output)
val aUpper = AttributeReference("A", IntegerType)(exprId = ExprId(1))
val initialSet = ExpressionSet(aUpper + 1 :: Rand(0) :: Nil)
val setToAddWithSameDeterministicExpression = ExpressionSet(aUpper + 1
:: Rand(0) :: Nil)
benchmark.addCase("Test ++") { _: Int =>
for (_ <- 0L until valuesPerIteration) {
initialSet ++ setToAddWithSameDeterministicExpression
}
}
benchmark.run()
```
**branch-3.3 result:**
```
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 4.14.0_1-0-0-45
Intel(R) Xeon(R) Gold 6XXXC CPU @ 2.60GHz
Test ExpressionSet ++ : Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Test ++ 14 16
4 7.2 139.1 1.0X
```
**master result before this pr:**
```
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 4.14.0_1-0-0-45
Intel(R) Xeon(R) Gold 6271C CPU @ 2.60GHz
Test ExpressionSet ++ : Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Test ++ 16 19
5 6.1 163.9 1.0X
```
**master result after this pr:**
```
Running benchmark: Test ExpressionSet ++
Running case: Test ++
Stopped after 154 iterations, 2010 ms
OpenJDK 64-Bit Server VM 1.8.0_312-b07 on Linux 4.14.0_1-0-0-45
Intel(R) Xeon(R) Gold 6271C CPU @ 2.60GHz
Test ExpressionSet ++ : Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
Test ++ 12 13
3 8.6 115.7 1.0X
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
