panbingkun commented on PR #48493:
URL: https://github.com/apache/spark/pull/48493#issuecomment-2416042475
- Benchmark code as follows:
```scala
object RandstrBenchmark extends SqlBasedBenchmark {
private val N = 1000000 // 1_000_00
private val M = 100
private val df = spark.range(N).to(new StructType().add("id", "int"))
private def doBenchmark(): Unit = {
df.selectExpr("randStr(10, 20)").noop()
}
override def runBenchmarkSuite(mainArgs: Array[String]): Unit = {
runBenchmark("randStr") {
val benchmark = new Benchmark("randStr", N, output = output)
benchmark.addCase("optimize", M) { _ =>
doBenchmark()
}
benchmark.run()
}
}
}
```
- Before (run 3 times)
```
Running benchmark: randStr
Running case: optimize
Stopped after 100 iterations, 12265 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 15.0.1
Apple M2
randStr: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 107 123
21 9.4 106.9 1.0X
Running benchmark: randStr
Running case: optimize
Stopped after 100 iterations, 11298 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 15.0.1
Apple M2
randStr: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 107 113
7 9.3 107.0 1.0X
Running benchmark: randStr
Running case: optimize
Stopped after 100 iterations, 11181 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 15.0.1
Apple M2
randStr: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 107 112
5 9.4 106.7 1.0X
```
- After (run 3 times)
```
Running benchmark: randStr
Running case: optimize
Stopped after 100 iterations, 10362 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 15.0.1
Apple M2
randStr: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 98 104
6 10.2 98.0 1.0X
Running benchmark: randStr
Running case: optimize
Stopped after 100 iterations, 10380 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 15.0.1
Apple M2
randStr: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 98 104
7 10.2 97.9 1.0X
Running benchmark: randStr
Running case: optimize
Stopped after 100 iterations, 10230 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 15.0.1
Apple M2
randStr: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 97 102
4 10.3 97.5 1.0X
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
