panbingkun commented on PR #48073:
URL: https://github.com/apache/spark/pull/48073#issuecomment-2343116127
- Benchmark code as follows:
```scala
object PrintfBenchmark extends SqlBasedBenchmark {
private val N = 10_000_00
private val M = 100
private val df = spark.range(N).to(new StructType().add("id", "int")).
withColumn("format", lit("aa%d%s")).
withColumn("value1", col("id") % M).
withColumn("value2", lit("xxx"))
private def doBenchmark(): Unit = {
df.selectExpr("printf(format, value1, value2)").noop()
}
override def runBenchmarkSuite(mainArgs: Array[String]): Unit = {
runBenchmark("printf") {
val benchmark = new Benchmark("printf", N, output = output)
benchmark.addCase("optimize", M) { _ =>
doBenchmark()
}
benchmark.run()
}
}
}
```
- Before (run 3 times)
```
Running benchmark: printf
Running case: has foldable optimize
Stopped after 100 iterations, 25558 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 14.6.1
Apple M2
printf: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 246 256
21 4.1 245.8 1.0X
Running benchmark: printf
Running case: has foldable optimize
Stopped after 100 iterations, 24545 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 14.6.1
Apple M2
printf: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 241 245
5 4.2 240.6 1.0X
Running benchmark: printf
Running case: has foldable optimize
Stopped after 100 iterations, 26481 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 14.6.1
Apple M2
printf: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 257 265
16 3.9 256.6 1.0X
```
- After (run 3 times)
```
Running benchmark: printf
Running case: has foldable optimize
Stopped after 100 iterations, 22811 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 14.6.1
Apple M2
printf: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 217 228
39 4.6 216.7 1.0X
Running benchmark: printf
Running case: has foldable optimize
Stopped after 100 iterations, 21136 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 14.6.1
Apple M2
printf: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 206 211
6 4.9 205.7 1.0X
Running benchmark: printf
Running case: has foldable optimize
Stopped after 100 iterations, 21805 ms
OpenJDK 64-Bit Server VM 17.0.10+7-LTS on Mac OS X 14.6.1
Apple M2
printf: Best Time(ms) Avg Time(ms)
Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
optimize 211 218
16 4.7 210.8 1.0X
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
