Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19083#discussion_r142576983
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/AggregateBenchmark.scala
---
@@ -333,33 +334,28 @@ class AggregateBenchmark extends BenchmarkBase {
.sum()
.collect()
- benchmark.addCase(s"codegen = F") { iter =>
- sparkSession.conf.set("spark.sql.codegen.wholeStage", "false")
+ benchmark.addCase(s"hugeMethodLimit = 8000") { iter =>
+ sparkSession.conf.set(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key, "true")
+ sparkSession.conf.set(SQLConf.WHOLESTAGE_HUGE_METHOD_LIMIT.key,
"8000")
f()
}
- benchmark.addCase(s"codegen = T maxLinesPerFunction = 10000") { iter =>
- sparkSession.conf.set("spark.sql.codegen.wholeStage", "true")
- sparkSession.conf.set("spark.sql.codegen.maxLinesPerFunction",
"10000")
- f()
- }
-
- benchmark.addCase(s"codegen = T maxLinesPerFunction = 1500") { iter =>
- sparkSession.conf.set("spark.sql.codegen.wholeStage", "true")
- sparkSession.conf.set("spark.sql.codegen.maxLinesPerFunction",
"1500")
+ benchmark.addCase(s"hugeMethodLimit = 16000") { iter =>
+ sparkSession.conf.set(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key, "true")
+ sparkSession.conf.set(SQLConf.WHOLESTAGE_HUGE_METHOD_LIMIT.key,
"16000")
f()
}
benchmark.run()
/*
- Java HotSpot(TM) 64-Bit Server VM 1.8.0_111-b14 on Windows 7 6.1
- Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
- max function length of wholestagecodegen: Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
-
----------------------------------------------------------------------------------------------
- codegen = F 462 / 533 1.4
704.4 1.0X
- codegen = T maxLinesPerFunction = 10000 3444 / 3447 0.2
5255.3 0.1X
- codegen = T maxLinesPerFunction = 1500 447 / 478 1.5
682.1 1.0X
+ Java HotSpot(TM) 64-Bit Server VM 1.8.0_31-b13 on Mac OS X 10.10.2
+ Intel(R) Core(TM) i7-4578U CPU @ 3.00GHz
+
+ max function bytecode size: Best/Avg Time(ms)
Rate(M/s) Per Row(ns) Relative
+
------------------------------------------------------------------------------------------------
+ hugeMethodLimit = 8000 1043 / 1159 0.6
1591.5 1.0X
--- End diff --
The original `codegen = F` case is removed? I think it is reasonable to
compare with it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
