Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19083#discussion_r142012782
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -585,10 +586,22 @@ object SQLConf {
.doc("The maximum lines of a single Java function generated by
whole-stage codegen. " +
"When the generated function exceeds this threshold, " +
"the whole-stage codegen is deactivated for this subtree of the
current query plan. " +
- "The default value 4000 is the max length of byte code JIT supported
" +
- "for a single function(8000) divided by 2.")
+ s"The default value
${CodeGenerator.DEFAULT_OPENJDK_JVM_HUGE_METHOD_LIMIT / 2} is " +
+ "the max length of byte code JIT supported for a single function" +
+ s"(${CodeGenerator.DEFAULT_OPENJDK_JVM_HUGE_METHOD_LIMIT}}) divided
by 2.")
.intConf
- .createWithDefault(4000)
+ .createOptional
+
+ val CODEGEN_HUGE_METHOD_LIMIT =
buildConf("spark.sql.codegen.hugeMethodLimit")
+ .internal()
+ .doc("The bytecode size of a single compiled Java function generated
by whole-stage codegen." +
+ "When the compiled function exceeds this threshold, " +
+ "the whole-stage codegen is deactivated for this subtree of the
current query plan. " +
--- End diff --
This threshold is not for whole-stage only, right? It is misleading.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
