Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/18972#discussion_r150284355
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
---
@@ -2081,10 +2081,8 @@ class DataFrameSuite extends QueryTest with
SharedSQLContext {
}
withSQLConf(SQLConf.CODEGEN_FALLBACK.key -> "false") {
- val e = intercept[SparkException] {
- df.filter(filter).count()
- }.getMessage
- assert(e.contains("grows beyond 64 KB"))
+ // SPARK-21720 avoids an exception due to JVM code size limit
--- End diff --
In general, I agree with you that we should create a config.
Although I create a PR to add a config for a constant in `CodeGenerator`,
it revealed that we need [additional (large)
work](https://github.com/apache/spark/pull/19449#pullrequestreview-67789784) to
fix active session management.
Can we introduce a config after fixing active session management?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
