Shixiong Zhu created SPARK-4812:
-----------------------------------
Summary: SparkPlan.codegenEnabled may be initialized to a wrong
value
Key: SPARK-4812
URL: https://issues.apache.org/jira/browse/SPARK-4812
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Shixiong Zhu
The problem is `codegenEnabled` is `val`, but it uses a `val` `sqlContext`,
which can be override by subclasses. Here is a simple example to show this
issue.
{code}
scala> :paste
// Entering paste mode (ctrl-D to finish)
abstract class Foo {
protected val sqlContext = "Foo"
val codegenEnabled: Boolean = {
println(sqlContext) // it will call subclass's `sqlContext` which has not
yet been initialized.
if (sqlContext != null) {
true
} else {
false
}
}
}
class Bar extends Foo {
override val sqlContext = "Bar"
}
println(new Bar().codegenEnabled)
// Exiting paste mode, now interpreting.
null
false
defined class Foo
defined class Bar
scala>
{code}
To fix it, we should mark `codegenEnabled` as `lazy`.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
