Github user bersprockets commented on a diff in the pull request:
https://github.com/apache/spark/pull/21144#discussion_r183853692
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/predicates.scala
---
@@ -36,6 +36,14 @@ object InterpretedPredicate {
case class InterpretedPredicate(expression: Expression) extends
BasePredicate {
override def eval(r: InternalRow): Boolean =
expression.eval(r).asInstanceOf[Boolean]
+
+ override def initialize(partitionIndex: Int): Unit = {
+ super.initialize(partitionIndex)
+ expression.foreach {
+ case n: Nondeterministic => n.initialize(partitionIndex)
+ case _ =>
+ }
+ }
}
--- End diff --
An alternative to this change: we could simply expect a caller to
InterpretedPredicate.create, like SparkPlan.genInterpretedPredicate, to
pre-initialized the expression before passing it to create.
This alternative is a little harder to unit test: we would also need to add
a flag to shut off predicate codegen so that we can force
SparkPlan.newPredicate to use InterpretedPredicate.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
