Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/10827#discussion_r50152663
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -635,6 +635,21 @@ object SimplifyConditionals extends Rule[LogicalPlan]
with PredicateHelper {
case q: LogicalPlan => q transformExpressionsUp {
case If(TrueLiteral, trueValue, _) => trueValue
case If(FalseLiteral, _, falseValue) => falseValue
+
+ case e @ CaseWhen(branches, elseValue) if branches.exists(_._1 ==
FalseLiteral) =>
+ // If there are branches that are always false, remove them.
+ // If there are no more branches left, just use the else value.
+ // Note that these two are handled together here in a single case
statement because
+ // otherwise we cannot determine the data type for the elseValue
if it is None (i.e. null).
+ val newBranches = branches.filter(_._1 != FalseLiteral)
+ if (newBranches.isEmpty) {
+ elseValue.getOrElse(Literal.create(null, e.dataType))
+ } else {
+ e.copy(branches = newBranches)
+ }
+
+ case e @ CaseWhen(branches, _) if branches.headOption.map(_._1) ==
Some(TrueLiteral) =>
--- End diff --
No it's not due to the previous rule. If the elseValue is None, and there
is no branches, there is actually no way to figure out the data type for the
null elseValue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
