maropu commented on a change in pull request #28810:
URL: https://github.com/apache/spark/pull/28810#discussion_r439661573
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##########
@@ -1334,13 +1340,13 @@ object PushPredicateThroughJoin extends
Rule[LogicalPlan] with PredicateHelper {
(rightFilterConditions ++ commonFilterCondition).
reduceLeftOption(And).map(Filter(_, newJoin)).getOrElse(newJoin)
- case FullOuter => f // DO Nothing for Full Outer Join
- case NaturalJoin(_) => sys.error("Untransformed NaturalJoin node")
- case UsingJoin(_, _) => sys.error("Untransformed Using join node")
+
+ case jt =>
+ sys.error(s"Unexpected join type: $jt")
Review comment:
`AnalysisException` is mainly used for an analyzing phase?
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/AnalysisException.scala#L24
yea, I know some rules in the optimizer throw `AnalysisException` though...
```
(base)
maropu@~/Repositories/spark/spark-master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer:
(SPARK-31705 $)$grep -nr "throw" .
./joins.scala:187: // `requires attributes from more than one child`,
we throw firstly here for better
./joins.scala:189: throw new AnalysisException("Using PythonUDF in
join condition of join type" +
./joins.scala:206: throw new AnalysisException("Using PythonUDF in
join condition of join type" +
./NormalizeFloatingNumbers.scala:100: throw new
IllegalStateException("grouping/join/window partition keys cannot be map type.")
./NormalizeFloatingNumbers.scala:134: case _ => throw new
IllegalStateException(s"fail to normalize $expr")
./objects.scala:250: throw new IllegalStateException("LambdaVariable
should never has 0 as its ID.")
./objects.scala:255: throw new IllegalStateException(
./objects.scala:264: throw new IllegalStateException(
./Optimizer.scala:1345: throw new
IllegalStateException(s"Unexpected join type: $jt")
./Optimizer.scala:1381: throw new
IllegalStateException(s"Unexpected join type: $jt")
./Optimizer.scala:1436: throw new AnalysisException(
./PushCNFPredicateThroughJoin.scala:64: throw new
IllegalStateException(s"Unexpected join type: $jt")
./ReplaceExceptWithFilter.scala:106: throw new
IllegalStateException("Leaf node is expected")
./ReplaceNullWithFalseInPredicate.scala:105: throw new
IllegalArgumentException(message)
./subquery.scala:79: throw new AnalysisException("Found
conflicting attributes " +
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
