amaliujia commented on code in PR #43727:
URL: https://github.com/apache/spark/pull/43727#discussion_r1388716816
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala:
##########
@@ -167,25 +167,21 @@ trait CheckAnalysis extends PredicateHelper with
LookupCatalog with QueryErrorsB
val inlineCTE = InlineCTE(alwaysInline = true)
val cteMap = mutable.HashMap.empty[Long, (CTERelationDef, Int,
mutable.Map[Long, Int])]
inlineCTE.buildCTEMap(plan, cteMap)
- cteMap.values.foreach { case (relation, _, _) =>
+ val visited: mutable.Map[Long, Boolean] =
mutable.Map.empty.withDefaultValue(false)
+ cteMap.foreach { case (cteId, (relation, refCount, _)) =>
// If a CTE relation is never used, it will disappear after inline. Here
we explicitly check
// analysis for it, to make sure the entire query plan is valid.
try {
// If a CTE relation ref count is 0, the other CTE relations that
reference it
// should also be checked by checkAnalysis0. This code will also
guarantee the leaf
// relations that do not reference any others are checked first.
- val visited: mutable.Map[Long, Boolean] =
mutable.Map.empty.withDefaultValue(false)
- cteMap.foreach { case (cteId, _) =>
- val (_, refCount, _) = cteMap(cteId)
- if (refCount == 0) {
- checkUnreferencedCTERelations(cteMap, visited, cteId)
- }
+ if (refCount == 0) {
Review Comment:
I think this misses some corners cases. For example
```
WITH
a as (select * from table_exists),
b as (select * from a),
c as (select * from table_non_exists),
d as (select * from c)
SELECT 1
```
So the code may only check `a` and ` b` but lose the checking over `c` and
`d`.
cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
