srielau commented on code in PR #55717:
URL: https://github.com/apache/spark/pull/55717#discussion_r3213244047
##########
sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala:
##########
@@ -205,6 +206,8 @@ abstract class BaseSessionStateBuilder(
override val singlePassPostHocResolutionRules: Seq[Rule[LogicalPlan]] =
DetectAmbiguousSelfJoin +:
ApplyCharTypePadding +:
+ ResolveSQLFunctions +:
Review Comment:
Yes, both are intentional.
Single-pass resolves the main tree but does not automatically run the full
post-resolution batch the classic Analyzer applies after certain subtrees. Two
gaps showed up with PATH + temp SQL routines + COUNT(*) under single-pass:
1. **ResolveSQLFunctions** — SQL-invoked routines still need the same
rewrite the fixed-point analyzer applies in extendedResolutionRules. Without it
in the single-pass post-hoc pipeline, analysis can leave SQL-function nodes in
a state the optimizer does not accept.
2. **ResolveDeserializer** — expanding SQL routine bodies can introduce
DeserializeToObject with UnresolvedDeserializer; the fixed-point analyzer
clears that via Analyzer.ResolveDeserializer. Wiring it here keeps single-pass
aligned with that step.
These are parity rules with fixed-point post-analysis, scoped to
singlePassPostHocResolutionRules.
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionStateBuilder.scala:
##########
@@ -100,6 +100,8 @@ class HiveSessionStateBuilder(
override val singlePassPostHocResolutionRules: Seq[Rule[LogicalPlan]] =
DetectAmbiguousSelfJoin +:
ApplyCharTypePadding +:
+ ResolveSQLFunctions +:
Review Comment:
Same intent as BaseSessionStateBuilder: **ResolveSQLFunctions** and
**ResolveDeserializer** are there for single-pass parity with the classic
analyzer when SQL routines / deserializers appear.
HiveSessionStateBuilder overrides analyzer and must keep
singlePassPostHocResolutionRules in lockstep with the classic builder so Hive
sessions do not miss those post-hoc steps when single-pass is enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
