allisonwang-db commented on a change in pull request #35404:
URL: https://github.com/apache/spark/pull/35404#discussion_r805015914
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -4249,7 +4250,30 @@ object ApplyCharTypePadding extends Rule[LogicalPlan] {
* rule right after the main resolution batch.
*/
object RemoveTempResolvedColumn extends Rule[LogicalPlan] {
- override def apply(plan: LogicalPlan): LogicalPlan = plan.resolveExpressions
{
- case t: TempResolvedColumn => UnresolvedAttribute(t.nameParts)
+ override def apply(plan: LogicalPlan): LogicalPlan = {
+ plan.foreachUp {
+ // HAVING clause will be resolved as a Filter. When having func(column
with wrong data type),
+ // the column could be wrapped by a TempResolvedColumn, e.g.
mean(tempresolvedcolumn(t.c)).
+ // Because TempResolvedColumn can still preserve column data type, here
is a chance to check
+ // if the data type matches with the required data type of the function.
We can throw an error
+ // when data types mismatches.
+ case operator: Filter =>
Review comment:
How about handling `TempResolvedColumn` in `CheckAnalysis`? For example:
https://github.com/apache/spark/blob/25a4c5fa84d64e37cf5c27c7b2f0f29867330bf2/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L161-L166
Are there new issues if we keep TempResolvedColumn during the analysis?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
