uros-b commented on PR #59310:
URL: https://github.com/apache/spark/pull/59310#issuecomment-6215213760
Other than scalafmt issues in this PR, the reclassification and the
transformWithRelations handling overall look good.
However, please handle @HyukjinKwon's remaining point on withColumns /
withColumnsRenamed. Their eager analysis runs before the plan ids are
substituted, and since the failure is now an AnalysisException, assertAnalyzed
posts a failed analyze SQL execution for valid queries, e.g. df.withColumn("x",
sub.scalar()) or a withColumnRenamed above filter(col.isin(sub)). The SQL tab
then shows a failed execution saying the plan "is not attached to the request",
although it was. Skipping the eager attempt when the input or the new
expressions still contain UNRESOLVED_PLAN_ID, plus a test that no SQL execution
is posted, would avoid that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
