aokolnychyi commented on code in PR #53202:
URL: https://github.com/apache/spark/pull/53202#discussion_r2560632894
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2Exec.scala:
##########
@@ -171,7 +172,11 @@ case class ReplaceTableAsSelectExec(
//
// RTAS must refresh and pin versions in query to read from original table
versions instead of
// newly created empty table that is meant to serve as target for
append/overwrite
- val refreshedQuery = V2TableRefreshUtil.refresh(session, query,
versionedOnly = true)
+ val refreshedQuery = V2TableRefreshUtil.refresh(
Review Comment:
The trick here is that `ReplaceTableAsSelect` is `AnalysisOnlyCommand`,
meaning its query is only analyzed and then hidden after that until the
execution. This means the query is never refreshed after the analysis. Other
operations would refresh when we construct `AppendData` but it is not allowed
for non-atomic REPLACE as we have to use original table versions.
Does it make sense, @cloud-fan?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
