cloud-fan commented on code in PR #52599:
URL: https://github.com/apache/spark/pull/52599#discussion_r2433564173
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala:
##########
@@ -82,6 +85,10 @@ class CacheManager extends Logging with
AdaptiveSparkPlanHelper {
cachedData.isEmpty
}
Review Comment:
The changes in this file is quite confusing. Before this PR,
`DataSourceV2Relation` does not contain the time travel spec, but the `Table`
instance should contain it. This means, if a table scan is cached with version
1, and then we uncache the same table scan but with version 2, we won't uncache
the version 1 scan.
Now we put time travel spec in `DataSourceV2Relation`, which makes this
behavior more reliable in case the `Table` instance does not contain the
version.
I don't quite understand what we try to do here. If we want to bring the old
behavior, we can simplily clear out the time travel spec in
`DataSourceV2Relation#canonicalized`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
