sunchao commented on a change in pull request #31172:
URL: https://github.com/apache/spark/pull/31172#discussion_r557095366
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
##########
@@ -60,21 +60,19 @@ class DataSourceV2Strategy(session: SparkSession) extends
Strategy with Predicat
session.sharedState.cacheManager.recacheByPlan(session, r)
}
+ private def recacheTable(r: ResolvedTable)(): Unit = {
+ val v2Relation = DataSourceV2Relation.create(r.table, Some(r.catalog),
Some(r.identifier))
+ session.sharedState.cacheManager.recacheByPlan(session, v2Relation)
+ }
Review comment:
I also think should be considered as a behavior change. Even though
query result remains the same, it now lazily caches data which changes query
execution time and memory/storage consumption. Because this, we should make a
similar change for v1 as well.
And please update the PR description to make it more clear (e.g.,
user-facing change).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
