viirya commented on a change in pull request #31172:
URL: https://github.com/apache/spark/pull/31172#discussion_r557077899
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
##########
@@ -60,21 +60,19 @@ class DataSourceV2Strategy(session: SparkSession) extends
Strategy with Predicat
session.sharedState.cacheManager.recacheByPlan(session, r)
}
+ private def recacheTable(r: ResolvedTable)(): Unit = {
+ val v2Relation = DataSourceV2Relation.create(r.table, Some(r.catalog),
Some(r.identifier))
+ session.sharedState.cacheManager.recacheByPlan(session, v2Relation)
+ }
Review comment:
This seems the main point of this change. And as other comments, this
looks like a behavior change or an inconsistent behavior to v1.
> Reduce the number of calls to the Cache Manager when need to recache a
table. Before the changes, invalidateCache() invokes the Cache Manager 3 times:
lookupCachedData(), uncacheQuery() and cacheQuery().
Hmm, does this stand for the reason of this change? If we don't consider the
behavior change part, `recacheByPlan` is actually more complicated than other
three calls `lookupCachedData`, `uncacheQuery` and `cacheQuery`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
