cloud-fan commented on a change in pull request #28852:
URL: https://github.com/apache/spark/pull/28852#discussion_r456233068
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala
##########
@@ -135,7 +136,16 @@ class SessionCatalog(
private val tableRelationCache: Cache[QualifiedTableName, LogicalPlan] = {
Review comment:
ah that's a good point. We should probably investigate how to design the
data source API so that sources don't need to infer schema can skip this cache.
It's hard to use the JDBC data source as we need to run REFRESH TABLE (or wait
for TTL after this PR) once the table is changed outside of spark (which is
common to JDBC source).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
