Github user mallman commented on a diff in the pull request:
https://github.com/apache/spark/pull/15539#discussion_r84508947
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/TableFileCatalog.scala
---
@@ -42,24 +43,21 @@ class TableFileCatalog(
protected val hadoopConf = sparkSession.sessionState.newHadoopConf
+ private val fileStatusCache = FileStatusCache.getOrInitializeShared(new
Object(), sparkSession)
--- End diff --
Odd, I figured we'd call the `refresh` method to refresh the table.
The reason I suggested using the table's name is so that all references to
the same table can use the same cacheâavoid redundant cache entries and
provide the ability to inherit cached entries for new instances of the same
logical table. Aside from implementation issues, does that make sense?
Anyway, this is something we can explore in a follow up PR if desired.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
