Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21289#discussion_r187372024
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala
---
@@ -75,16 +77,31 @@ class InMemoryFileIndex(
}
override protected def leafFiles: mutable.LinkedHashMap[Path,
FileStatus] = {
+ refreshIfCacheOutdated()
--- End diff --
what's the gain here? seems you only delay the refresh, not eliminate it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
