aokolnychyi commented on issue #1485:
URL: https://github.com/apache/iceberg/issues/1485#issuecomment-696923006
Here is what I see in `CacheManager`:
```
/**
* Tries to re-cache all the cache entries that contain `resourcePath` in
one or more
* `HadoopFsRelation` node(s) as part of its logical plan.
*/
def recacheByPath(spark: SparkSession, resourcePath: String): Unit = {
val (fs, qualifiedPath) = {
val path = new Path(resourcePath)
val fs = path.getFileSystem(spark.sessionState.newHadoopConf())
(fs, fs.makeQualified(path))
}
recacheByCondition(spark, _.plan.find(lookupAndRefresh(_, fs,
qualifiedPath)).isDefined)
}
/**
* Traverses a given `plan` and searches for the occurrences of
`qualifiedPath` in the
* [[org.apache.spark.sql.execution.datasources.FileIndex]] of any
[[HadoopFsRelation]] nodes
* in the plan. If found, we refresh the metadata and return true.
Otherwise, this method returns
* false.
*/
private def lookupAndRefresh(plan: LogicalPlan, fs: FileSystem,
qualifiedPath: Path): Boolean = {
plan match {
case lr: LogicalRelation => lr.relation match {
case hr: HadoopFsRelation =>
refreshFileIndexIfNecessary(hr.location, fs, qualifiedPath)
case _ => false
}
case DataSourceV2Relation(fileTable: FileTable, _, _, _, _) =>
refreshFileIndexIfNecessary(fileTable.fileIndex, fs, qualifiedPath)
case _ => false
}
}
```
Here, `DataSourceV2Relation` for `FileTable` was already added.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
