Manu Zhang created SPARK-31047:
----------------------------------
Summary: Improve file listing for ViewFileSystem
Key: SPARK-31047
URL: https://issues.apache.org/jira/browse/SPARK-31047
Project: Spark
Issue Type: Improvement
Components: Input/Output
Affects Versions: 3.1.0
Reporter: Manu Zhang
https://issues.apache.org/jira/browse/SPARK-27801 has improved file listing for
DistributedFileSystem, where {{InMemoryFileIndex.listLeafFiles}} makes use of
DistributedFileSystem's one single {{listLocatedStatus}} to namenode. This
ticket intends to improve the case where ViewFileSystem is used to manage
multiple DistributedFileSystems. It has also overridden the
{{listLocatedStatus}} method by delegating to the filesystem it resolves to,
e.g. DistributedFileSystem.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
