[
https://issues.apache.org/jira/browse/HADOOP-16269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16835736#comment-16835736
]
Steve Loughran commented on HADOOP-16269:
-----------------------------------------
+1, committed to trunk!
Now, one warning: adding new FS API calls is great for internal stuff and for
writing custom code to work with your store, but does have a few risks
* people get sad when you take things away
* it makes it hard/impossible to put another layered FS on top of this (to
measure performance, cache results, etc)
* we diverge across stores
* apps don't use it, or if they do, they break when new versions ship.
Ideally there should be a stable API For this in the filesystem. We do actually
have listStatus(Path, recursive) which returns an iterable so can be used to
iterate through a directory in pages, or down an entire directory tree -which
delivers fundamental performance gains for any store with a flat list
operation.
So now this is in, how about you use it or some other mechanism to implement
{{FileSystem.listFiles()}} efficiently? The more stores which do, the more we
can encourage people to switch to it in their code, for maximum speedup.
> ABFS: add listFileStatus with StartFrom
> ---------------------------------------
>
> Key: HADOOP-16269
> URL: https://issues.apache.org/jira/browse/HADOOP-16269
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.2.0
> Reporter: Da Zhou
> Assignee: Da Zhou
> Priority: Major
> Attachments: HADOOP-16269-001.patch, HADOOP-16269-002.patch,
> HADOOP-16269-003.patch
>
>
> Adding a ListFileStatus in a path from a entry name in lexical order.
> This is added to AzureBlobFileSystemStore and won't be exposed to FS level
> api.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
