[
https://issues.apache.org/jira/browse/HDFS-10823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HDFS-10823:
-------------------------------
Attachment: HDFS-10823.001.patch
WIP Patch attached. This depends on HDFS-10837 which isn't committed yet, so
not marking this Patch Available.
As described above, we have an unfortunate mismatch between the FS API and
WebHDFS for implementing a generic iterator. This is why we have both a
DirectoryListing and DirectoryEntries class that are slightly different. We
could discard DL in favor of the more generic DE, though then the WebHDFS
protocol would diverge a bit from the HDFS protobuf protocol. As it stands, if
you are a downstream who wants to be generic to WebHDFS and HTTPFS, you need to
treat {{remainingEntries}} like a boolean. This is kind of true anyway, since
there could always be a concurrent writer adding more files to a listing which
throws off the count.
I also didn't implement the subclass tricks that would let us make the
FileSystem additions protected. I annotated them Private with strong warnings
in javadoc, which I hope is sufficient.
> Implement HttpFSFileSystem#listStatusIterator
> ---------------------------------------------
>
> Key: HDFS-10823
> URL: https://issues.apache.org/jira/browse/HDFS-10823
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: httpfs
> Affects Versions: 2.6.4
> Reporter: Andrew Wang
> Assignee: Andrew Wang
> Attachments: HDFS-10823.001.patch
>
>
> Let's expose the same functionality added in HDFS-10784 for WebHDFS in HttpFS
> too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
