[
https://issues.apache.org/jira/browse/HDFS-10176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685217#comment-15685217
]
Lucas Lustosa Madureira commented on HDFS-10176:
------------------------------------------------
Greetings
I gave it a try on resolving his issue.
As I understand it, the WebHDFS is supposed to expose the methods of
[org.apache.hadoop.fs.FileSystem|https://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html].
As it stands, this class does not have a version of listStatus that supports
ordering. Ideally, to solving this issue involves adding such method to this
class. Adding this method was easy enough (take a look at [this
patch|http://pastebin.com/raw/R4G5FjQq]).
The next step should be to modify the WebHDFS url to understand a new
parameter, for instance:
{code:borderStyle=solid}
curl -i
"http://<HOST>:<PORT>/webhdfs/v1/<PATH>?op=LISTSTATUS[&orderBy=<ORDERBY>]"
{code}
Then it would be necessary to check if the parameter is set and call the
appropriate version of listStatus.
Unfortunately I could not figure this part. I could verify that the request to
the URL above is (at least partially) fulfilled by the class
[org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java].
Looking at the class
[org.apache.hadoop.hdfs.web.WebHdfsFileSystem|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java],
I got the impression that this class somehow causes the code on
NamenodeWebHdfsMethods to run. If that is what happens, I think I need to know
which part of the code calls WebHdfsFileSystem.listStatus, and apply there the
changes proposed on the previous paragraph.
Unfortunatelly I could not figure this out, specially as there is no
documentation of the inner workings of the hadoop project.
To add to my confusion, there are also the classes at
[org.apache.hadoop.http|https://github.com/apache/hadoop/tree/trunk/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http],
which I am not sure are related to WebHdfs or not (they also deal with http
requests and hdfs).
Should I make no progress, at least this comment is a starting point for anyone
who tries to address this issue.
PS. How do you guys package a new hadoop source? It takes me almost half an
hour to package it with maven, even though I only change a few lines of a file.
> WebHdfs LISTSTATUS does not offer any sorting
> ---------------------------------------------
>
> Key: HDFS-10176
> URL: https://issues.apache.org/jira/browse/HDFS-10176
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 2.7.0
> Reporter: Romain Rigaux
>
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#List_a_Directory
> Times, names, sizes would allow a Web client to offer a richer experience to
> its user:
> {code}
> {
> "accessTime" : 1320171722771,
> "blockSize" : 33554432,
> "group" : "supergroup",
> "length" : 24930,
> "modificationTime": 1320171722771,
> "owner" : "webuser",
> "pathSuffix" : "a.patch",
> "permission" : "644",
> "replication" : 1,
> "type" : "FILE"
> },
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
