[
https://issues.apache.org/jira/browse/HDFS-14369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
CR Hota updated HDFS-14369:
---------------------------
Description:
WebHDFS doesn't trim trailing slash causing discrepancy in operations.
Example below
--------------------------
Using HDFS API, two directory are listed.
{code}
$ hdfs dfs -ls hdfs://<router>:8888/tmp/
Found 2 items
drwxrwxrwx - hdfs supergroup 0 2018-11-09 17:50
hdfs://<router>:8888/tmp/tmp1
drwxrwxrwx - hdfs supergroup 0 2018-11-09 17:50
hdfs://<router>:8888/tmp/tmp2
{code}
Using WebHDFS API, only one directory is listed.
{code}
$ curl -u : --negotiate -i "http://<router>:50071/webhdfs/v1/tmp/?op=LISTSTATUS"
(snip)
{"FileStatuses":{"FileStatus":[
{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":16387,"group":"supergroup","length":0,"modificationTime":1552016766769,"owner":"hdfs","pathSuffix":"tmp1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
]}}
{code}
The mount table is as follows:
{code}
$ hdfs dfsrouteradmin -ls /tmp
Mount Table Entries:
Source Destinations Owner
Group Mode Quota/Usage
/tmp ns1->/tmp aajisaka
users rwxr-xr-x [NsQuota: -/-, SsQuota: -/-]
/tmp/tmp1 ns1->/tmp/tmp1 aajisaka
users rwxr-xr-x [NsQuota: -/-, SsQuota: -/-]
/tmp/tmp2 ns2->/tmp/tmp2 aajisaka
users rwxr-xr-x [NsQuota: -/-, SsQuota: -/-]
{code}
Without trailing thrash, two directories are listed.
{code}
$ curl -u : --negotiate -i "http://<router>:50071/webhdfs/v1/tmp?op=LISTSTATUS"
(snip)
{"FileStatuses":{"FileStatus":[
{"accessTime":1541753421917,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753421917,"owner":"hdfs","pathSuffix":"tmp1","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"},
{"accessTime":1541753429812,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753429812,"owner":"hdfs","pathSuffix":"tmp2","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"}
]}}
{code}
[~ajisakaa] Thanks for reporting this, I borrowed the text from
HDFS-13972
was:
WebHDFS doesn't trim trailing slash causing discrepancy in operations.
Example below
--------------------------
Using HDFS API, two directory are listed.
$ hdfs dfs -ls hdfs://<router>:8888/tmp/
Found 2 items
drwxrwxrwx - hdfs supergroup 0 2018-11-09 17:50
hdfs://<router>:8888/tmp/tmp1
drwxrwxrwx - hdfs supergroup 0 2018-11-09 17:50
hdfs://<router>:8888/tmp/tmp2
Using WebHDFS API, only one directory is listed.
$ curl -u : --negotiate -i "http://<router>:50071/webhdfs/v1/tmp/?op=LISTSTATUS"
(snip)
{"FileStatuses":{"FileStatus":[
{"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":16387,"group":"supergroup","length":0,"modificationTime":1552016766769,"owner":"hdfs","pathSuffix":"tmp1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
]}}
The mount table is as follows:
$ hdfs dfsrouteradmin -ls /tmp
Mount Table Entries:
Source Destinations Owner
Group Mode Quota/Usage
/tmp ns1->/tmp aajisaka
users rwxr-xr-x [NsQuota: -/-, SsQuota: -/-]
/tmp/tmp1 ns1->/tmp/tmp1 aajisaka
users rwxr-xr-x [NsQuota: -/-, SsQuota: -/-]
/tmp/tmp2 ns2->/tmp/tmp2 aajisaka
users rwxr-xr-x [NsQuota: -/-, SsQuota: -/-]
Without trailing thrash, two directories are listed.
$ curl -u : --negotiate -i "http://<router>:50071/webhdfs/v1/tmp?op=LISTSTATUS"
(snip)
{"FileStatuses":{"FileStatus":[
{"accessTime":1541753421917,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753421917,"owner":"hdfs","pathSuffix":"tmp1","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"},
{"accessTime":1541753429812,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753429812,"owner":"hdfs","pathSuffix":"tmp2","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"}
]}}
[~ajisakaa] Thanks for reporting this, I borrowed the text from
HDFS-13972
> RBF: Fix trailing "/" for webhdfs
> ---------------------------------
>
> Key: HDFS-14369
> URL: https://issues.apache.org/jira/browse/HDFS-14369
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: CR Hota
> Assignee: CR Hota
> Priority: Major
>
> WebHDFS doesn't trim trailing slash causing discrepancy in operations.
> Example below
> --------------------------
> Using HDFS API, two directory are listed.
> {code}
> $ hdfs dfs -ls hdfs://<router>:8888/tmp/
> Found 2 items
> drwxrwxrwx - hdfs supergroup 0 2018-11-09 17:50
> hdfs://<router>:8888/tmp/tmp1
> drwxrwxrwx - hdfs supergroup 0 2018-11-09 17:50
> hdfs://<router>:8888/tmp/tmp2
> {code}
> Using WebHDFS API, only one directory is listed.
> {code}
> $ curl -u : --negotiate -i
> "http://<router>:50071/webhdfs/v1/tmp/?op=LISTSTATUS"
> (snip)
> {"FileStatuses":{"FileStatus":[
> {"accessTime":0,"blockSize":0,"childrenNum":0,"fileId":16387,"group":"supergroup","length":0,"modificationTime":1552016766769,"owner":"hdfs","pathSuffix":"tmp1","permission":"755","replication":0,"storagePolicy":0,"type":"DIRECTORY"}
> ]}}
> {code}
> The mount table is as follows:
> {code}
> $ hdfs dfsrouteradmin -ls /tmp
> Mount Table Entries:
> Source Destinations Owner
> Group Mode Quota/Usage
> /tmp ns1->/tmp aajisaka
> users rwxr-xr-x [NsQuota: -/-, SsQuota:
> -/-]
> /tmp/tmp1 ns1->/tmp/tmp1 aajisaka
> users rwxr-xr-x [NsQuota: -/-, SsQuota:
> -/-]
> /tmp/tmp2 ns2->/tmp/tmp2 aajisaka
> users rwxr-xr-x [NsQuota: -/-, SsQuota:
> -/-]
> {code}
> Without trailing thrash, two directories are listed.
> {code}
> $ curl -u : --negotiate -i
> "http://<router>:50071/webhdfs/v1/tmp?op=LISTSTATUS"
> (snip)
> {"FileStatuses":{"FileStatus":[
> {"accessTime":1541753421917,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753421917,"owner":"hdfs","pathSuffix":"tmp1","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"},
> {"accessTime":1541753429812,"blockSize":0,"childrenNum":0,"fileId":0,"group":"supergroup","length":0,"modificationTime":1541753429812,"owner":"hdfs","pathSuffix":"tmp2","permission":"777","replication":0,"storagePolicy":0,"symlink":"","type":"DIRECTORY"}
> ]}}
> {code}
> [~ajisakaa] Thanks for reporting this, I borrowed the text from
> HDFS-13972
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
