Stefaan Lippens created HDFS-14341:
--------------------------------------
Summary: Weird handling of plus sign in paths in WebHDFS REST API
Key: HDFS-14341
URL: https://issues.apache.org/jira/browse/HDFS-14341
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 3.1.1
Reporter: Stefaan Lippens
We're using Hadoop 3.1.1 at the moment and have an issue with the handling of
paths that contain plus signs (generated by Kafka HDFS Connector).
For example, I created this example directory {{tmp/plus+plus}}
{code:java}
$ hadoop fs -ls tmp/plus+plus
Found 1 items
-rw-r--r-- 3 stefaan supergroup 7079 2019-03-06 14:31
tmp/plus+plus/foo.txt{code}
When trying to list this folder through WebHDFS the naive way:
{code:java}
$ curl
'http://hadoopname05:9870/webhdfs/v1/user/stefaan/tmp/plus+plus?user.name=stefaan&op=LISTSTATUS'
{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File
/user/stefaan/tmp/plus plus does not exist."}}{code}
Fair enough, the plus sign {{+}} is a special character in URLs, let's encode
it as {{%2B}}:
{code:java}
$ curl
'http://hadoopname05:9870/webhdfs/v1/user/stefaan/tmp/plus%2Bplus?user.name=stefaan&op=LISTSTATUS'
{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File
/user/stefaan/tmp/plus plus does not exist."}}{code}
Doesn't work.
After some trial and error I found that I could get it working by encode the
thing twice ({{"+" -> "%2B" -> "%252B"}}):
{code:java}
curl
'http://hadoopname05:9870/webhdfs/v1/user/stefaan/tmp/plus%252Bplus?user.name=stefaan&op=LISTSTATUS'
{"FileStatuses":{"FileStatus":[
{"accessTime":1551882704527,"blockSize":134217728,"childrenNum":0,"fileId":314914,"group":"supergroup","length":7079,"modificationTime":1551882704655,"owner":"stefaan","pathSuffix":"foo.txt","permission":"644","replication":3,"storagePolicy":0,"type":"FILE"}
]}}{code}
Seems like there is some double decoding going on in WebHDFS REST API.
I also tried with some other special characters like {{@}} and {{=}}, and for
these it seems to work both when encoding once ({{%40}} and {{%3D}}
respectively) and encoding twice ({{%2540}} and {{%253D}} respectively)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
