[
https://issues.apache.org/jira/browse/HADOOP-16258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16825854#comment-16825854
]
Masatake Iwasaki commented on HADOOP-16258:
-------------------------------------------
The url in WebHdfsFileSystem looks like doubly encoded as reported while I got
no FileNotFoundException. The cause seems to be HDFS-13176. HDFS-13582 is the
follow-up. [~ebyhr], are you accessing old server by new client?
{noformat}
[iwasakims@centos7 hadoop-3.3.0-SNAPSHOT]$ bin/hdfs dfs -mkdir -p
'/tmp/dt=1/test1'
{noformat}
{noformat}
scala> val p = new Path("webhdfs://localhost:9870/tmp/dt=1")
p: org.apache.hadoop.fs.Path = webhdfs://localhost:9870/tmp/dt=1
scala> val fs = FileSystem.get(p.toUri, new Configuration())
fs: org.apache.hadoop.fs.FileSystem =
org.apache.hadoop.hdfs.web.WebHdfsFileSystem@4fa0e166
scala> val it = fs.listLocatedStatus(p)
2019-04-25 11:12:05,335 TRACE web.WebHdfsFileSystem:
url=http://localhost:9870/webhdfs/v1/tmp/dt%253D1?op=LISTSTATUS&user.name=iwasakims
2019-04-25 11:12:05,335 TRACE web.WebHdfsFileSystem:
url=http://localhost:9870/webhdfs/v1/tmp/dt%253D1?op=LISTSTATUS&user.name=iwasakims
it: org.apache.hadoop.fs.RemoteIterator[org.apache.hadoop.fs.LocatedFileStatus]
= org.apache.hadoop.fs.FileSystem$4@1d12d735
{noformat}
> FileSystem.listLocatedStatus for path including '=' encodes it and returns
> FileNotFoundException
> ------------------------------------------------------------------------------------------------
>
> Key: HADOOP-16258
> URL: https://issues.apache.org/jira/browse/HADOOP-16258
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.2.0
> Reporter: Yuya Ebihara
> Priority: Minor
> Labels: webhdfs
>
> Recently, we upgraded hadoop library from 2.7.7 to 3.2.0. This issue happens
> after the update. When we call FileSystem.listLocatedStatus with location
> 'webhdfs://hadoop-master:50070/user/hive/warehouse/test_part/dt=1', the
> internal calls are
> * 2.7.7
> [http://hadoop-master:50070/webhdfs/v1/user/hive/warehouse/test_part/dt=1?op=LISTSTATUS&user.name=xxx|http://hadoop-master:50070/webhdfs/v1/user/hive/warehouse/test_part/dt=1?op=LISTSTATUS&user.name=xxx%27,]
> * 3.2.0
> [http://hadoop-master:50070/webhdfs/v1/user/hive/warehouse/test_part/dt%253D1?op=LISTSTATUS&user.name=xxx]'
> As a result, it returns RemoteException with FileNotFoundException.
> {code:java}
> {"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File
> /user/hive/warehouse/test_part/dt%3D1 does not exist."}}
> {code}
> Could you please tell me whether it's a bug and the way to avoid it?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
