Alexander Shorin created HDFS-9903:
--------------------------------------
Summary: File could be created, but still not found when path is
sort of Unicode
Key: HDFS-9903
URL: https://issues.apache.org/jira/browse/HDFS-9903
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Environment: {code}
>>> import requests
>>> requests.put('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9?user.name=test&op=MKDIRS')
<Response [200]>
>>> requests.get('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9?user.name=test&op=GETFILESTATUS')
<Response [200]>
>>> resp =
>>> requests.put('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9/test.txt?user.name=test&op=CREATE',
>>> allow_redirects=False)
>>> resp
<Response [307]>
>>> loc = resp.headers['location']
>>> resp = requests.put(loc, data='bug')
>>> resp
<Response [201]>
>>> resp =
>>> requests.get('http://localhost:50070/webhdfs/v1/tmp/bug/%E1%BF%B9/test.txt?user.name=test&op=GETFILESTATUS')
>>> resp.content
'{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File
does not exist: /tmp/bug/\xe1\xbf\xb9/test.txt"}}'
>> resp = requests.put(loc, data='bug')
>>> resp
<Response [403]>
>>> resp.content
'{"RemoteException":{"exception":"FileAlreadyExistsException","javaClassName":"org.apache.hadoop.fs.FileAlreadyExistsException","message":"/tmp/bug/?/test.txt
for client 127.0.0.1 already exists\\n\\tat
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2782)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2674)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2559)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:592)\\n\\tat
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.create(AuthorizationProviderProxyClientProtocol.java:110)\\n\\tat
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:395)\\n\\tat
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)\\n\\tat
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)\\n\\tat
org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)\\n\\tat
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)\\n\\tat
org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)\\n\\tat
java.security.AccessController.doPrivileged(Native Method)\\n\\tat
javax.security.auth.Subject.doAs(Subject.java:415)\\n\\tat
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)\\n\\tat
org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)\\n"}}'
{code}
Things to notice:
1. While we receive HTTP 201 Created on file creation, it couldn't be found via
API. Physically it exists and really created.
2. The GETFILESTATUS against the file returns, again, not the best JSON, but
Python can parse UTF-8 bits there;
3. The second attempt to create the file on the same location borks the Unicode
in path within the error message;
Reporter: Alexander Shorin
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
