Toshihiko Uchida created HDFS-15465:
---------------------------------------
Summary: Support WebHDFS accesses to the data stored in secure
Datanode through insecure Namenode
Key: HDFS-15465
URL: https://issues.apache.org/jira/browse/HDFS-15465
Project: Hadoop HDFS
Issue Type: Wish
Components: federation, webhdfs
Reporter: Toshihiko Uchida
Assignee: Toshihiko Uchida
Attachments: webhdfs-federation.pdf
We're federating a secure HDFS cluster with an insecure cluster.
Using HDFS RPC, we can access the data managed by insecure Namenode and stored
in secure Datanode.
However, it does not work for WebHDFS due to HadoopIllegalArgumentException.
{code}
$ curl -i "http://<INSECURE_NAMENODE>:<PORT>/webhdfs/v1/<PATH>?op=OPEN"
HTTP/1.1 307 TEMPORARY_REDIRECT
(omitted)
Location:
http://<SECURE_DATANODE>:<PORT>/webhdfs/v1/<PATH>?op=OPEN&namenoderpcaddress=<INSECURE_NAMENODE>&offset=0
$ curl -i
"http://<SECURE_DATANODE>:<PORT>/webhdfs/v1/<PATH>?op=OPEN&namenoderpcaddress=<INSECURE_NAMENODE>&offset=0"
HTTP/1.1 400 Bad Request
(omitted)
{"RemoteException":{"exception":"HadoopIllegalArgumentException","javaClassName":"org.apache.hadoop.HadoopIllegalArgumentException","message":"Invalid
argument, newValue is null"}}
{code}
This is because secure Datanode expects a delegation token, but insecure
Namenode does not return it to a client.
- org.apache.hadoop.security.token.Token.decodeWritable
{code}
private static void decodeWritable(Writable obj,
String newValue) throws IOException {
if (newValue == null) {
throw new HadoopIllegalArgumentException(
"Invalid argument, newValue is null");
}
{code}
The issue proposes to support the access also for WebHDFS.
The attached PDF file [^webhdfs-federation.pdf] depicts our current
architecture and proposal.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
