[
https://issues.apache.org/jira/browse/HDFS-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568320#comment-14568320
]
Yongjun Zhang commented on HDFS-4564:
-------------------------------------
HI [~daryn],
Thanks for your earlier work on this jira. I need some help related to this
jira, thanks in advance for your time!
I'm looking into an issue that the following AuthenticationException is thrown
when accessing httpfs via webhdfs in a secure cluster,
{code}
systest@yj52s ~]$ java -cp $(hadoop classpath):. Foo
15/05/30 00:05:03 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.io.IOException:
org.apache.hadoop.security.authentication.client.AuthenticationException:
Unauthorized
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:309)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$700(WebHdfsFileSystem.java:108)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.getResponse(WebHdfsFileSystem.java:625)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:528)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$200(WebHdfsFileSystem.java:443)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:473)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:470)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:469)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:436)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:999)
at
org.apache.hadoop.hdfs.web.TokenAspect.ensureTokenInitialized(TokenAspect.java:140)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:235)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getAuthParameters(WebHdfsFileSystem.java:393)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toUrl(WebHdfsFileSystem.java:414)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathRunner.getUrl(WebHdfsFileSystem.java:649)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.init(WebHdfsFileSystem.java:483)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:524)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$200(WebHdfsFileSystem.java:443)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:473)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:470)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:469)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:436)
at
org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:982)
at Foo.main(Foo.java:7)
Caused by:
org.apache.hadoop.security.authentication.client.AuthenticationException:
Unauthorized
... 30 more
[systest@yj52s ~]$
{code}
The source code Foo.java
{code}
1 import org.apache.hadoop.conf.*;
2 import org.apache.hadoop.fs.*;
3 import org.apache.hadoop.security.*;
4 class Foo {
5 public static void main(String args[]) throws Exception {
6 FileSystem fs = FileSystem.get(new
java.net.URI("webhdfs://yj54s.vpc.cloudera.com:14000/"), new Configuration());
//accessing httpfs via webhdfs
7 System.out.println(fs.listStatus(new Path("/tmp"))[0]);
8 java.io.OutputStream os = fs.create(new Path("/tmp/foo"));
9 os.write('a');
10 os.close();
11 }
12 }
{code}
When I ran without HDFS-4564, it appears to run ok. So the behavior of the
above program is changed by HDFS-4564. I guess it worked because "incorrect
http response returned" (which is fixed by HDFS-4564).
I traced the execution a bit and I see no "fallback to pseudo auth" happens in
this working run.
Would you please shed some light about your understanding here?
Thanks a lot!
> Webhdfs returns incorrect http response codes for denied operations
> -------------------------------------------------------------------
>
> Key: HDFS-4564
> URL: https://issues.apache.org/jira/browse/HDFS-4564
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: webhdfs
> Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Blocker
> Fix For: 2.4.0
>
> Attachments: HDFS-4564.branch-23.patch, HDFS-4564.branch-23.patch,
> HDFS-4564.branch-23.patch, HDFS-4564.patch, HDFS-4564.patch
>
>
> Webhdfs is returning 401 (Unauthorized) instead of 403 (Forbidden) when it's
> denying operations. Examples including rejecting invalid proxy user attempts
> and renew/cancel with an invalid user.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
