Can you show us the curl command that you are using when sending
requests to WebHdfs via the gateway?
On 5/13/14 10:41 PM, Roshan Punnoose wrote:
Kevin,
I don't see anything in the /etc/hadoop/conf/* or
/etc/hadoop-httpfs/conf/* with the property
"hadoop.http.authentication.type".
Dilli,
Yeah I was afraid of that. I will have to test Oozie soon and make
sure it is working too.
Here are the tests I have run:
Directly to webhdfs: (after kinit as knox)
1. curl -i --negotiate -u :
"http://<webhdfs_box>/webhdfs/v1?op=GETHOMEDIRECTORY&doAs=myuser"
- Returns /user/knox (doAs did not work)
2. curl -i --negotiate -u :
"http://<webhdfs_box>/webhdfs/v1?op=GETHOMEDIRECTORY&doas=myuser"
- Returns /user/myuser (doas works)
I made the change in knox, to the constant 'doAs' -> 'doas', and the
webhdfs url rewriting seems to work well. I will have to try and make
sure Oozie works too. Can you think of a good way I can incorporate an
Integration Test that would test this out as part of the build as well?
Roshan
On Tue, May 13, 2014 at 6:23 PM, Dilli Arumugam
<[email protected] <mailto:[email protected]>> wrote:
To answer your question on Oozie, yes, my testing sometime back
showed that Oozie does not recognize "doas". Oozie recognized
only "doAs". Have not tested Oozie with "doas" recently.
Testing also showed that webhdfs recognized both "doas" and "doAs".
Thanks
Dilli
On Tue, May 13, 2014 at 1:49 PM, Kevin Minder
<[email protected]
<mailto:[email protected]>> wrote:
Would it be possible for you to send us your core-site.xml.
I'm wondering if you are using some variant of
AuthenticatorHandler (i.e. core-site.xml
hadoop.http.authentication.type ) that we haven't tested with.
On 5/13/14 4:03 PM, Roshan Punnoose wrote:
Sure thing. The version we are testing against is
"2.2.0.2.0.6.0-101" which came with HDP.
Yup, I modified that constant and it worked perfectly. I was
surprised too that this was the case. However, in the docs it
looks like webhdfs is expecting 'doas', all lowercase. I hope
this change will not affect other http endpoints. (Oozie?)
On Tue, May 13, 2014 at 3:53 PM, larry mccay
<[email protected] <mailto:[email protected]>> wrote:
Hi Roshan -
The constant can be found in:
./gateway-provider-identity-assertion-pseudo/src/main/java/org/apache/hadoop/gateway/identityasserter/filter/IdentityAsserterHttpServletRequestWrapper.java
But...
I'd like to understand why you are seeing this behavior.
Can you provide more details about your deployment?
Specifically, the version of wehdfs you are running would
be great?
We have done pretty extensive testing with secure
clusters and should have seen this issue if it existed on
the versions that we were using.
thanks!
--larry
On Tue, May 13, 2014 at 2:05 PM, Roshan Punnoose
<[email protected] <mailto:[email protected]>> wrote:
I'm running knox 0.4.0, and I noticed that the
webhdfs proxy query parameter is 'doAs'. However, my
httpfs (webhdfs) server does not recognize that
parameter, but requires that it be 'doas'. The docs
also mention this as well.
(http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Proxy_Users)
Has anyone else ran into this issue? Is it a simple
fix in the codebase, I did a search for 'doAs' and
did not seem to find anything.
Roshan
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual
or entity to which it is addressed and may contain information
that is confidential, privileged and exempt from disclosure
under applicable law. If the reader of this message is not the
intended recipient, you are hereby notified that any printing,
copying, dissemination, distribution, disclosure or forwarding
of this communication is strictly prohibited. If you have
received this communication in error, please contact the
sender immediately and delete it from your system. Thank You.
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
entity to which it is addressed and may contain information that
is confidential, privileged and exempt from disclosure under
applicable law. If the reader of this message is not the intended
recipient, you are hereby notified that any printing, copying,
dissemination, distribution, disclosure or forwarding of this
communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and
delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
