Also, looking at the hadoop-httpfs source, I see that 'doas' is the default parameter used: https://github.com/apache/hadoop-common/blob/release-2.2.0/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/java/org/apache/hadoop/fs/http/client/HttpFSFileSystem.java
I wonder if there is another spot in the code that checks and ignores the case, something I'm missing. On Tue, May 13, 2014 at 10:41 PM, Roshan Punnoose <[email protected]> wrote: > Kevin, > > I don't see anything in the /etc/hadoop/conf/* or > /etc/hadoop-httpfs/conf/* with the property "hadoop.http.authentication. > type". > > Dilli, > > Yeah I was afraid of that. I will have to test Oozie soon and make sure it > is working too. > > Here are the tests I have run: > > Directly to webhdfs: (after kinit as knox) > 1. curl -i --negotiate -u : > "http://<webhdfs_box>/webhdfs/v1?op=GETHOMEDIRECTORY&doAs=myuser" > > - Returns /user/knox (doAs did not work) > 2. curl -i --negotiate -u : > "http://<webhdfs_box>/webhdfs/v1?op=GETHOMEDIRECTORY&doas=myuser" > > - Returns /user/myuser (doas works) > > I made the change in knox, to the constant 'doAs' -> 'doas', and the > webhdfs url rewriting seems to work well. I will have to try and make sure > Oozie works too. Can you think of a good way I can incorporate an > Integration Test that would test this out as part of the build as well? > > Roshan > > > On Tue, May 13, 2014 at 6:23 PM, Dilli Arumugam <[email protected] > > wrote: > >> To answer your question on Oozie, yes, my testing sometime back showed >> that Oozie does not recognize "doas". Oozie recognized only "doAs". Have >> not tested Oozie with "doas" recently. >> >> Testing also showed that webhdfs recognized both "doas" and "doAs". >> >> Thanks >> Dilli >> >> >> >> On Tue, May 13, 2014 at 1:49 PM, Kevin Minder < >> [email protected]> wrote: >> >>> Would it be possible for you to send us your core-site.xml. I'm >>> wondering if you are using some variant of AuthenticatorHandler (i.e. >>> core-site.xml hadoop.http.authentication.type ) that we haven't tested >>> with. >>> >>> >>> On 5/13/14 4:03 PM, Roshan Punnoose wrote: >>> >>> Sure thing. The version we are testing against is "2.2.0.2.0.6.0-101" >>> which came with HDP. >>> >>> Yup, I modified that constant and it worked perfectly. I was surprised >>> too that this was the case. However, in the docs it looks like webhdfs is >>> expecting 'doas', all lowercase. I hope this change will not affect other >>> http endpoints. (Oozie?) >>> >>> >>> >>> On Tue, May 13, 2014 at 3:53 PM, larry mccay <[email protected]> wrote: >>> >>>> Hi Roshan - >>>> >>>> The constant can be found in: >>>> ./gateway-provider-identity-assertion-pseudo/src/main/java/org/apache/hadoop/gateway/identityasserter/filter/IdentityAsserterHttpServletRequestWrapper.java >>>> >>>> But... >>>> >>>> I'd like to understand why you are seeing this behavior. >>>> >>>> Can you provide more details about your deployment? >>>> >>>> Specifically, the version of wehdfs you are running would be great? >>>> >>>> We have done pretty extensive testing with secure clusters and should >>>> have seen this issue if it existed on the versions that we were using. >>>> >>>> thanks! >>>> >>>> --larry >>>> >>>> >>>> On Tue, May 13, 2014 at 2:05 PM, Roshan Punnoose <[email protected]>wrote: >>>> >>>>> I'm running knox 0.4.0, and I noticed that the webhdfs proxy query >>>>> parameter is 'doAs'. However, my httpfs (webhdfs) server does not >>>>> recognize >>>>> that parameter, but requires that it be 'doas'. The docs also mention this >>>>> as well. ( >>>>> http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html#Proxy_Users) >>>>> >>>>> >>>>> Has anyone else ran into this issue? Is it a simple fix in the >>>>> codebase, I did a search for 'doAs' and did not seem to find anything. >>>>> >>>>> Roshan >>>>> >>>> >>>> >>> >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or entity >>> to which it is addressed and may contain information that is confidential, >>> privileged and exempt from disclosure under applicable law. If the reader >>> of this message is not the intended recipient, you are hereby notified that >>> any printing, copying, dissemination, distribution, disclosure or >>> forwarding of this communication is strictly prohibited. If you have >>> received this communication in error, please contact the sender immediately >>> and delete it from your system. Thank You. >> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> > >
