Oops - that solution was actually specific to a different part of our code
and when I changed the latter it broke. Updating my earlier message for
future Google searches.
We're always calling UserGroupInformation.createProxy() when we're using
user impersonation. If we do that then the three argument form of
FileSystem.getFileSystem that takes the name of a user to impersonate works
in SIMPLE auth but fails with Kerberos auth. There's no indication
whatsoever that it isn't supported, you just get an AccessControlException
with the generic "Unable to Authenticate[KERBEROS, TOKEN]" message.
I haven't tried using that method without calling createProxy() first.
Fortunately the two argument method, FileSystem.newFileSystem(URI,
Configuration), supports user impersonation with both SIMPLE and Kerberos
auth if you call createProxy() first. IIRC we had had problems earlier but
I now think that was due to our internal architecture where we created the
UGI in a different maven project than where we created the HDFS and they
got out of sync.
I haven't checked whether the three argument form works with Kerberos auth
if you don't call createProxy() since we want to use the proxied user since
it allows us to avoid passing around an extra parameter. It would
definitely be nice if the method was smart enough to recognize that the
user is already proxied for the same short username and didn't try an
additional doAs() block. I'm sure we're not the only organization that
handles user authentication (and creating the UGI) in a different
subproject than where we use it - it's easy to get out of sync in these
circumstances.
Bear
On Wed, Jan 31, 2018 at 7:38 AM, Bear Giles wrote:
> I figured it out. Of course it's obvious in retrospect.
>
> The tests passed after I added a call to user.setAuthMethod(KERBEROS)
> after createProxy(). I didn't need to do that with SIMPLE auth so I assumed
> the same would be true with Kerberos auth. The UGI's authentication method
> was set to PROXY but the real user had the SIMPLE or KERBEROS
> authentication method and information.
>
> It would probably be a Good Idea to mention this in the javadoc if it's
> not in the most recent versions. :-)
>
>
> On Tue, Jan 30, 2018 at 3:29 PM, Bear Giles wrote:
>
>> Back with a Kerberos impersonation question. The hadoop.proxyuser.*
>> settings are correct, at least the same settings worked on a different
>> cluster that doesn't require Kerberos authentication.
>>
>> I can perform my action as the basic user.
>>
>> When I use the same UGI code, add
>>
>> user = UGI.createProxy("new user", user);
>>
>> and attempt to perform the same action I get:
>>
>> java.io.IOException: Failed on local exception: java.io.IOException:
>> org.apache.hadoop.security.AccessControlException: Client cannot
>> authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: "
>> cdhclusterqa-2-2.clouddev.snaplogic.com/10.164.199.241"; destination
>> host is: "cdhclusterqa-2-1.clouddev.snaplogic.com":8020;
>>
>> Nothing else has changed. Literally - it's a checkbox toggle that does
>> nothing but conditionally call the code in blue.
>>
>> Any ideas? I did a 'relogin from keytab file' with the original user -
>> would I need to do that after the proxy call?
>>
>> (Hmm... I'm not familiar with this code but looking at the stack trace I
>> realize that the HDFS call is being made in a separate thread from the one
>> that acquired the original UGI credentials. The thread is created in a
>> privileged action so it has the basic information but may not have all
>> threadlocal information. I don't know why that decision was made. It's
>> suspicious... but the basic Kerberos authentication works. It's the
>> impersonation that's failing.)
>>
>> FWIW the bottommost few exceptions are:
>>
>> exc: java.io.IOException: Failed on local exception:
>> java.io.IOException: org.apache.hadoop.security.AccessControlException:
>> Client cannot authenticate via:[TOKEN, KERBEROS]; Host Details│
>> exc: at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:776)
>>
>> │
>> exc: at org.apache.hadoop.ipc.Client.call(Client.java:1480)
>>
>> │
>> exc: at org.apache.hadoop.ipc.Client.call(Client.java:1407)
>>
>> │
>> exc: at
>> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
>>
>> │
>> exc: at com.sun.proxy.$Proxy91.getFileInfo(Unknown Source)
>>
>> │
>> exc: at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTran
>> slatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
>> │
>> exc: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> │
>> exc: at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>
>
