That is not the way to do impersonation. Please see the following:
UserGroupInformation ugi =
UserGroupInformation.createProxyUser("qa",
UserGroupInformation.getLoginUser()); // <-- login user is subroto, the
realUser. "qa" is the expected effective-user
ugi.doAs(new PrivilegedExceptionAction<Void>() {
public Void run() throws Exception {
..
}
}
+Vinod
Hortonworks Inc.
http://hortonworks.com/
On Thu, Oct 2, 2014 at 6:22 AM, Subroto Sanyal <[email protected]>
wrote:
> hi Bikas,
>
> My code snippet to create TezClient looks like (TEZ-0.5):
>
> new PrivilegedExceptionAction<TezClient>() {
>
>
> @Override
>
> public TezClient run() throws Exception {
>
> UserGroupInformation currentUser =
> UserGroupInformation.getCurrentUser();
>
> LOG.info("Current User:" + currentUser);
>
> File tokenFile = new File(System.getProperty(
> "java.io.tmpdir"), tezSessionName.replaceAll("[^a-zA-Z0-9]", ""));
>
> LOG.info("Token File:" + tokenFile.getAbsolutePath());
>
>
> currentUser.getCredentials().writeTokenStorageFile(UriUtil.toPath(tokenFile.getAbsoluteFile()),
> conf);
>
> tezConf.set(TezConfiguration.TEZ_CREDENTIALS_PATH,
> tokenFile.getAbsolutePath());
>
> return TezClient.create(tezSessionName, tezConf,
> createSession, localResourceMap, credentials);
>
> }
>
> }
>
>
> The logs generated from this piece of code during execution looks like:
>
> (TezClientFacade.java:142) - Current User:qa (auth:PROXY) via
> [email protected] (auth:KERBEROS)
>
> (TezClientFacade.java:144) - Token File:/home/subroto/tmp/testTezJob
>
>
> When this runs on cluster the job runs as "subroto" but, what I expect is
> to run it as "qa".
>
>
> Please let me know if there is something missing or wrong in the code.
>
> On Sat, Sep 13, 2014 at 3:33 AM, Bikas Saha <[email protected]> wrote:
>
>> If by impersonation you mean what Oozie does where Oozie runs as Oozie
>> but get delegation tokens for user FOO then you will need to follow the
>> mechanism that Oozie uses. Oozie writes the delegation tokens into a file
>> and put that file in a specific path which is picked up by the application
>> (in this case TezClient) and the application loads credentials from that
>> file. In case of Tez the location of the credentials file is the value of
>> config "tez.credentials.path"
>>
>>
>>
>> Bikas
>>
>>
>>
>> *From:* Bikas Saha [mailto:[email protected]]
>> *Sent:* Monday, September 01, 2014 5:34 PM
>> *To:* [email protected]
>> *Subject:* RE: Tez with secured hadoop
>>
>>
>>
>> They way this is supposed to work is the following in a secure cluster.
>>
>> 1) The user that is running TezClient/DAGClient needs to be
>> Kerberos authenticated. This allows the process running DAGClient/TezClient
>> to contact the RM and get tokens to communicate with the AM.
>>
>> 2) The TezClient/DAGClient uses the tokens obtained from the RM and
>> populates it into the current user UGI (ie the use who is running
>> TezClient/DAGClient). The RPC to the AM will try to authenticate the
>> current user using the tokens just added to the current users UGI.
>>
>>
>>
>> In a non-secure environment, no tokens are needed. So I am guessing that
>> your are running in a secure env.
>>
>>
>>
>> Given the above info, what is happening in your case. Whichever user the
>> client is running under, it looks like it can authenticate to the RM to get
>> the app report. So it should have gotten tokens to access the AM. Its not
>> clear what you mean by user “subroto” being privileged and the real user
>> not considered by Tez. It looks like you are running the client as user
>> “subroto”. Who is “subroto” and who is the real user?
>>
>>
>>
>> Does this happen always or occasionally. There is a known race condition
>> in YARN where the client gets tokens before the AM gets the key to validate
>> the tokens.
>>
>>
>>
>> You can turn on debug logging and see the SASL negotiation logs to get
>> more info on whats happening. You may add a debug log in getAMProxy() to
>> verify that token were obtained from the RM and added to the UGI.
>>
>>
>>
>> It may help if you describe your scenario. What are you trying to achieve
>> by impersonation and how are you trying to do that. We recently added ACLs
>> in case that works for your scenario.
>>
>>
>>
>> *From:* Subroto Sanyal [mailto:[email protected]]
>> *Sent:* Sunday, August 31, 2014 8:59 PM
>> *To:* [email protected]
>> *Subject:* Re: Tez with secured hadoop
>>
>>
>>
>> Hi Bikas,
>>
>>
>>
>> In the method:
>>
>> org.apache.tez.client.TezClientUtils.getAMProxy(Configuration, String,
>> int, Token) a UGI is getting created with name of the current user. I think
>> in this process it ignores all the security things and making the
>> authentication mode as "SIMPLE". I have piece of code which tries to create
>> a TezClient and it keeps throwing the exception:
>>
>>
>>
>> [anonymous] WARN [2014-08-28 03:37:50.181] [MrPlanRunnerV2]
>> (UserGroupInformation.java:1551) - PriviledgedActionException as:subroto
>> (auth:SIMPLE) cause:java.io.IOException:
>> org.apache.hadoop.security.AccessControlException: Client cannot
>> authenticate via:[TOKEN, KERBEROS]
>>
>> [anonymous] INFO [2014-08-28 03:37:50.182] [MrPlanRunnerV2]
>> (TezClient.java:539) - Failed to retrieve AM Status via proxy
>>
>> com.google.protobuf.ServiceException: java.io.IOException: Failed on
>> local exception: java.io.IOException:
>> org.apache.hadoop.security.AccessControlException: Client cannot
>> authenticate via:[TOKEN, KERBEROS]; Host Details : local host is:
>> "domU-12-31-39-0F-74-32/10.193.119.192"; destination host is:
>> "domU-12-31-39-0C-7D-37":59431;
>>
>> at
>> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216)
>>
>> at com.sun.proxy.$Proxy111.getAMStatus(Unknown Source)
>>
>> at org.apache.tez.client.TezClient.getAppMasterStatus(TezClient.java:532)
>>
>> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:607)
>>
>> at subroto.tez.TezClusterSession$2.run(TezClusterSession.java:180)
>>
>>
>>
>> I m trying to achieve impersonation. Here user "subroto" is privileged
>> user and the real user is not at all considered by the Tez Code.
>>
>>
>>
>> Request some suggestion on this.
>>
>>
>>
>> On Tue, Aug 19, 2014 at 11:18 PM, Bikas Saha <[email protected]>
>> wrote:
>>
>> There is nothing special that you need to do if you are already running
>> secure Map Reduce jobs. The client needs to run in a Kerberized
>> authenticated context. After that if you are using the built-in library of
>> inputs/outputs etc then they should be taking care of all the access
>> credentials for you when using the 0.5 API. I
>>
>>
>>
>> If you are using 0.4 API to write your job then you may need to use
>> additional APIs for passing credentials to the application. Look for
>> credentials in
>> https://github.com/apache/tez/blob/branch-0.4.0-incubating/tez-mapreduce-examples/src/main/java/org/apache/tez/mapreduce/examples/FilterLinesByWord.java
>> and also *public* *synchronized* DAG *addURIsForCredentials(*Collection
>> *<*URI*>* uris*)*
>>
>>
>>
>> The second method is a shortcut if you are using HDFS files for input. It
>> obtains credentials for you from a collection of HDFS input URIs.
>>
>>
>>
>> Bikas
>>
>>
>>
>> *From:* Subroto Sanyal [mailto:[email protected]]
>> *Sent:* Tuesday, August 19, 2014 3:30 AM
>> *To:* [email protected]
>> *Subject:* Tez with secured hadoop
>>
>>
>>
>> hi
>>
>>
>>
>> Tez works on secure hadoop cluster since tez-0.3.
>>
>> Is there any documentation available about configuring TezClient to make
>> it work?
>>
>>
>>
>> --
>> Cheers,
>> *Subroto Sanyal*
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>>
>>
>>
>>
>> --
>> Cheers,
>> *Subroto Sanyal*
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>
>
> --
> Cheers,
> *Subroto Sanyal*
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
