hi, Please let me know how it is possible to get tokens and related stuffs in TEZ framework?
On Tue, Oct 7, 2014 at 9:11 AM, Subroto Sanyal <[email protected]> wrote: > hi Vinod, hi Bikas > > Thanks for your inputs. > Though I am able to spawn the DAGAppMaster with proxy user (I can see the > DAGAppMaster running with proxy user in resource manager UI) but, the call: > TezClient.getAppMasterStatus > fails with security issues. > I have raised a ticket with my client code and more details: > https://issues.apache.org/jira/browse/TEZ-1640 > > (UserGroupInformation.java:1551) - PriviledgedActionException as:qa > (auth:SIMPLE) cause:java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS] > Failed to retrieve AM Status via proxy > com.google.protobuf.ServiceException: java.io.IOException: Failed on local > exception: java.io.IOException: > org.apache.hadoop.security.AccessControlException: Client cannot authenticate > via:[TOKEN, KERBEROS]; Host Details : local host is: > "ip-10-178-144-254/10.178.144.254"; destination host is: > "ip-10-187-33-206":56660; > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216) > at com.sun.proxy.$Proxy111.getAMStatus(Unknown Source) > at > org.apache.tez.client.TezClient.getAppMasterStatus(TezClient.java:522) > > > > > On Tue, Oct 7, 2014 at 4:25 AM, Bikas Saha <[email protected]> wrote: > >> Thanks Vinod. >> >> >> >> Subroto, looks like there are 2 issues here. >> >> >> >> One is running as use foo but submitting the job as user bar. Vinod’s >> code snippet is relevant to that (Please see my answer with createProxyUser >> earlier and Vinod’s exact code below). The real user is foo but the >> effective user is bar. So when the processes connects to YARN then it YARN >> will use the effective user bar as the job user instead of foo. >> >> >> >> Second is how to obtain credentials for bar while running as foo. That is >> covered by the tez-credentials-file-path solution mentioned earlier. This >> allows user foo to collect credentials from HDFS for user bar (foo is >> expected to be a trusted proxy user in HDFS to be allowed to do this. Like >> Oozie). Then the credentials are passed to the client via the file. >> >> >> >> Hope both these clarify your situation and move you forward. >> >> >> >> Bikas >> >> >> >> *From:* Vinod Kumar Vavilapalli [mailto:[email protected]] >> *Sent:* Monday, October 06, 2014 11:39 AM >> >> *To:* [email protected] >> *Subject:* Re: Tez with secured hadoop >> >> >> >> That is not the way to do impersonation. Please see the following: >> >> UserGroupInformation ugi = >> UserGroupInformation.createProxyUser("qa", >> UserGroupInformation.getLoginUser()); // <-- login user is subroto, the >> realUser. "qa" is the expected effective-user >> ugi.doAs(new PrivilegedExceptionAction<Void>() { >> public Void run() throws Exception { >> .. >> } >> } >> >> >> +Vinod >> Hortonworks Inc. >> http://hortonworks.com/ >> >> >> >> On Thu, Oct 2, 2014 at 6:22 AM, Subroto Sanyal <[email protected]> >> wrote: >> >> hi Bikas, >> >> >> >> My code snippet to create TezClient looks like (TEZ-0.5): >> >> new PrivilegedExceptionAction<TezClient>() { >> >> >> >> @Override >> >> public TezClient run() throws Exception { >> >> UserGroupInformation currentUser = >> UserGroupInformation.getCurrentUser(); >> >> LOG.info("Current User:" + currentUser); >> >> File tokenFile = new >> File(System.getProperty("java.io.tmpdir"), >> tezSessionName.replaceAll("[^a-zA-Z0-9]", "")); >> >> LOG.info("Token File:" + tokenFile.getAbsolutePath()); >> >> >> currentUser.getCredentials().writeTokenStorageFile(UriUtil.toPath(tokenFile.getAbsoluteFile()), >> conf); >> >> tezConf.set(TezConfiguration.TEZ_CREDENTIALS_PATH, >> tokenFile.getAbsolutePath()); >> >> return TezClient.create(tezSessionName, tezConf, >> createSession, localResourceMap, credentials); >> >> } >> >> } >> >> >> >> The logs generated from this piece of code during execution looks like: >> >> (TezClientFacade.java:142) - Current User:qa (auth:PROXY) via >> [email protected] (auth:KERBEROS) >> >> (TezClientFacade.java:144) - Token File:/home/subroto/tmp/testTezJob >> >> >> >> When this runs on cluster the job runs as "subroto" but, what I expect is >> to run it as "qa". >> >> >> >> Please let me know if there is something missing or wrong in the code. >> >> >> >> On Sat, Sep 13, 2014 at 3:33 AM, Bikas Saha <[email protected]> >> wrote: >> >> If by impersonation you mean what Oozie does where Oozie runs as Oozie >> but get delegation tokens for user FOO then you will need to follow the >> mechanism that Oozie uses. Oozie writes the delegation tokens into a file >> and put that file in a specific path which is picked up by the application >> (in this case TezClient) and the application loads credentials from that >> file. In case of Tez the location of the credentials file is the value of >> config "tez.credentials.path" >> >> >> >> Bikas >> >> >> >> *From:* Bikas Saha [mailto:[email protected]] >> *Sent:* Monday, September 01, 2014 5:34 PM >> *To:* [email protected] >> *Subject:* RE: Tez with secured hadoop >> >> >> >> They way this is supposed to work is the following in a secure cluster. >> >> 1) The user that is running TezClient/DAGClient needs to be >> Kerberos authenticated. This allows the process running DAGClient/TezClient >> to contact the RM and get tokens to communicate with the AM. >> >> 2) The TezClient/DAGClient uses the tokens obtained from the RM and >> populates it into the current user UGI (ie the use who is running >> TezClient/DAGClient). The RPC to the AM will try to authenticate the >> current user using the tokens just added to the current users UGI. >> >> >> >> In a non-secure environment, no tokens are needed. So I am guessing that >> your are running in a secure env. >> >> >> >> Given the above info, what is happening in your case. Whichever user the >> client is running under, it looks like it can authenticate to the RM to get >> the app report. So it should have gotten tokens to access the AM. Its not >> clear what you mean by user “subroto” being privileged and the real user >> not considered by Tez. It looks like you are running the client as user >> “subroto”. Who is “subroto” and who is the real user? >> >> >> >> Does this happen always or occasionally. There is a known race condition >> in YARN where the client gets tokens before the AM gets the key to validate >> the tokens. >> >> >> >> You can turn on debug logging and see the SASL negotiation logs to get >> more info on whats happening. You may add a debug log in getAMProxy() to >> verify that token were obtained from the RM and added to the UGI. >> >> >> >> It may help if you describe your scenario. What are you trying to achieve >> by impersonation and how are you trying to do that. We recently added ACLs >> in case that works for your scenario. >> >> >> >> *From:* Subroto Sanyal [mailto:[email protected]] >> *Sent:* Sunday, August 31, 2014 8:59 PM >> *To:* [email protected] >> *Subject:* Re: Tez with secured hadoop >> >> >> >> Hi Bikas, >> >> >> >> In the method: >> >> org.apache.tez.client.TezClientUtils.getAMProxy(Configuration, String, >> int, Token) a UGI is getting created with name of the current user. I think >> in this process it ignores all the security things and making the >> authentication mode as "SIMPLE". I have piece of code which tries to create >> a TezClient and it keeps throwing the exception: >> >> >> >> [anonymous] WARN [2014-08-28 03:37:50.181] [MrPlanRunnerV2] >> (UserGroupInformation.java:1551) - PriviledgedActionException as:subroto >> (auth:SIMPLE) cause:java.io.IOException: >> org.apache.hadoop.security.AccessControlException: Client cannot >> authenticate via:[TOKEN, KERBEROS] >> >> [anonymous] INFO [2014-08-28 03:37:50.182] [MrPlanRunnerV2] >> (TezClient.java:539) - Failed to retrieve AM Status via proxy >> >> com.google.protobuf.ServiceException: java.io.IOException: Failed on >> local exception: java.io.IOException: >> org.apache.hadoop.security.AccessControlException: Client cannot >> authenticate via:[TOKEN, KERBEROS]; Host Details : local host is: >> "domU-12-31-39-0F-74-32/10.193.119.192"; destination host is: >> "domU-12-31-39-0C-7D-37":59431; >> >> at >> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216) >> >> at com.sun.proxy.$Proxy111.getAMStatus(Unknown Source) >> >> at org.apache.tez.client.TezClient.getAppMasterStatus(TezClient.java:532) >> >> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:607) >> >> at subroto.tez.TezClusterSession$2.run(TezClusterSession.java:180) >> >> >> >> I m trying to achieve impersonation. Here user "subroto" is privileged >> user and the real user is not at all considered by the Tez Code. >> >> >> >> Request some suggestion on this. >> >> >> >> On Tue, Aug 19, 2014 at 11:18 PM, Bikas Saha <[email protected]> >> wrote: >> >> There is nothing special that you need to do if you are already running >> secure Map Reduce jobs. The client needs to run in a Kerberized >> authenticated context. After that if you are using the built-in library of >> inputs/outputs etc then they should be taking care of all the access >> credentials for you when using the 0.5 API. I >> >> >> >> If you are using 0.4 API to write your job then you may need to use >> additional APIs for passing credentials to the application. Look for >> credentials in >> https://github.com/apache/tez/blob/branch-0.4.0-incubating/tez-mapreduce-examples/src/main/java/org/apache/tez/mapreduce/examples/FilterLinesByWord.java >> and also *public* *synchronized* DAG *addURIsForCredentials(*Collection >> *<*URI*>* uris*)* >> >> >> >> The second method is a shortcut if you are using HDFS files for input. It >> obtains credentials for you from a collection of HDFS input URIs. >> >> >> >> Bikas >> >> >> >> *From:* Subroto Sanyal [mailto:[email protected]] >> *Sent:* Tuesday, August 19, 2014 3:30 AM >> *To:* [email protected] >> *Subject:* Tez with secured hadoop >> >> >> >> hi >> >> >> >> Tez works on secure hadoop cluster since tez-0.3. >> >> Is there any documentation available about configuring TezClient to make >> it work? >> >> >> >> -- >> Cheers, >> *Subroto Sanyal* >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> >> >> >> >> >> -- >> Cheers, >> *Subroto Sanyal* >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> >> >> >> >> >> -- >> Cheers, >> *Subroto Sanyal* >> >> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity >> to which it is addressed and may contain information that is confidential, >> privileged and exempt from disclosure under applicable law. If the reader >> of this message is not the intended recipient, you are hereby notified that >> any printing, copying, dissemination, distribution, disclosure or >> forwarding of this communication is strictly prohibited. If you have >> received this communication in error, please contact the sender immediately >> and delete it from your system. Thank You. >> > > > > -- > Cheers, > *Subroto Sanyal* > -- Cheers, *Subroto Sanyal*
