Hi Raja, Yes the keytab would be copied over to HDFS and reused for getting a new token before the old one expires. By default it is 7 days. If it is different in your cluster please set the properties dt.resourcemanager.delegation.token.max-lifetime and dt.namenode.delegation.token.max-lifetime in dt-site.xml. Also if you don't the default keytab to be copied over into HDFS and reused you can specify your own keytab file for fetching a new token by putting it in HDFS and specifying the property dt.authentication.store.keytab.All this is described in the document that Thomas sent over.
Thanks On Mon, Jun 20, 2016 at 1:54 PM, Raja.Aravapalli <[email protected] > wrote: > > Hi Thomas, > > To ensure auto renewal of delegation tokens life time, Can I use the the > below properties in properties.xml file ? > > <property> > <name>dt.authentication.principal</name> > <value>kerberos-principal-of-user</value> > </property> > <property> > <name>dt.authentication.keytab</name> > <value>absolute-path-to-keytab-file</value> > </property> > > > FYI, > I am launching application from Apex CLI! And till this time I haven’t > used the above properties when launching apex applications in our secure > hadoop environment, still they worked fine without any issues, but failing > after 7days!! > > If I set the above properties in properties.xml, will that do auto-renewal > and run successfully without any issues of failing again due to delegation > token lifetime expiry ?? > > Please advise. > > > Thanks a lot in advance. > > > Regards, > Raja. > > From: "Raja.Aravapalli" <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Sunday, June 19, 2016 at 3:30 PM > > To: "[email protected]" <[email protected]> > Subject: Re: how to increase lifetime of hdfs delegation tokens ? > > > Thanks a lot Thomas. > > Will take this as reference and test our application. Great! > > > Regards, > Raja. > > From: Thomas Weise <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Sunday, June 19, 2016 at 2:01 PM > To: "[email protected]" <[email protected]> > Subject: Re: how to increase lifetime of hdfs delegation tokens ? > > Token expiration working as expected! > > Please have a look on how to extend or refresh it: > > > https://github.com/apache/apex-core/blob/master/docs/security.md#token-refresh > > Thanks, > Thomas > > > On Sat, Jun 18, 2016 at 10:26 PM, Raja.Aravapalli < > [email protected]> wrote: > >> >> Hi, >> >> My Apex application failed exactly after running 7days in our distributed >> hadoop environment, with delegation token expiry!! >> >> Can someone pls help me with details, on how I can increase the >> delegation token time to lifetime or any other process running in parallel >> to renew the tokens ? >> >> *Exception details below:* >> >> ERROR hdfs.DFSClient (DFSClient.java:closeAllFilesBeingWritten(954)) - >> Failed to close inode 11111111 >> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): >> token (HDFS_DELEGATION_TOKEN token 111111 for XXXXXX) is expired >> at org.apache.hadoop.ipc.Client.call(Client.java:1427) >> at org.apache.hadoop.ipc.Client.call(Client.java:1358) >> >> >> >> Thanks a lot in advance. >> >> >> Regards, >> Raja. >> >> >
