[
https://issues.apache.org/jira/browse/YARN-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16497157#comment-16497157
]
Eric Yang commented on YARN-8308:
---------------------------------
[~gsaha] The patch is moving removeHdfsDelegationToken logic to outside of
doSecureLogin. Is this step necessary for cluster without Kerberos? What is
the reasoning behind adding mandatory CLI options for keytab and principal?
User can submit yarnfile that includes those two parameters.
> Yarn service app fails due to issues with Renew Token
> -----------------------------------------------------
>
> Key: YARN-8308
> URL: https://issues.apache.org/jira/browse/YARN-8308
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn-native-services
> Affects Versions: 3.1.0
> Reporter: Yesha Vora
> Assignee: Gour Saha
> Priority: Major
> Attachments: YARN-8308.001.patch, YARN-8308.002.patch
>
>
> Run Yarn service application beyond
> dfs.namenode.delegation.token.max-lifetime.
> Here, yarn service application fails with below error.
> {code}
> 2018-05-15 23:14:35,652 [main] WARN ipc.Client - Exception encountered while
> connecting to the server :
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
> token (token for hbase: HDFS_DELEGATION_TOKEN owner=hbase, renewer=yarn,
> realUser=rm/[email protected], issueDate=1526423999164, maxDate=1526425799164,
> sequenceNumber=7, masterKeyId=8) is expired, current time: 2018-05-15
> 23:14:35,651+0000 expected renewal time: 2018-05-15 23:09:59,164+0000
> 2018-05-15 23:14:35,654 [main] INFO service.AbstractService - Service
> Service Master failed in state INITED
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
> token (token for hbase: HDFS_DELEGATION_TOKEN owner=hbase, renewer=yarn,
> realUser=rm/[email protected], issueDate=1526423999164, maxDate=1526425799164,
> sequenceNumber=7, masterKeyId=8) is expired, current time: 2018-05-15
> 23:14:35,651+0000 expected renewal time: 2018-05-15 23:09:59,164+0000
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1491)
> at org.apache.hadoop.ipc.Client.call(Client.java:1437)
> at org.apache.hadoop.ipc.Client.call(Client.java:1347)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:883)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy12.getFileInfo(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1654)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
> at
> org.apache.hadoop.yarn.service.utils.JsonSerDeser.load(JsonSerDeser.java:182)
> at
> org.apache.hadoop.yarn.service.utils.ServiceApiUtil.loadServiceFrom(ServiceApiUtil.java:337)
> at
> org.apache.hadoop.yarn.service.ServiceMaster.loadApplicationJson(ServiceMaster.java:242)
> at
> org.apache.hadoop.yarn.service.ServiceMaster.serviceInit(ServiceMaster.java:91)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.service.ServiceMaster.main(ServiceMaster.java:316)
> 2018-05-15 23:14:35,659 [main] INFO service.ServiceMaster - Stopping app
> master
> 2018-05-15 23:14:35,660 [main] ERROR service.ServiceMaster - Error starting
> service master
> org.apache.hadoop.service.ServiceStateException:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
> token (token for hbase: HDFS_DELEGATION_TOKEN owner=hbase, renewer=yarn,
> realUser=rm/[email protected], issueDate=1526423999164, maxDate=1526425799164,
> sequenceNumber=7, masterKeyId=8) is expired, current time: 2018-05-15
> 23:14:35,651+0000 expected renewal time: 2018-05-15 23:09:59,164+0000
> at
> org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:173)
> at
> org.apache.hadoop.yarn.service.ServiceMaster.main(ServiceMaster.java:316)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
> token (token for hbase: HDFS_DELEGATION_TOKEN owner=hbase, renewer=yarn,
> realUser=rm/[email protected], issueDate=1526423999164, maxDate=1526425799164,
> sequenceNumber=7, masterKeyId=8) is expired, current time: 2018-05-15
> 23:14:35,651+0000 expected renewal time: 2018-05-15 23:09:59,164+0000
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1491)
> at org.apache.hadoop.ipc.Client.call(Client.java:1437)
> at org.apache.hadoop.ipc.Client.call(Client.java:1347)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:883)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy12.getFileInfo(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1654)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
> at
> org.apache.hadoop.yarn.service.utils.JsonSerDeser.load(JsonSerDeser.java:182)
> at
> org.apache.hadoop.yarn.service.utils.ServiceApiUtil.loadServiceFrom(ServiceApiUtil.java:337)
> at
> org.apache.hadoop.yarn.service.ServiceMaster.loadApplicationJson(ServiceMaster.java:242)
> at
> org.apache.hadoop.yarn.service.ServiceMaster.serviceInit(ServiceMaster.java:91)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> ... 1 more
> 2018-05-15 23:14:35,662 [main] INFO util.ExitUtil - Exiting with status 1:
> Error starting service master
> 2018-05-15 23:14:35,668 [pool-1-thread-1] INFO service.ServiceMaster -
> SHUTDOWN_MSG{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
