[ 
https://issues.apache.org/jira/browse/HDFS-13419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433353#comment-16433353
 ] 

wangqiang.shen commented on HDFS-13419:
---------------------------------------

I don't have this use case, but i think sink token has expire time, we should 
prevent the use of token to access services if it was expired. Is that right?

> client can communicate to server even if hdfs delegation token expired
> ----------------------------------------------------------------------
>
>                 Key: HDFS-13419
>                 URL: https://issues.apache.org/jira/browse/HDFS-13419
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: wangqiang.shen
>            Priority: Minor
>
> i was testing hdfs delegation token expired problem use spark streaming, if i 
> set my batch interval small than 10 sec, my spark streaming program will not 
> dead, but if batch interval was setted bigger than 10 sec, the spark 
> streaming program will dead because of hdfs delegation token expire problem, 
> and the exception as follows
> {code:java}
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
>  token (HDFS_DELEGATION_TOKEN token 14042 for test) is expired
>       at org.apache.hadoop.ipc.Client.call(Client.java:1468)
>       at org.apache.hadoop.ipc.Client.call(Client.java:1399)
>       at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
>       at com.sun.proxy.$Proxy11.getListing(Unknown Source)
>       at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:554)
>       at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:606)
>       at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
>       at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
>       at com.sun.proxy.$Proxy12.getListing(Unknown Source)
>       at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1969)
>       at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:1952)
>       at 
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:693)
>       at 
> org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:105)
>       at 
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:755)
>       at 
> org.apache.hadoop.hdfs.DistributedFileSystem$15.doCall(DistributedFileSystem.java:751)
>       at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>       at 
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:751)
>       at com.envisioncn.arch.App$2$1.call(App.java:120)
>       at com.envisioncn.arch.App$2$1.call(App.java:91)
>       at 
> org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:218)
>       at 
> org.apache.spark.api.java.JavaRDDLike$$anonfun$foreachPartition$1.apply(JavaRDDLike.scala:218)
>       at 
> org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:902)
>       at 
> org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:902)
>       at 
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
>       at 
> org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1899)
>       at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
>       at org.apache.spark.scheduler.Task.run(Task.scala:86)
>       at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>       at java.lang.Thread.run(Thread.java:745)
> {code}
> the spark streaming program only call FileSystem.listStatus function in every 
> batch
> {code:java}
>                         FileSystem fs = FileSystem.get(new Configuration());
>                         FileStatus[] status =  fs.listStatus(new Path("/"));
>                         for(FileStatus status1 : status){
>                             System.out.println(status1.getPath());
>                         }
> {code}
> and i found when hadoop client send rpc request to server, it will first get 
> a connection object and set up the connection if the connection dose not 
> exists.And  it will get a SaslRpcClient to connect to the server side in the 
> connection setup stage.Also server will authenticate the client at the 
> connection setup stage. But if the connection exists, client will use the 
> existed connection, so the authentication stage will not happen. 
> The connection between client and server will be closed if it's idle time 
> exceeds ipc.client.connection.maxidletime, and 
> ipc.client.connection.maxidletime default value is 10sec. Therefore, if i 
> continue send request to server at fixed interval as long as the interval 
> small than 10sec, the connection will not be closed, so delegation token 
> expire problem will not happen.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to