[
https://issues.apache.org/jira/browse/HDFS-1965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13037090#comment-13037090
]
Tsz Wo (Nicholas), SZE commented on HDFS-1965:
----------------------------------------------
It seems that the reasons of {{TestFileConcurrentReader}} failing are:
- The test open many files within a short period of time, says in a few seconds.
- {{DFSClient}} creates a proxy for each open.
- Since the default ipc.client.connection.maxidletime is 10 seconds, so the
proxies are not yet closed.
- Therefore, {{TestFileConcurrentReader}} fails with runtime exceptions (out of
descriptors?)
Todd, do you agree?
*Questions*: We already have {{RPC.stopProxy(cdp)}} in a finally-block. Why
the resource is still not released? Is it because {{TestFileConcurrentReader}}
opens files so fast that the finally-block is not yet reached? Or
{{RPC.stopProxy(..)}} does not work?
> IPCs done using block token-based tickets can't reuse connections
> -----------------------------------------------------------------
>
> Key: HDFS-1965
> URL: https://issues.apache.org/jira/browse/HDFS-1965
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Fix For: 0.22.0
>
> Attachments: hdfs-1965.txt, hdfs-1965.txt
>
>
> This is the reason that TestFileConcurrentReaders has been failing a lot.
> Reproducing a comment from HDFS-1057:
> The test has a thread which continually re-opens the file which is being
> written to. Since the file's in the middle of being written, it makes an RPC
> to the DataNode in order to determine the visible length of the file. This
> RPC is authenticated using the block token which came back in the
> LocatedBlocks object as the security ticket.
> When this RPC hits the IPC layer, it looks at its existing connections and
> sees none that can be re-used, since the block token differs between the two
> requesters. Hence, it reconnects, and we end up with hundreds or thousands of
> IPC connections to the datanode.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
