[
https://issues.apache.org/jira/browse/HADOOP-3067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12581214#action_12581214
]
rangadi edited comment on HADOOP-3067 at 3/21/08 4:07 PM:
---------------------------------------------------------------
Attached patch closes the sockets opened in {{fetchBlockByteRange()}}. Closing
'reader' is strictly not required but probably a good thing.
I think this patch uses IOUtils.closeStream() appropriately, let me know if you
see other wise.
Regd unit test : TestCrcCorruption fails without this patch and passes with it
when the the process is restricted to 1024 file descriptors. We could make
hudson restrict the fds to 1024 as well. Otherwise it does not seem straight
fwd to make sure all the sockets opened any where in Hadoop are closed when the
test finishes..
was (Author: rangadi):
Attached patch closes the sockets opened in {{fetchBlockByteRange()}}. Closing
'reader' is strictly not required but probably a good thing.
I think this patch uses IOUtils.closeStream() appropriately, let me know it see
other wise.
Regd unit test : TestCrcCorruption fails without this patch and passes with it
when the the process is restricted to 1024 file descriptors. We could make
hudson restrict the fds to 1024 as well. Otherwise it does not seem straight
fwd to make sure all the sockets opened any where in Hadoop are closed when the
test finishes..
> DFSInputStream 'pread' does not close its sockets
> -------------------------------------------------
>
> Key: HADOOP-3067
> URL: https://issues.apache.org/jira/browse/HADOOP-3067
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.17.0
>
> Attachments: HADOOP-3067.patch
>
>
> {{DFSInputStream.read(int, buffer)}} does not close the sockets it opens.
> Main reason this problem did not show up till now is that pread interface is
> not used much.
> TestCrcCorruption failure first reported in HADOOP-2902 is caused by this
> bug. Hadoop 0.17 uses more file descriptors for each thread waiting on socket
> io. Since client does not close sockets, it leaves a lot of DataNode threads
> waiting in the unit test.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
