[
https://issues.apache.org/jira/browse/HADOOP-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12566250#action_12566250
]
Raghu Angadi commented on HADOOP-2346:
--------------------------------------
Java Selector creation seems pretty heavy for a single filedescriptor polling.
The current patch creates one selector for each stream. Looks like each
selector takes up 3 fds: 2 for a pipe (used for {{wakeup()}, I guess) and for
epoll().
Even if we are ok with these extra fds for each stream, it still requires users
to close the stream explicitly (to close the selector). Opening and closing new
selector each time we wait might be costly be because of this extra pipe it
uses.
If these streams aim to be drop in replacements for Socket's own streams, they
should either clean up automatically when the channel is closed, or better,
should not require a {{close()}}.
Anyone aware of a bare-bones selector or an alternate to a selector? Some thing
that is a thin wrapper over {{poll()}}, without extra set up and extra fds for
pipe and epoll?
I think it is still possible to implement these streams with a single global
Selector, without an extra thread. Registering and deregistering a channel each
time we want to wait won't be costly. This way, stream does not need a close().
> DataNode should have timeout on socket writes.
> ----------------------------------------------
>
> Key: HADOOP-2346
> URL: https://issues.apache.org/jira/browse/HADOOP-2346
> Project: Hadoop Core
> Issue Type: Bug
> Components: dfs
> Affects Versions: 0.15.1
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Fix For: 0.16.1
>
> Attachments: HADOOP-2346.patch, HADOOP-2346.patch, HADOOP-2346.patch,
> HADOOP-2346.patch
>
>
> If a client opens a file and stops reading in the middle, DataNode thread
> writing the data could be stuck forever. For DataNode sockets we set read
> timeout but not write timeout. I think we should add a write(data, timeout)
> method in IOUtils that assumes it the underlying FileChannel is non-blocking.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
