[
https://issues.apache.org/jira/browse/HADOOP-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580162#action_12580162
]
Hairong Kuang commented on HADOOP-2910:
---------------------------------------
> Note that removing connect timeout does not mean connect will wait as long as
> server is up.
Yes, I agree. It also depends on the number of the pending connections that
TCP/IP allows. If we do not want connect to fail when the server is up, then we
need a separate thread to accept connections that do not block when the call
queue is full. But I'd like to do it in a different jira.
> One thing I am not sure is if there is a possibility where connect() will
> hang forever..
I do not think so. Connect is more like send, so the client can detect if the
server is down or not. HADOOP-2188 handles the case that receive is not able to
detect when the server machine gets power down unexpectedly.
> Throttle IPC Client/Server during bursts of requests or server slowdown
> -----------------------------------------------------------------------
>
> Key: HADOOP-2910
> URL: https://issues.apache.org/jira/browse/HADOOP-2910
> Project: Hadoop Core
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.16.0
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.17.0
>
> Attachments: callQueue.patch, callQueue1.patch, callQueue2.patch
>
>
> I propose the following to avoid an IPC server being swarmed by too many
> requests and connections
> 1. Limit call queue length or limit the amount of memory used in the call
> queue. This can be done by including the size of a request in the header and
> storing unmarshaled requests in the call queue.
> 2. If the call queue is full or queue buffer is full, stop reading requests
> from sockets. So requests stay at the server's system buffer or at the client
> side and thus eventually throttle the client.
> 3. Limit the total number of connections. Do not accept new connections if
> the connection limit is exceeded. (Note: this solution is unfair to new
> connections.)
> 4. If receive out of memory exception, close the current connection.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
