[
https://issues.apache.org/jira/browse/HADOOP-2232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12561595#action_12561595
]
dhruba borthakur commented on HADOOP-2232:
------------------------------------------
The exception messages are related to HADOOP-1707. This JIRA changed the error
recovery model. Earlier, the client used to cache the entire disk block. When
it is full it uploads the entire block to a pipeline of datanodes. If the
upload to the first datanode succeeded, the operation was deemed as successful.
In the new model, the client will upload the block to all datanodes in the
pipeline. In the case of error, the client establishes a new pipeline (by
removing the bad datanode(s) from the pipeline) and resending outstanding data
for this block. This change means that a client is now *more* likely to detect
failures of datanodes.
My question then is: do you see any of these exceptions when you run your test
on trunk without the patch for this JIRA?
> Add option to disable nagles algorithm in the IPC Server
> --------------------------------------------------------
>
> Key: HADOOP-2232
> URL: https://issues.apache.org/jira/browse/HADOOP-2232
> Project: Hadoop
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.16.0
> Reporter: Clint Morgan
> Assignee: Clint Morgan
> Attachments: 2232-3.patch, HADOOP-2232-1.patch, HADOOP-2232-2.patch
>
>
> While investigating hbase performance, I found a bottleneck caused by
> Nagles algorithm. For some reads I would get a bi-modal distribution
> of read times, with about half the times being around 20ms, and half
> around 200ms. I tracked this down to the well-known interaction between
> Nagle's algorithm and TCP delayed acknowledgments.
> I found that calling setTcpNoDelay(true) on the server's socket
> connection dropped all of my read times back to a constant 20 ms.
> I propose a patch to have this TCP_NODELAY option be configurable. The
> attacked patch allows one to set the TCP_NODELAY option on both the
> client and the server side. Currently this is defaulted to false
> (i.e., with Nagle's enabled).
> To see the effect, I have included a Test which provokes the issue by
> sending a MapWriteable over an IPC call. On my machine this test shows
> a speedup of 117 times when using TCP_NODELAY.
> These tests were done on OSX 10.4. Your milage may very with other
> TCP/IP implementation stacks.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
