GitHub user aarondav opened a pull request:
https://github.com/apache/spark/pull/5584
[SPARK-7003] Improve reliability of connection failure detection between
Netty block transfer service endpoints
Currently we rely on the assumption that an exception will be raised and
the channel closed if two endpoints cannot communicate over a Netty TCP
channel. However, this guarantee does not hold in all network environments, and
[SPARK-6962](https://issues.apache.org/jira/browse/SPARK-6962) seems to point
to a case where only the server side of the connection detected a fault.
This patch improves robustness of fetch/rpc requests by having an explicit
timeout in the transport layer which closes the connection if there is a period
of inactivity while there are outstanding requests.
NB: This patch is actually only around 50 lines added if you exclude the
testing-related code.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aarondav/spark timeout
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5584.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5584
----
commit aa5278b5e558c5416caa165412f24364a6e1df78
Author: Aaron Davidson <[email protected]>
Date: 2015-04-20T02:46:15Z
[SPARK-7003] Improve reliability of connection failure detection between
Netty block transfer service endpoints
Currently we rely on the assumption that an exception will be raised and
the channel closed if two endpoints cannot communicate over a Netty TCP
channel. However, this guarantee does not hold in all network environments, and
[SPARK-6962](https://issues.apache.org/jira/browse/SPARK-6962) seems to point
to a case where only the server side of the connection detected a fault.
This patch improves robustness of fetch/rpc requests by having an explicit
timeout in the transport layer which closes the connection if there is a period
of inactivity while there are outstanding requests.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
