GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/1758
[SPARK-2583] ConnectionManager error reporting
This patch modifies the ConnectionManager so that error messages are sent
in reply when uncaught exceptions occur during message processing. This
prevents message senders from hanging while waiting for an acknowledgment if
the remote message processing failed.
This is an updated version of @sarutak's PR, #1490. The main change is to
use Futures / Promises to signal errors.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark connection-manager-fixes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1758.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1758
----
commit e2b8c4a2595c3196821ffed582ceea487d0d65d4
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-19T01:01:35Z
Modify to propagete error using ConnectionManager
commit 66354672c5743994d89811be7ccc99e1ca161f81
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-19T20:16:16Z
Merge branch 'master' of git://git.apache.org/spark into SPARK-2583
commit 717c9c33d7a2c8fe091efeb397960a28cc355f04
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-20T08:25:04Z
Merge branch 'master' of git://git.apache.org/spark into SPARK-2583
commit 4117b8fb0da568da1c6c41a06bab8004ef4dc422
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-20T09:00:22Z
Modified ConnectionManager to be alble to handle error during processing
message
commit 12d3de84a4d9d6d47df92059bc41965723bf02f7
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-20T18:02:24Z
Added BlockFetcherIteratorSuite.scala
commit ffaa83dda26da5e0e251b66a952281d84bad4aaa
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-22T18:38:28Z
Merge branch 'master' of git://git.apache.org/spark into SPARK-2583
commit 06541286abe434d2daea37c26212ef7284515d9d
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-23T05:28:11Z
Merge branch 'master' of git://git.apache.org/spark into SPARK-2583
commit 281589c5018b0cadbebc6f8cfac8a831dd40edeb
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-23T05:55:35Z
Add a test case to BlockFetcherIteratorSuite.scala for fetching block from
remote from successfully
commit e579302e629e859dcfece60b01b82754ac05af93
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-23T18:14:52Z
Merge branch 'master' of git://git.apache.org/spark into SPARK-2583
commit 22d7ebde6d706e4336154af5bec87f36d1501b20
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-24T00:45:01Z
Add test cases to BlockManagerSuite for SPARK-2583
commit 2a18d6b734dda995b43ef9a9e54dac5452ba6000
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-24T03:58:29Z
Merge branch 'master' of git://git.apache.org/spark into SPARK-2583
commit 326a17f05e078f81486a9c64361b40eb991d5e01
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-24T07:21:42Z
Add test cases to ConnectionManagerSuite.scala for SPARK-2583
commit e7d9aa64eedb019d6faffd61f67707712555fa8a
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-28T11:08:24Z
rebase to master
commit 9dfd0d8de8129a23f4e0535f9d04ba345dfd9158
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-29T19:56:06Z
Merge branch 'master' of git://git.apache.org/spark into SPARK-2583
commit ee91bb793666b260c9018e532dcf32c7115b3194
Author: Kousuke Saruta <[email protected]>
Date: 2014-07-29T19:57:42Z
Modified BufferMessage.scala to keep the spark code style
commit 7399c6b12dda6a85caa1c8e89563e746379bd15a
Author: Josh Rosen <[email protected]>
Date: 2014-08-01T21:47:40Z
Merge remote-tracking branch 'origin/pr/1490' into connection-manager-fixes
Conflicts:
core/src/test/scala/org/apache/spark/storage/BlockFetcherIteratorSuite.scala
core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala
commit f1cd1bb103607adc9b8d6bbcaac9eea61a1696ec
Author: Josh Rosen <[email protected]>
Date: 2014-08-04T00:58:26Z
Clean up @sarutak's PR #1490 for [SPARK-2583]: ConnectionManager error
reporting
Use Futures to signal failures, rather than exposing empty messages to user
code.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
