GitHub user zsxwing reopened a pull request:
https://github.com/apache/spark/pull/6294
[SPARK-5681][Streaming] Use a lock to eliminate the race condition when
stopping receivers and registering receivers happen at the same time
This is an alternative way to fix `SPARK-5681`. It minimizes the changes.
Closes #4467
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark pr4467
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6294.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6294
----
commit 77983f35c634083d5a0398bd86f6f594221bba7a
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-02-09T07:43:18Z
Add tracker status and stop to receive messages when stopping tracker.
commit ae0d9fd08caac2a602d3809998c78d1045172313
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-02-11T04:23:35Z
Merge branch 'master' into tracker_status_timeout
commit 3d568e8ecac5ef53a96ae463e496cf51813eedfd
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-02-11T04:26:40Z
Let receivers get registered first before going started.
commit 355f9ce34cb0d01048b15c96d5028fb291604286
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-02-11T06:19:43Z
Separate register and start events for receivers.
commit 9e1a760f8920aafab6aa5fc42d4ffda5fb1c3e5e
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-04-07T07:29:28Z
Merge remote-tracking branch 'upstream/master' into tracker_status_timeout
Conflicts:
streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisor.scala
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
commit c419677c97d3c9cbe87540353c8df196fde7ecc0
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-04-07T07:40:25Z
Fix style.
commit 34c18dc5a21107cf40d837fa0017aa1c339dc5c9
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-05-05T10:16:51Z
Merge remote-tracking branch 'upstream/master' into tracker_status_timeout
Conflicts:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
commit 19b76d954378dcddfa751ffa19ec4edd61769f6c
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-05-10T14:53:24Z
Remove timeout.
commit e0ef72a862c1badcd225df634e25822177b433d9
Author: Liang-Chi Hsieh <[email protected]>
Date: 2015-05-10T14:53:51Z
Merge remote-tracking branch 'upstream/master' into tracker_status_timeout
commit fff63f963d57c7b2a3fc9fd05fbd67eeab6171f0
Author: zsxwing <[email protected]>
Date: 2015-05-20T19:54:44Z
Use a lock to eliminate the race condition when stopping receivers and
registering receivers happen at the same time.
commit 15ed4a13128fb8205a567828f4010c323b0b01db
Author: zsxwing <[email protected]>
Date: 2015-05-21T21:35:24Z
Register before starting the receiver
commit 7b1d9af13d80220559978956f5654a0e61985e92
Author: zsxwing <[email protected]>
Date: 2015-05-22T18:32:34Z
"case Throwable" => "case NonFatal"
commit a8120c019bfa4b72261100a2620f9a0ece6d47d2
Author: zsxwing <[email protected]>
Date: 2015-05-22T18:41:21Z
Merge branch 'master' into pr4467
commit 7c73c1fdc5ff93823c4256830d6bbcb2a1815d4f
Author: zsxwing <[email protected]>
Date: 2015-05-22T20:00:55Z
Use trackerStateLock to protect trackerState
commit c41ee9464b0a7bbe288bb1ec70df3368f75bfe99
Author: zsxwing <[email protected]>
Date: 2015-05-22T20:06:50Z
Make stopReceivers private
commit b4c29e7a8254db44abc825b85b784698d24404fe
Author: zsxwing <[email protected]>
Date: 2015-05-25T08:59:20Z
Stop receiver only if we start it
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
