Github user viirya closed the pull request at:
https://github.com/apache/spark/pull/4467
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r30020503
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -81,12 +112,14 @@ class ReceiverTracker(ssc:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-100667831
[Test build #32342 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32342/consoleFull)
for PR 4467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-100667838
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-100667839
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-100652963
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-100652983
[Test build #32342 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32342/consoleFull)
for PR 4467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-100652970
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-99064341
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-99064338
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-99064323
[Test build #31869 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31869/consoleFull)
for PR 4467 at commit
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r29704468
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -81,12 +112,14 @@ class ReceiverTracker(ssc:
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r29697767
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -247,9 +310,12 @@ class ReceiverTracker(ssc:
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r29699606
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -81,12 +112,14 @@ class ReceiverTracker(ssc:
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r29654705
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -203,8 +256,17 @@ class ReceiverTracker(ssc:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-99022596
[Test build #31869 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31869/consoleFull)
for PR 4467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-99021654
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-99021693
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r29630952
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -203,8 +256,17 @@ class ReceiverTracker(ssc:
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-98874818
Could you resolve the conflicts?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r29631826
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -203,8 +256,17 @@ class ReceiverTracker(ssc:
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90423241
[Test build #29785 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29785/consoleFull)
for PR 4467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90424094
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90424077
[Test build #29785 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29785/consoleFull)
for PR 4467 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90431533
[Test build #29786 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29786/consoleFull)
for PR 4467 at commit
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90414046
Currently, `Receiver.onStart()` is called before it sends
`RegisterReceiver` message to `ReceiverSupervisor` for registration.
In `Receiver.onStart()`, it already
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90476820
[Test build #29786 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29786/consoleFull)
for PR 4467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-90476829
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-89207216
@viirya Sorry for slacking on this, been busy. I think understand your
explanation. But I also spent some more time thinking about this ground up.
Correct me if I
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-84933187
/cc @tdas Still busy?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-82210437
@tdas Any updated ideas or comments?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-77659282
Hey ... lets continue the discussion. I took a quick look at the logic,
sounds good. Let me think a bit more and look at the code.
---
If your project is set up for it,
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-74280513
@tdas, Do you have time to take a look of the analysis and the current
implementation?
---
If your project is set up for it, you can reply to this email and have your
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-74314807
Sorry I am bit tied with stuff. I will definitely take a look as soon as i
get a chance. :)
On Fri, Feb 13, 2015 at 8:24 AM, Liang-Chi Hsieh
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73963794
I dont think that correct. If the state becomes Stopping before a receiver
has registered, it may so happen that the receiver starts up and receives
some data and
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-74010907
Let's analyze it clearly. The following is a simplified status
transformation of the problem:
tracker receivers
t = 1 started
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r24468323
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -202,16 +232,21 @@ class ReceiverTracker(ssc:
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r24468104
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -67,13 +67,39 @@ class ReceiverTracker(ssc:
Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4467#discussion_r24468119
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala
---
@@ -67,13 +67,39 @@ class ReceiverTracker(ssc:
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73831487
The state behavior should be:
* If state = Initialized, then disallow all actions. (actor is not
initialized yet)
* If state = Started, then allow all
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73832125
It should accept addblock even it is stopping because there might be
receivers processing data.
Modified state behavior should be:
* If state =
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73824469
I realized that this is a tricky thing to fix while maintaining the
stopGracefully semantics. Stop gracefully must ensure that if there are
receivers that have already
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73833008
[Test build #27273 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27273/consoleFull)
for PR 4467 at commit
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73737830
@tdas Please take a look of this when you have time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73835874
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73835869
[Test build #27273 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27273/consoleFull)
for PR 4467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73843764
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73843759
[Test build #27277 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27277/consoleFull)
for PR 4467 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73838853
[Test build #27277 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27277/consoleFull)
for PR 4467 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73476913
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73476905
[Test build #27091 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27091/consoleFull)
for PR 4467 at commit
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/4467
[SPARK-5681][Streaming] Add tracker status and stop to receive messages
when stopping tracker
Related to #4364.
Sometimes the receiver will be registered into tracker after `ssc.stop()`
is
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73469002
cc @tdas.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4467#issuecomment-73469302
[Test build #27091 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/27091/consoleFull)
for PR 4467 at commit
54 matches
Mail list logo
