[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2018-02-14 Thread squito
Github user squito commented on the issue:

https://github.com/apache/spark/pull/17208
  
hmm I think you're right @zsxwing that we should be updating `isZombie` 
before `sched.dagScheduler.taskEnded` and `sched.dagScheduler.taskSetFailed` is 
called, just to keep state consistent.  I don't think you'll actually hit the 
bug described here, as (a) if it was from a fetch failure, `isZombie` is 
already set first or if (b) its just a regular task failure, and it leads to 
the stage getting aborted, then there aren't any more retries of the stage 
anyway.  


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2018-02-14 Thread zsxwing
Github user zsxwing commented on the issue:

https://github.com/apache/spark/pull/17208
  
I think `handleFailedTask` has the similar issue. Right?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread kayousterhout
Github user kayousterhout commented on the issue:

https://github.com/apache/spark/pull/17208
  
LGTM merged this to master


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75312/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/17208
  
**[Test build #75312 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75312/testReport)**
 for PR 17208 at commit 
[`fd67392`](https://github.com/apache/spark/commit/fd6739255df23c54e7c19080d73eeeca389754b4).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/17208
  
**[Test build #75312 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75312/testReport)**
 for PR 17208 at commit 
[`fd67392`](https://github.com/apache/spark/commit/fd6739255df23c54e7c19080d73eeeca389754b4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread squito
Github user squito commented on the issue:

https://github.com/apache/spark/pull/17208
  
Looks like the tests were manually killed (-9).

Thanks for catching that and fixing @liujianhuiouc 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread squito
Github user squito commented on the issue:

https://github.com/apache/spark/pull/17208
  
Jenkins, retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread liujianhuiouc
Github user liujianhuiouc commented on the issue:

https://github.com/apache/spark/pull/17208
  
@squito update the no-args ManualClock constructor with initialized time


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75299/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-28 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/17208
  
**[Test build #75299 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75299/testReport)**
 for PR 17208 at commit 
[`fd67392`](https://github.com/apache/spark/commit/fd6739255df23c54e7c19080d73eeeca389754b4).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread liujianhuiouc
Github user liujianhuiouc commented on the issue:

https://github.com/apache/spark/pull/17208
  
@squito tests fails


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Merged build finished. Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75290/
Test FAILed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/17208
  
**[Test build #75290 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75290/testReport)**
 for PR 17208 at commit 
[`17acd55`](https://github.com/apache/spark/commit/17acd55f57498195360d4b5876047c92005c3a68).
 * This patch **fails Spark unit tests**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread squito
Github user squito commented on the issue:

https://github.com/apache/spark/pull/17208
  
lgtm assuming tests pass


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/17208
  
**[Test build #75290 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75290/testReport)**
 for PR 17208 at commit 
[`17acd55`](https://github.com/apache/spark/commit/17acd55f57498195360d4b5876047c92005c3a68).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread liujianhuiouc
Github user liujianhuiouc commented on the issue:

https://github.com/apache/spark/pull/17208
  
@kayousterhout Done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread liujianhuiouc
Github user liujianhuiouc commented on the issue:

https://github.com/apache/spark/pull/17208
  
@kayousterhout Done


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread kayousterhout
Github user kayousterhout commented on the issue:

https://github.com/apache/spark/pull/17208
  
Yes can you also merge @squito's test case?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-27 Thread liujianhuiouc
Github user liujianhuiouc commented on the issue:

https://github.com/apache/spark/pull/17208
  
@kayousterhout I have already update the comments, and fix this issue, do 
you mean i should merge the test case by squito


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-23 Thread kayousterhout
Github user kayousterhout commented on the issue:

https://github.com/apache/spark/pull/17208
  
@liujianhuiouc have you had time to fix this up yet?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-20 Thread liujianhuiouc
Github user liujianhuiouc commented on the issue:

https://github.com/apache/spark/pull/17208
  
ok, I will  update that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-15 Thread squito
Github user squito commented on the issue:

https://github.com/apache/spark/pull/17208
  
to be clear, I agree with Kay's rewording (in particular, I meant stage 
attempt, not task attempt).

Also I think its worth including a test.  You can use this: 
https://github.com/squito/spark/commit/3da535f5664746f57928c693a45cdc1ffdce5311

I know its very narrowly focused but it seems worth including.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-15 Thread kayousterhout
Github user kayousterhout commented on the issue:

https://github.com/apache/spark/pull/17208
  
@liujianhuiouc do you have time to update the comment here?  It would be 
great to get this in soon.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-09 Thread jinxing64
Github user jinxing64 commented on the issue:

https://github.com/apache/spark/pull/17208
  
@squito 
Thanks for notification :) this is not in my pr.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread kayousterhout
Github user kayousterhout commented on the issue:

https://github.com/apache/spark/pull/17208
  
Looks good.  Expanding on Imran's comment, how about:

We must set the taskset to zombie before the DAGScheduler handles the 
taskEnded event.  It's possible the taskEnded event will cause the DAGScheduler 
to launch a new stage attempt (this happens when map output data was lost), and 
if this happens before the taskSet has been set to zombie, it will appear that 
we have conflicting task sets.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Test PASSed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74212/
Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Merged build finished. Test PASSed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/17208
  
**[Test build #74212 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74212/testReport)**
 for PR 17208 at commit 
[`6c40b9f`](https://github.com/apache/spark/commit/6c40b9f3d8b5935d5930f17b61ee11384cf2b011).
 * This patch passes all tests.
 * This patch merges cleanly.
 * This patch adds no public classes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread squito
Github user squito commented on the issue:

https://github.com/apache/spark/pull/17208
  
This looks like the right change.  In fact, I could have sworn we had 
recently merged in something like this -- maybe there is another pr still in 
flight which includes this?  @jinxing64 perhaps this is in one of your open prs?

The description needs to be updated, and we really should have a unit test 
(though with a very quick look I don't see a good way to test, I'll need to 
think about that part).  Here is my suggestion for the description:

> We must set the taskset to zombie before the dagscheduler handles the 
`taskEnded` event, because that event may cause the dagscheduler to launch 
another task attempt.  If that happens before the taskSet has been set to 
zombie, it will appear that we have conflicting task sets.

> The code worked before this change because dagScheduler.taskEnded() is 
async, so the task ended was almost always processed after the zombie status 
had been updated.  However, that left a race, which would occasionally go the 
wrong way.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread SparkQA
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/17208
  
**[Test build #74212 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/74212/testReport)**
 for PR 17208 at commit 
[`6c40b9f`](https://github.com/apache/spark/commit/6c40b9f3d8b5935d5930f17b61ee11384cf2b011).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread squito
Github user squito commented on the issue:

https://github.com/apache/spark/pull/17208
  
Jenkins, ok to test


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread srowen
Github user srowen commented on the issue:

https://github.com/apache/spark/pull/17208
  
CC @kayousterhout or @squito 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] spark issue #17208: [SPARK-19868] conflict TasksetManager lead to spark stop...

2017-03-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/17208
  
Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org