[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113414210 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113414080 @squito, can you help me to review the test again. 3ks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113414174 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113415556 [Test build #35247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35247/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113418472 [Test build #35248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35248/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113437368 [Test build #35247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35247/console) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113437398 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113438626 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113438608 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113416966 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113417006 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113432930 [Test build #35251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35251/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32811839 --- Diff: core/src/main/scala/org/apache/spark/MapOutputTracker.scala --- @@ -305,7 +305,7 @@ private[spark] class MapOutputTrackerMaster(conf: SparkConf)

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113438818 [Test build #35248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35248/console) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113438854 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113438222 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113439039 [Test build #35255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35255/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113432541 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113432599 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-19 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32811940 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1193,8 +1193,10 @@ class DAGScheduler( // TODO: This will

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113169097 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113169123 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113166545 @andrewor14 @srowen Already refine with the comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113166266 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113166319 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113166827 [Test build #35133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35133/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113169866 [Test build #35134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35134/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113167204 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113167200 [Test build #35133 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35133/console) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113191005 [Test build #35134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35134/console) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113191053 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32799824 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -598,6 +598,49 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113356815 @andrewor14 The rest failed, because I change the `DagScheduler.executorLost` logical DAGSCheduler.scala: Origin version: Change mapOutPutTracker epoch

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32800016 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -598,6 +598,49 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113284343 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113284614 [Test build #35169 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35169/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113285019 I think its actually a real test failure -- ignore late map task completions in DAGSchedulerSuite was failing for me when run locally as well. --- If your project is

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113283506 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32778875 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -598,6 +598,49 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113284364 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32778983 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -598,6 +598,49 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32781070 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -598,6 +598,49 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32781253 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -598,6 +598,49 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread squito
Github user squito commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113297759 Hi @suyanNone I'm still trying to understand this. Your high-level description of the problem makes sense, but I'm confused by the test case. I put in a few comments,

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113300261 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-113300218 [Test build #35169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/35169/console) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-06-17 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r32600352 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -65,4 +65,6 @@ private[spark] class ResultTask[T, U]( override def

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-04-16 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-93883077 @andrewor14 that problem occurs while have stage-retry. Our user had been meet that problem under having a Executor Lost because of killed by yarn or sth, while we

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-04-14 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-93013150 Hi @suyanNone do you have a small example application code that reproduces the hanging issue? @kayousterhout @markhamstra it would be good if you could take a look at

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-04-14 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r28356812 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -543,6 +543,53 @@ class DAGSchedulerSuite extends FunSuiteLike

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-04-14 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r28356730 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -543,6 +543,53 @@ class DAGSchedulerSuite extends FunSuiteLike

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-04-14 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-93014371 Hi @suyanNone do you have a small example application code that reproduces the hanging issue? @kayousterhout @markhamstra it would be good if you could take a look at

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-04-14 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r28356278 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -543,6 +543,53 @@ class DAGSchedulerSuite extends FunSuiteLike

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-03-19 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-83422662 This patch is forgotten by us... @srowen @markhamstra @kayousterhout this patch can prevent from endless retry which may occurs after a executor is killed or

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-03-05 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-77404911 @markhamstra do you have any thoughts on this one? would love to get another set of eyes on this for a yea or nay. I don't feel like I know this code well enough. ---

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-03-03 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76916713 @cloud-fan yeah I hear you although this method is defined in the superclass, not just made up for this PR. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-03-02 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76870332 I'm still a little against the `canEqual` method. In this particular context, I think (stageId, partitionId) is meaningful enough to identify a task. --- If your

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76462511 cc @markhamstra and @kayousterhout also --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-27 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76464316 I'll take a look over the weekend. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76362137 [Test build #28061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28061/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76375414 [Test build #28061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28061/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76375421 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76378618 @cloud-fan @rxin do you have any final thoughts on this? it's looking reasonable to me though I admit I don't know this scheduler code well enough to be confident. ---

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-26 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r25414968 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -483,8 +483,9 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r25416698 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -483,8 +483,9 @@ private[spark] class TaskSetManager( // a

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76321748 [Test build #28038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28038/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76326132 [Test build #28038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/28038/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-76326137 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-25 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r25333545 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -483,8 +483,9 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r25333773 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -483,8 +483,9 @@ private[spark] class TaskSetManager( // a

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r24806228 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -483,8 +483,9 @@ private[spark] class TaskSetManager( // a

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-17 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-74646530 @srowen @JoshRosen can some one verify this patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r24806252 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -65,4 +65,6 @@ private[spark] class ResultTask[T, U]( override def

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-02-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r24806282 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -65,4 +65,6 @@ private[spark] class ResultTask[T, U]( override def

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-30 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72167520 [Test build #26386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26386/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72167526 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71993494 [Test build #26307 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26307/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71993235 [Test build #26307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26307/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71993498 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72147882 jenkins retests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72148555 [Test build #26374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26374/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72148352 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72150515 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72150509 [Test build #26373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26373/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72150939 [Test build #26374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26374/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72150943 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72148180 [Test build #26373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26373/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72148232 @cloud-fan--! [error] /home/jenkins/workspace/SparkPullRequestBuilder/core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala:69: class ResultTask

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72162266 [Test build #26386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26386/consoleFull) for PR 4055 at commit

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r23827693 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -65,4 +65,6 @@ private[spark] class ResultTask[T, U]( override def

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-29 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-72163353 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-27 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r23592779 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -65,4 +65,6 @@ private[spark] class ResultTask[T, U]( override def

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-27 Thread suyanNone
Github user suyanNone commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r23592724 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala --- @@ -106,7 +106,21 @@ private[spark] abstract class Task[T](val stageId: Int, var

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-26 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71458822 Last week and current week, my work is not in Spark, so reply was slower --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-26 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71457733 @cloud-fan According current code, it may not easy to change to not re-submit task in pendingTasks. To be honest, current DAGScheduler is complicated but at some

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-26 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71460355 @suyanNone Thanks for the explanation of re-submit! What's the Chinese name of HarryZhang? We don't use English name in the lab…… --- If your project is set

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-26 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71458406 @cloud-fan btw, Do you know HarryZhang? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r23589337 --- Diff: core/src/main/scala/org/apache/spark/scheduler/ResultTask.scala --- @@ -65,4 +65,6 @@ private[spark] class ResultTask[T, U]( override def

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-26 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/4055#issuecomment-71579227 @cloud-fan ZhangLei, SunHongLiang, HanLi, ChenXingYu, blabla...I am ZhangLei's classmate in ZJU. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-5259][CORE]Make sure mapStage.pendingta...

2015-01-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/4055#discussion_r23589410 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -540,6 +540,52 @@ class DAGSchedulerSuite extends

  1   2   >