Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106229705
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -172,7 +172,7 @@ private[spark] class TaskSchedulerImpl
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106238544
--- Diff:
core/src/test/scala/org/apache/spark/util/collection/MedianHeapSuite.scala ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106237464
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -893,6 +893,7 @@ class TaskSetManagerSuite extends
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106237709
--- Diff:
core/src/test/scala/org/apache/spark/util/collection/MedianHeapSuite.scala ---
@@ -0,0 +1,82 @@
+/*
+ * Licensed to the Apache
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106236633
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/MedianHeap.scala ---
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106238777
--- Diff:
core/src/test/scala/org/apache/spark/util/collection/MedianHeapSuite.scala ---
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106235943
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/MedianHeap.scala ---
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106229967
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -141,6 +143,9 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r106237170
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/MedianHeap.scala ---
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17208
@liujianhuiouc do you have time to update the comment here? It would be
great to get this in soon.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15604#discussion_r106225480
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/SchedulableBuilder.scala ---
@@ -191,7 +191,10 @@ private[spark] class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15604#discussion_r106227249
--- Diff: core/src/test/scala/org/apache/spark/scheduler/PoolSuite.scala ---
@@ -201,12 +202,96 @@ class PoolSuite extends SparkFunSuite
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17166
The task killed messages should be informative, and I don't think we should
sacrifice informative messages just so they can be shown concisely in the stage
summary view. I think it's much
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17166
Why not just just "X killed" in the stage summary? It seems like overkill
to put the reasons for all of the killings there, now that I'm seeing the
screenshot, since they'
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/15604
@erenavsarogullari please file a JIRA when you see test failures instead of
ignoring them. I updated https://issues.apache.org/jira/browse/SPARK-19803 for
the first failure, but please file
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/15604
Jenkins retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r106063119
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -710,7 +710,11 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r106063061
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -239,8 +244,9 @@ private[spark] class Executor
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17139
Can you also post the time differences for some smaller jobs (e.g., 100
tasks, 1000 tasks, 10K tasks) to get a sense of how this varies with size?
---
If your project is set up for it, you
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17208
Looks good. Expanding on Imran's comment, how about:
We must set the taskset to zombie before the DAGScheduler handles the
taskEnded event. It's possible the taskEnded event
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17139
Why is the time improvement so much larger here than in the other PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104778912
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends
Logging
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104778628
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -467,7 +474,7 @@ private[spark] class TaskSchedulerImpl
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/12524
@seayi -- can you close this PR, since it's a duplicate of #16620?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17144
Ok this LGTM and I merged to master.
I tested this a bunch because in theory, it seems like the check that the
block has been properly re-replicated should / could happen inside
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17144#discussion_r104602172
--- Diff:
core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala
---
@@ -494,7 +494,9 @@ class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17144#discussion_r104602103
--- Diff:
core/src/test/scala/org/apache/spark/storage/BlockManagerReplicationSuite.scala
---
@@ -494,7 +494,9 @@ class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104600104
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala ---
@@ -64,7 +64,7 @@ private[spark] object UIData {
var
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104599195
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskScheduler.scala ---
@@ -54,6 +54,9 @@ private[spark] trait TaskScheduler
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104598647
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -239,8 +244,9 @@ private[spark] class Executor
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104599383
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends
Logging
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104599319
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends
Logging
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104600582
--- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala ---
@@ -538,10 +538,37 @@ class SparkContextSuite extends SparkFunSuite
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104600689
--- Diff: core/src/test/scala/org/apache/spark/SparkContextSuite.scala ---
@@ -538,10 +538,37 @@ class SparkContextSuite extends SparkFunSuite
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104599420
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends
Logging
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104599899
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -710,7 +710,11 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104598515
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/executor/MesosExecutorBackend.scala
---
@@ -104,7 +104,8 @@ private[spark] class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104598293
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ---
@@ -467,7 +474,7 @@ private[spark] class TaskSchedulerImpl
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17166#discussion_r104598221
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -2250,6 +2250,25 @@ class SparkContext(config: SparkConf) extends
Logging
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r104519160
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/MedianHeap.scala ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r104517879
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -740,6 +743,7 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r104515761
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/MedianHeap.scala ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r104515525
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -740,6 +743,7 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r104514689
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -740,6 +743,7 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17140
@squito sounds good. Thanks for fixing this!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r104508618
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -740,6 +743,7 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r104508178
--- Diff:
core/src/main/scala/org/apache/spark/util/collection/MedianHeap.scala ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17140
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17144
Also @uncleGen would you mind filing a JIRA for the second failed test case?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17144
cc @shubhamchopra who wrote the original code and @joshrosen who did the
main review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17144
I'm not really the right person to review this code, but that being said,
I'm not crazy about this fix, because 1s is kind of a long time to consistently
wait. It's better for tests
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16867
Also, thanks for doing the timing measurements!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16867
I'm a little on the fence about this because of the added complexity, but
it does seem to be a significant time improvement. Did you consider
implementing this as a median heap (see the last
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/15505
@witgo I don't think the ~1.5% improvement in runtime merits the added
complexity of this change. I could be convinced to merge this if it simplified
the code or the ability to reason about
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16639
I merged this into master. Thanks @squito!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17140#discussion_r104063441
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala ---
@@ -86,7 +86,10 @@ private[spark] object TaskDescription
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16959
LGTM -- I merged this into master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17140#discussion_r104018967
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala ---
@@ -86,7 +86,10 @@ private[spark] object TaskDescription
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17140#discussion_r104016148
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala ---
@@ -86,7 +86,10 @@ private[spark] object TaskDescription
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17140#discussion_r104015455
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala ---
@@ -86,7 +86,10 @@ private[spark] object TaskDescription
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/10506
OK fine to leave this open then (I don't have the time or expertise to
review this unfortunately)
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/10506
Is anyone still working on this and if not, can you close the PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16855
@GavinGavinNo1 are you ZhengYaofeng on JIRA? I want to correctly give you
credit on JIRA for fixing this.
---
If your project is set up for it, you can reply to this email and have your
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16855
LGTM thanks for fixing this! I've merged this into master.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16959#discussion_r103854313
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala ---
@@ -158,13 +158,17 @@ private[spark] class
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17111
Thanks for fixing this! I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17088
Can you please file a JIRA for the flaky jenkins failure?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/15505
@witgo OK I'll hold off on doing another pass on the code until you have
the test results.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103565779
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -621,6 +615,80 @@ class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103563609
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -993,6 +993,12 @@ class DAGScheduler
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103569795
--- Diff: core/src/test/scala/org/apache/spark/executor/ExecutorSuite.scala
---
@@ -164,17 +164,18 @@ class ExecutorSuite extends SparkFunSuite
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103147391
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala ---
@@ -130,7 +152,7 @@ private[spark] object TaskDescription
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103566378
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -621,6 +615,80 @@ class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103566031
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -621,6 +615,80 @@ class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103569761
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/local/LocalSchedulerBackend.scala
---
@@ -82,9 +88,15 @@ private[spark] class LocalEndpoint
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103564565
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
---
@@ -195,6 +197,11 @@ class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103564111
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -454,33 +452,15 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17088
Why is this a no-op when the shuffle service isn't enabled? It looks like
you mark the slave as lost in all cases?
---
If your project is set up for it, you can reply to this email and have
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/17088
Can you update the JIRA and PR description to say "un-register the output
locations" (or similar) instead of "remove the files"? The current description
is misl
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/17088#discussion_r103541395
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1331,7 +1332,7 @@ class DAGScheduler
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16867
LGTM and @squito's JIRA re-reorging sounds perfect
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16855#discussion_r103538649
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -664,6 +665,63 @@ class TaskSetManagerSuite extends
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16639
Jenkins retest this please (filed
https://issues.apache.org/jira/browse/SPARK-19772)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16867#discussion_r103390199
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ---
@@ -911,14 +916,14 @@ private[spark] class TaskSetManager
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16959#discussion_r103354382
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala ---
@@ -111,13 +115,13 @@ private[spark] class
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16959#discussion_r103348105
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala ---
@@ -48,25 +48,29 @@ private[spark] class
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16867
This looks like a real test failure resulting from this change
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16639#discussion_r103289664
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -400,8 +410,16 @@ private[spark] class Executor
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/15505#discussion_r103147221
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/TaskDescription.scala ---
@@ -23,7 +23,10 @@ import java.util.Properties
import
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16930
I merged this into master
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16892
I merged this into master -- thanks for the review @squito.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16892#discussion_r103017616
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ---
@@ -2031,6 +2051,11 @@ class DAGSchedulerSuite extends
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16639#discussion_r103014993
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -400,8 +410,16 @@ private[spark] class Executor
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16639#discussion_r103015714
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/FetchFailedException.scala ---
@@ -45,6 +50,12 @@ private[spark] class FetchFailedException
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16930
LGTM assuming tests pass
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/15505
@witgo what's the status of this? I'd like to get this merged and am happy
to take this over if you don't have time to work on it.
---
If your project is set up for it, you can reply
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16930#discussion_r102863614
--- Diff: core/src/test/scala/org/apache/spark/executor/ExecutorSuite.scala
---
@@ -133,6 +122,79 @@ class ExecutorSuite extends SparkFunSuite
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16855#discussion_r102860963
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -664,6 +665,55 @@ class TaskSetManagerSuite extends
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/16855#discussion_r102848520
--- Diff:
core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala ---
@@ -664,6 +665,63 @@ class TaskSetManagerSuite extends
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/12524
Can you update the PR description here to have the JIRA number
(SPARK-14658), not the PR number?
---
If your project is set up for it, you can reply to this email and have your
reply appear
101 - 200 of 1796 matches
Mail list logo
