Github user davies commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-102270390
@pwendell It's discussed here:
https://issues.apache.org/jira/browse/SPARK-7624
---
If your project is set up for it, you can reply to this email and have your
reply
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-102264575
This patch cause another problem [1], which is not trivial. Since
SPARK-4939 is resolved by #3779 , I will revert this.
[1]
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-102265530
@davies can you document the exact problem somewhere?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72782236
[Test build #576 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/576/consoleFull)
for PR 4147 at commit
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72775916
@kayousterhout done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72782830
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72782813
[Test build #26702 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26702/consoleFull)
for PR 4147 at commit
Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72777970
LGTM; I'll merge this as soon as tests pass. @tdas @pwendell this is fine
with me to merge into 1.2 (although I realize it won't make it until 1.2.2);
does that
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72783843
@kayousterhout sure - that seems fine to me!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72775964
[Test build #576 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/576/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72776322
[Test build #26702 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26702/consoleFull)
for PR 4147 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/4147
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/4147#discussion_r24056992
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala ---
@@ -74,10 +78,16 @@ private[spark] class LocalActor(
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/4147#discussion_r24057066
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala ---
@@ -74,10 +77,20 @@ private[spark] class LocalActor(
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/4147#discussion_r23978147
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala ---
@@ -74,10 +77,20 @@ private[spark] class LocalActor(
def
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72596416
[Test build #26610 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26610/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72596551
[Test build #26610 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26610/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72601967
[Test build #26619 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26619/consoleFull)
for PR 4147 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72608285
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72608276
[Test build #26619 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26619/consoleFull)
for PR 4147 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72596553
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72125553
[Test build #572 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/572/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72125698
[Test build #26345 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26345/consoleFull)
for PR 4147 at commit
Github user tdas commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72133897
@kayousterhout Can you take a look now, is it good to go for Spark 1.3?
---
If your project is set up for it, you can reply to this email and have your
reply appear on
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72131840
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72131832
[Test build #26345 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26345/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72124838
[Test build #570 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/570/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72124873
[Test build #571 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/571/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72125144
[Test build #571 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/571/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-72133490
[Test build #572 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/572/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70937940
[Test build #25924 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25924/consoleFull)
for PR 4147 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70940288
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/4147#discussion_r23341377
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala ---
@@ -74,10 +77,20 @@ private[spark] class LocalActor(
def
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70942190
@kayousterhout It works, but a little bit leggy, some stages will need 3
more seconds to finish.
---
If your project is set up for it, you can reply to this email and
Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70942407
Is the 3 seconds entirely due to the locality wait? Or can you make this
better by more aggressively reviving offers (e.g., start reviving each time a
task gets
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70947153
@kayousterhout After fix the locality wait, the streaming job run very
smoothly. I think both of them are useful.
---
If your project is set up for it, you can reply to
GitHub user davies opened a pull request:
https://github.com/apache/spark/pull/4147
[SPARK-4939] revive offers periodically in LocalBackend
The locality timeout assume that the SchedulerBackend can revive offers
periodically, but currently LocalBackend did do that, then some job
Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70937891
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70938099
[Test build #25924 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25924/consoleFull)
for PR 4147 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70938103
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70939442
Jenkins, test this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70939407
@kayousterhout so what is your thought on whether this should go into 1.2
or not, given the final version (/cc @tdas who asked about this).
---
If your project is set
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70936615
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70936613
[Test build #25922 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25922/consoleFull)
for PR 4147 at commit
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/4147#discussion_r23341164
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala ---
@@ -74,10 +77,20 @@ private[spark] class LocalActor(
Github user kayousterhout commented on a diff in the pull request:
https://github.com/apache/spark/pull/4147#discussion_r23341154
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/local/LocalBackend.scala ---
@@ -74,10 +77,20 @@ private[spark] class LocalActor(
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70936460
[Test build #25922 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25922/consoleFull)
for PR 4147 at commit
Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70940400
@pwendell that seems fine -- since this just modifies the local mode, it
seems quite low-risk.
@davies I assume you've run this with the offending python
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70940284
[Test build #25926 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25926/consoleFull)
for PR 4147 at commit
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70940151
[Test build #25926 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/25926/consoleFull)
for PR 4147 at commit
Github user kayousterhout commented on the pull request:
https://github.com/apache/spark/pull/4147#issuecomment-70942542
Ah NVM that won't help.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
51 matches
Mail list logo
