Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122356119
LGTM2, but it would be great to have some unit tests on this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122035925
What happens if my accumulator update is huge? Could this lead to dropped
heartbeats?
---
If your project is set up for it, you can reply to this email and have
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122035997
Ah, I see now that it's only for specific accumulators and is internal-only.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122041450
[Test build #37510 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37510/console)
for PR 7448 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122005849
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122005883
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122007344
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122007293
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122009320
[Test build #37510 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37510/consoleFull)
for PR 7448 at commit
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/7448
[SPARK-8857][SPARK-8859][Core]Add an internal flag to Accumulable and send
internal accumulator updates to the driver via heartbeats
This PR includes the following changes:
1. Remove the
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122006903
[Test build #37509 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37509/consoleFull)
for PR 7448 at commit
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7448#discussion_r34837552
--- Diff: core/src/main/scala/org/apache/spark/TaskContext.scala ---
@@ -152,4 +152,10 @@ abstract class TaskContext extends Serializable {
* Returns the
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7448#discussion_r34837468
--- Diff: core/src/main/scala/org/apache/spark/TaskContext.scala ---
@@ -152,4 +152,10 @@ abstract class TaskContext extends Serializable {
* Returns the
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122103259
I took a look at accumulator implementation -- one thing that is really
annoying is boxing. Right now every update will box -- and it will really sucks
for SQL if we do
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122039945
[Test build #37509 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37509/console)
for PR 7448 at commit
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7448#discussion_r34828801
--- Diff: core/src/main/scala/org/apache/spark/Accumulators.scala ---
@@ -42,22 +43,37 @@ import org.apache.spark.util.Utils
* @tparam R the full
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122101767
cc @andrewor14 for review also
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122039989
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user rxin commented on a diff in the pull request:
https://github.com/apache/spark/pull/7448#discussion_r34838220
--- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala ---
@@ -52,7 +54,7 @@ private[spark] abstract class Task[T](val stageId: Int,
var
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122041601
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122147482
LGTM.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122166223
Thanks - I've merged this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/7448
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122159604
[Test build #37559 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37559/console)
for PR 7448 at commit
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122159661
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122143068
[Test build #37559 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37559/consoleFull)
for PR 7448 at commit
Github user zsxwing commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122142910
Added comments.
In order for SQL to work better, I think we'd need to create a subtype of
accumulator called LongAccumulator and DoubleAccumulator, that
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122142425
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/7448#issuecomment-122142419
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
29 matches
Mail list logo
