Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50857088
Alright, I'm going to merge this as is then. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/1076
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15629314
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -154,6 +154,8 @@ private[spark] class Master(
}
override
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50726663
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50726957
QA tests have started for PR 1076. This patch merges cleanly. brView
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17570/consoleFull
---
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50732054
QA results for PR 1076:br- This patch PASSES unit tests.br- This patch
merges cleanlybr- This patch adds no public classesbrbrFor more
information see test
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15670431
--- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala ---
@@ -121,6 +121,10 @@ private[spark] class Executor(
}
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15670767
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -990,6 +990,7 @@ class SparkContext(config: SparkConf) extends Logging {
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15670874
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -990,6 +990,7 @@ class SparkContext(config: SparkConf) extends Logging {
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50820863
Hi @rahulsinghaliitd, looks like we only ever `report()` when we also want
to stop the metrics system. I wonder if it makes sense for each of the sinks to
just do its
Github user rahulsinghaliitd commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50825980
Hi @andrewor14, you are right. But I thought this would be a nice addition
(although out of scope for this change). Using the new method applications
(possibly
Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50827072
I see, so it acts like a flush, then it's fine. It's just a little strange
to see `report()` but not `stop()` in Master/Worker/SparkContext's `stop()`
method
---
If
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15673816
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -154,6 +154,8 @@ private[spark] class Master(
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15673827
--- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
---
@@ -357,6 +357,7 @@ private[spark] class Worker(
}
Github user rahulsinghaliitd commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15675679
--- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
---
@@ -357,6 +357,7 @@ private[spark] class Worker(
}
Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15676118
--- Diff: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
---
@@ -357,6 +357,7 @@ private[spark] class Worker(
}
Github user rahulsinghaliitd commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15681558
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -154,6 +154,8 @@ private[spark] class Master(
}
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50690789
Jenkins, test this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15617668
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -91,6 +91,10 @@ private[spark] class MetricsSystem private (val
instance:
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50690937
Alright, that makes sense. Still seems like a good change. @pwendell
@andrewor14 any objection to this?
---
If your project is set up for it, you can reply to this email
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15617766
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -154,6 +154,8 @@ private[spark] class Master(
}
override
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50691087
QA tests have started for PR 1076. This patch merges cleanly. brView
progress:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/17499/consoleFull
---
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15617739
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -154,6 +154,8 @@ private[spark] class Master(
}
override
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50691352
QA results for PR 1076:br- This patch FAILED unit tests.br- This patch
merges cleanlybr- This patch adds no public classesbrbrFor more
information see test
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15618258
--- Diff: core/src/main/scala/org/apache/spark/metrics/sink/Sink.scala ---
@@ -20,4 +20,5 @@ package org.apache.spark.metrics.sink
private[spark] trait
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15618239
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -91,6 +91,10 @@ private[spark] class MetricsSystem private (val
instance:
Github user rahulsinghaliitd commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15626358
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -91,6 +91,10 @@ private[spark] class MetricsSystem private (val
Github user rahulsinghaliitd commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15626384
--- Diff: core/src/main/scala/org/apache/spark/metrics/sink/Sink.scala ---
@@ -20,4 +20,5 @@ package org.apache.spark.metrics.sink
Github user rahulsinghaliitd commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15626682
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -154,6 +154,8 @@ private[spark] class Master(
}
Github user rahulsinghaliitd commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50545834
Hi Matei, yes, the report() call is blocking. The time it takes will depend
on the underlying reporter. For e.g., the CsvReporter which updates files on
local
Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r15503634
--- Diff: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala
---
@@ -91,6 +91,10 @@ private[spark] class MetricsSystem private (val
instance:
Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-50428479
Hey Rahul, sorry for the long delay, but this seems like a good change. I
had one small comment on the style. Also one question: are the report() calls
blocking? If so,
GitHub user rahulsinghaliitd opened a pull request:
https://github.com/apache/spark/pull/1076
SPARK-2134: Report metrics before application finishes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Guavus/spark SPARK-2134
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1076#issuecomment-46001763
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user rahulsinghaliitd commented on a diff in the pull request:
https://github.com/apache/spark/pull/1076#discussion_r13750359
--- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala
---
@@ -154,6 +154,8 @@ private[spark] class Master(
}
35 matches
Mail list logo
