[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-27 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r51061122 --- Diff: project/MimaExcludes.scala --- @@ -145,6 +145,15 @@ object MimaExcludes { // SPARK-12510 Refactor ActorReceiver to support Java

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-27 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175893531 I was able to verify that the changes in `Accumulable` and `Accumulator` do not break compatibility. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-27 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r51062164 --- Diff: core/src/main/scala/org/apache/spark/InternalAccumulator.scala --- @@ -17,42 +17,193 @@ package org.apache.spark +import

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-27 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175892453 I have compiled a list of breaking `@DeveloperAPI` changes: ExceptionFailure: - changed: `apply`, `unapply`, `copy` - removed: old constructor -

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-27 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r51062999 --- Diff: core/src/main/scala/org/apache/spark/Accumulator.scala --- @@ -75,43 +84,65 @@ private[spark] object Accumulators extends Logging { *

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-27 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175804655 I'm going to merge this now and we'll deal with those other tasks in followup patches. Thanks! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10835 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808344 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1116,23 +1120,33 @@ class DAGScheduler( */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50810673 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/AllStagesResource.scala --- @@ -237,7 +237,8 @@ private[v1] object AllStagesResource {

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50810761 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -787,10 +851,13 @@ private[spark] object JsonProtocol { val

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808540 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -1606,6 +1574,25 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50809201 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -273,14 +273,13 @@ class JsonProtocolSuite extends SparkFunSuite {

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174918882 According to Reviewable I'm all caught up with the latest changes. This looks like it's in really good shape to me; I only had a few very minor comments upthread,

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808975 --- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala --- @@ -212,6 +212,8 @@ class HadoopRDD[K, V]( logInfo("Input split: " +

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174917738 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174917736 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50810568 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala --- @@ -141,6 +136,18 @@ private[spark] abstract class Task[T]( def

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808303 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1116,23 +1120,33 @@ class DAGScheduler( */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50810468 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1074,39 +1074,43 @@ class DAGScheduler( } }

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808661 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -691,10 +724,42 @@ private[spark] object JsonProtocol { def

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808701 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -284,11 +284,44 @@ private[spark] object JsonProtocol { }

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808645 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -284,11 +284,44 @@ private[spark] object JsonProtocol { }

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r5080 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -691,10 +724,42 @@ private[spark] object JsonProtocol { def

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808924 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -787,10 +851,13 @@ private[spark] object JsonProtocol { val

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175386657 **[Test build #50163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50163/consoleFull)** for PR 10835 at commit

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175387162 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175387145 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804424 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala --- @@ -82,7 +84,19 @@ private[spark] class TaskResultGetter(sparkEnv:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804451 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala --- @@ -82,7 +84,19 @@ private[spark] class TaskResultGetter(sparkEnv:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804432 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -289,8 +288,10 @@ private[sql] class SQLListener(conf:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805502 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805566 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala --- @@ -82,7 +84,19 @@ private[spark] class TaskResultGetter(sparkEnv:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50807297 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1116,23 +1120,33 @@ class DAGScheduler( */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50807272 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1116,23 +1120,33 @@ class DAGScheduler( */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804159 --- Diff: core/src/main/scala/org/apache/spark/scheduler/AccumulableInfo.scala --- @@ -22,44 +22,32 @@ import org.apache.spark.annotation.DeveloperApi

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804115 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala --- @@ -141,6 +136,18 @@ private[spark] abstract class Task[T]( def

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50807374 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1116,23 +1120,33 @@ class DAGScheduler( */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804859 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala --- @@ -182,5 +213,39 @@ class TaskResultGetterSuite extends

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805776 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50807020 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1074,39 +1074,43 @@ class DAGScheduler( } }

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808023 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1074,39 +1074,43 @@ class DAGScheduler( } }

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50808097 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -189,7 +189,8 @@ class DAGSchedulerSuite override def

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804324 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskResultGetter.scala --- @@ -35,8 +35,8 @@ private[spark] class TaskResultGetter(sparkEnv:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805368 --- Diff: core/src/main/scala/org/apache/spark/scheduler/Task.scala --- @@ -141,6 +136,18 @@ private[spark] abstract class Task[T]( def

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50807613 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1116,23 +1120,33 @@ class DAGScheduler( */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804348 --- Diff: core/src/main/scala/org/apache/spark/scheduler/AccumulableInfo.scala --- @@ -22,44 +22,32 @@ import org.apache.spark.annotation.DeveloperApi

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805932 --- Diff: core/src/main/scala/org/apache/spark/InternalAccumulator.scala --- @@ -17,42 +17,193 @@ package org.apache.spark +import

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50806915 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1074,39 +1074,43 @@ class DAGScheduler( } }

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50807963 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805220 --- Diff: core/src/test/scala/org/apache/spark/InternalAccumulatorSuite.scala --- @@ -0,0 +1,329 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804555 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala --- @@ -72,6 +78,31 @@ class ResultDeletingTaskResultGetter(sparkEnv:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805064 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805003 --- Diff: project/MimaExcludes.scala --- @@ -145,6 +145,15 @@ object MimaExcludes { // SPARK-12510 Refactor ActorReceiver to support Java

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805074 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/AllStagesResource.scala --- @@ -237,7 +237,8 @@ private[v1] object AllStagesResource {

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50806627 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -184,10 +185,14 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804692 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala --- @@ -72,6 +78,31 @@ class ResultDeletingTaskResultGetter(sparkEnv:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50804726 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/PartitionBatchPruningSuite.scala --- @@ -114,7 +132,7 @@ class

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805184 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805180 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805552 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805681 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50805637 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175113006 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175116745 **[Test build #50110 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50110/consoleFull)** for PR 10835 at commit

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50877549 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -273,14 +273,13 @@ class JsonProtocolSuite extends SparkFunSuite {

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175175672 **[Test build #50110 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50110/consoleFull)** for PR 10835 at commit

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175175989 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175175988 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50884386 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1125,24 +1130,31 @@ class DAGScheduler(

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50891801 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -284,11 +284,44 @@ private[spark] object JsonProtocol { }

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50891740 --- Diff: core/src/main/scala/org/apache/spark/util/JsonProtocol.scala --- @@ -787,10 +851,13 @@ private[spark] object JsonProtocol { val

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50895675 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1125,24 +1130,31 @@ class DAGScheduler(

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50896327 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1125,24 +1130,31 @@ class DAGScheduler(

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50911176 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -273,14 +273,13 @@ class JsonProtocolSuite extends SparkFunSuite {

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175352386 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50934263 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1116,23 +1120,33 @@ class DAGScheduler( */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50934443 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -1606,6 +1574,25 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50934553 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -1606,6 +1574,25 @@ class DAGSchedulerSuite

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175354375 @JoshRosen I believe I addressed all of your comments. The latest commit should pass tests. There are a few things I would like to do on my end that shouldn't block

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-175356574 **[Test build #50163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50163/consoleFull)** for PR 10835 at commit

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-26 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50809333 --- Diff: core/src/test/scala/org/apache/spark/util/JsonProtocolSuite.scala --- @@ -273,14 +273,13 @@ class JsonProtocolSuite extends SparkFunSuite {

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50802605 --- Diff: core/src/test/scala/org/apache/spark/executor/TaskMetricsSuite.scala --- @@ -17,12 +17,543 @@ package org.apache.spark.executor

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50802114 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/AllStagesResource.scala --- @@ -237,7 +237,8 @@ private[v1] object AllStagesResource {

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50802781 --- Diff: core/src/main/scala/org/apache/spark/InternalAccumulator.scala --- @@ -17,42 +17,193 @@ package org.apache.spark +import

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174828808 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174828804 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50802278 --- Diff: core/src/test/scala/org/apache/spark/executor/TaskMetricsSuite.scala --- @@ -17,12 +17,345 @@ package org.apache.spark.executor

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174828051 **[Test build #50064 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50064/consoleFull)** for PR 10835 at commit

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50802530 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50803281 --- Diff: core/src/test/scala/org/apache/spark/InternalAccumulatorSuite.scala --- @@ -0,0 +1,329 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50803319 --- Diff: core/src/test/scala/org/apache/spark/InternalAccumulatorSuite.scala --- @@ -0,0 +1,329 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50802571 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -230,86 +297,119 @@ class TaskMetrics extends Serializable { */

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174730415 **[Test build #50033 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50033/consoleFull)** for PR 10835 at commit

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174758239 **[Test build #50033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50033/consoleFull)** for PR 10835 at commit

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174758685 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10835#issuecomment-174758681 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50785852 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLListener.scala --- @@ -289,8 +288,10 @@ private[sql] class SQLListener(conf:

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50786156 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -115,22 +118,34 @@ case class ExceptionFailure( description: String,

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50786113 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -115,22 +118,34 @@ case class ExceptionFailure( description: String,

[GitHub] spark pull request: [SPARK-12895][SPARK-12896] Migrate TaskMetrics...

2016-01-25 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10835#discussion_r50786403 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/PartitionBatchPruningSuite.scala --- @@ -114,7 +132,7 @@ class

  1   2   >