[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-19 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16261 Merged this to master. Thanks @shivaram! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-19 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/16261 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70275/ Test PASSed. ---

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16261 **[Test build #70275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70275/testReport)** for PR 16261 at commit [`c8dbd75`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70267/ Test FAILed. ---

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16261 **[Test build #70267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70267/testReport)** for PR 16261 at commit [`3a76605`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16261 **[Test build #70275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70275/testReport)** for PR 16261 at commit [`c8dbd75`](https://github.com/apache/spark/commit/c8

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16261 **[Test build #70267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70267/testReport)** for PR 16261 at commit [`3a76605`](https://github.com/apache/spark/commit/3a

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16261 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70239/ Test FAILed. ---

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-16 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16261 **[Test build #70239 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70239/testReport)** for PR 16261 at commit [`3a76605`](https://github.com/apache/spark/commit/3a

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16261 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wi

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-15 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16261 With the help of @kayousterhout I ran a scheduling microbenchmark (Code [1]) with 1 tasks per stage on 20 m2.4xlarge machines on EC2 (160 cores). From 10 trials, I measured the average time tak

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-12 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/16261 Woohoo! This change has been sitting on my laptop for a while but I haven't had the time to clean it up for submission. Thanks for picking this up. --- If your project is set up for it, you can r

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70046/ Test FAILed. ---

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-12 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/16261 This code was originally written by @JoshRosen. BTW while I put this up for review, I am going to run some performance tests on EC2 today / tomm to see exactly how much benefit this gives

[GitHub] spark issue #16261: [SPARK-18836] [CORE] Serialize one copy of task metrics ...

2016-12-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16261 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e