[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-02-05 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-180659073 @JoshRosen , do you have any further comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-02-05 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-180659170 /cc @cloud-fan @andrewor14 , did you guys see spill size > 0 when the UI was introduced? Can you take a look at this fix? --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-02-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-180659438 **[Test build #50857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50857/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-02-05 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-180660983 is it possible to write a test for this bug? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-180678331 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-02-05 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-180678084 **[Test build #50857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/50857/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-02-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-180678330 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174452085 **[Test build #49984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49984/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174452336 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174452340 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174469710 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174469712 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174469120 **[Test build #49983 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49983/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread carsonwang
Github user carsonwang commented on a diff in the pull request: https://github.com/apache/spark/pull/10634#discussion_r50648300 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java --- @@ -125,7 +125,8 @@ public UnsafeKVExternalSorter(

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174398084 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174398266 **[Test build #49974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49974/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174399126 **[Test build #49976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49976/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174424260 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174424261 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174417597 **[Test build #49974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49974/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174426598 **[Test build #49984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49984/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174421284 @JoshRosen , I now also update `diskBytesSpilled`. Previously it is not updated for aggregation. Please help review this. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread carsonwang
Github user carsonwang commented on a diff in the pull request: https://github.com/apache/spark/pull/10634#discussion_r50659285 --- Diff: core/src/main/java/org/apache/spark/util/collection/unsafe/sort/UnsafeExternalSorter.java --- @@ -202,6 +201,7 @@ public long spill(long size,

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174422121 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174424100 **[Test build #49976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49976/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174417708 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174417707 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-174425086 **[Test build #49983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/49983/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-24 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10634#discussion_r50641512 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java --- @@ -125,7 +125,8 @@ public UnsafeKVExternalSorter(

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-13 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/10634#discussion_r49656738 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeKVExternalSorter.java --- @@ -125,7 +125,8 @@ public UnsafeKVExternalSorter(

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169857794 **[Test build #48992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48992/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169628627 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169881254 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-07 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169881257 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-07 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169881129 **[Test build #48992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48992/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169586729 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169586700 **[Test build #48913 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48913/consoleFull)** for PR 10634 at commit

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169588742 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12688][SQL] Fix spill size metric in un...

2016-01-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10634#issuecomment-169588705 **[Test build #48915 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48915/consoleFull)** for PR 10634 at commit