[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17471 thanks, merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78851/ Test PASSed. ---

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78851 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78851/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78851/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17471 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78830/ Test FAILed. ---

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78830/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78830/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78821/ Test FAILed. ---

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78821 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78821/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78821/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 >> shall we fix it for other sorters? e.g. UnsafeExternalRowSorter, UnsafeKVExternalSorter, etc. Actually other sorters are using UnsafeExternalSorter

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17471 LGTM, shall we fix it for other sorters? e.g. `UnsafeExternalRowSorter`, `UnsafeKVExternalSorter`, etc. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78405/ Test PASSed. ---

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78405 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78405/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/17471 also, see my message to the dev@ list. jenkins is up, but not reachable via the amplab link. On Wed, Jun 21, 2017 at 3:23 PM, shane knapp ☠ wrote:

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/17471 it was actually a cut power line due to street construction. apparently power is back online, and we're waiting for machines and switches to finish booting back up. i'm WFH

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/17471 I think the power is still out in the CS building at Berkeley because of the earthquake, so I'm guessing Jenkins is down as a result (note that even the vanilla AMP website doesn't work:

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 @shaneknapp, @sameeragarwal - Not able to access the jenkins build link- https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78405/testReport, something wrong with the jenkins

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78405/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 Jenkins retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78363/ Test FAILed. ---

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78363/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/17471 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 Thanks for the ping @HyukjinKwon. Updated the PR with test case. cc - @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #78348 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78348/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17471 I was just looking though PRs for my curiosity. Please let me leave a gentle ping @sitalkedia. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 @sameeragarwal - Thanks for taking a look. I will update the PR adding test case soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sameeragarwal commented on the issue: https://github.com/apache/spark/pull/17471 @sitalkedia any updates here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 I did not see we already have an open PR for this. Sure, I will add test to this PR and also file a separate JIRA. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user squito commented on the issue: https://github.com/apache/spark/pull/17471 this is basically the same as this old pr, right? https://github.com/apache/spark/pull/15616 the last comment on there is a request for a test, which I think still applies here. I also

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 @cloud-fan - There is discussion in the JIRA to report the disk spill size as well. I can create a new JIRA if you prefer. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17471 SPARK-3577 is about reporting spill time, seems not related to this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user rxin commented on the issue: https://github.com/apache/spark/pull/17471 cc @cloud-fan / @ueshin / @sameeragarwal can you review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/17471 cc - @rxin, @kayousterhout, @squito --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75363/ Test PASSed. ---

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17471 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #75363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75363/testReport)** for PR 17471 at commit

[GitHub] spark issue #17471: [SPARK-3577] Report Spill size on disk for UnsafeExterna...

Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17471 **[Test build #75363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75363/testReport)** for PR 17471 at commit