[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20472 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user lucio-yz commented on the issue: https://github.com/apache/spark/pull/20472 @srowen Any other problems? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87865/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87865 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87865/testReport)** for PR 20472 at commit [`fea3aad`](https://github.com/apache/spark/commit/fea3aad46d1094cf67c2770edab1158b3dece225). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87865/testReport)** for PR 20472 at commit [`fea3aad`](https://github.com/apache/spark/commit/fea3aad46d1094cf67c2770edab1158b3dece225). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87864/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87864/testReport)** for PR 20472 at commit [`656abef`](https://github.com/apache/spark/commit/656abef989717ab2c66e4e3aa6f9b1f76a0f41a8). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87864/testReport)** for PR 20472 at commit [`656abef`](https://github.com/apache/spark/commit/656abef989717ab2c66e4e3aa6f9b1f76a0f41a8). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87839/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87839/testReport)** for PR 20472 at commit [`b5a4741`](https://github.com/apache/spark/commit/b5a47411867c9bc532e9b7e680cc131b232937d0). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87839/testReport)** for PR 20472 at commit [`b5a4741`](https://github.com/apache/spark/commit/b5a47411867c9bc532e9b7e680cc131b232937d0). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87834 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87834/testReport)** for PR 20472 at commit [`827061c`](https://github.com/apache/spark/commit/827061ce8c87f483a196a1f5355136978b97ee46). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87834/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87834 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87834/testReport)** for PR 20472 at commit [`827061c`](https://github.com/apache/spark/commit/827061ce8c87f483a196a1f5355136978b97ee46). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87824/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87824/testReport)** for PR 20472 at commit [`bd7d3b2`](https://github.com/apache/spark/commit/bd7d3b264c88412aa1001e30b0301d54dad3f159). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87824/testReport)** for PR 20472 at commit [`bd7d3b2`](https://github.com/apache/spark/commit/bd7d3b264c88412aa1001e30b0301d54dad3f159). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87823 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87823/testReport)** for PR 20472 at commit [`d634bbd`](https://github.com/apache/spark/commit/d634bbda06af43532240ba498cda461e374e4037). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87823/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87823/testReport)** for PR 20472 at commit [`d634bbd`](https://github.com/apache/spark/commit/d634bbda06af43532240ba498cda461e374e4037). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87820/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87820/testReport)** for PR 20472 at commit [`51900da`](https://github.com/apache/spark/commit/51900da3266a9025ace567e3cbd5bf2b26051651). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87820/testReport)** for PR 20472 at commit [`51900da`](https://github.com/apache/spark/commit/51900da3266a9025ace567e3cbd5bf2b26051651). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87557/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87557 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87557/testReport)** for PR 20472 at commit [`1716acc`](https://github.com/apache/spark/commit/1716accb70b86123f1bbab00ff7af962199c9cd9). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #87557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87557/testReport)** for PR 20472 at commit [`1716acc`](https://github.com/apache/spark/commit/1716accb70b86123f1bbab00ff7af962199c9cd9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #4101 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4101/testReport)** for PR 20472 at commit [`1716acc`](https://github.com/apache/spark/commit/1716accb70b86123f1bbab00ff7af962199c9cd9). * This patch **fails Scala style tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20472 **[Test build #4101 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4101/testReport)** for PR 20472 at commit [`1716acc`](https://github.com/apache/spark/commit/1716accb70b86123f1bbab00ff7af962199c9cd9). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20472 Jenkins, add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user sethah commented on the issue: https://github.com/apache/spark/pull/20472 @srowen Can you trigger the tests? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20472 @srowen now the PR is good IMHO, do you have other comments? Or do you think we can trigger a build? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user lucio-yz commented on the issue: https://github.com/apache/spark/pull/20472 I tested on 2 datasets: 1. _rcv1.binary_, which has 47,236 dimensions. Before improvement, the shuffle write size in _findSplitsBySorting_ is 1GB. After improvement, the shuffle size is 7.7MB. 2. _news20.binary_, which has 1,355,191 dimensions. Before improvement, the shuffle write size in _findSplitsBySorting_ is 51 GB. After improvement, the shuffle size is 24.1 MB. ps: I tested on a cluster which has 10 nodes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user lucio-yz commented on the issue: https://github.com/apache/spark/pull/20472 previous problems have been solved --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20472: [SPARK-22751][ML]Improve ML RandomForest shuffle perform...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20472 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
