[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18904 Thanks @MLnick, I will be glad if you can continue it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18904 Because I don't have the environment to continue this work, I will close it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18904 This is another case. Table 1 shows the improvement of random tree algorithm with sparse expression. We can see that when we use sparse expression, I/O can be reduced by 61% and total run time can be reduced by 39%. The dataset has 100k samples and 10k features in Gaussian distribution and its number of partitions is 300. The max depth of RF is 17 and number of bins is 40.  Only when the network is a bottleneck, this optimization will work better. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18904  This is one of my test results. Now, I am not working on Spark MLLIB, and don't have hardware to do more test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18904 @mpjlu could you post the actual results of test runs (timing numbers and shuffle data)? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/18904 @mpjlu could you post the actual results of test runs (timing numbers and shuffle data)? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80601/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80601 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80601/testReport)** for PR 18904 at commit [`b349668`](https://github.com/apache/spark/commit/b34966871dbc5d13c697965e227b6136faed4c9a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80601 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80601/testReport)** for PR 18904 at commit [`b349668`](https://github.com/apache/spark/commit/b34966871dbc5d13c697965e227b6136faed4c9a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18904 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80527/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80527/testReport)** for PR 18904 at commit [`b349668`](https://github.com/apache/spark/commit/b34966871dbc5d13c697965e227b6136faed4c9a). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80527/testReport)** for PR 18904 at commit [`b349668`](https://github.com/apache/spark/commit/b34966871dbc5d13c697965e227b6136faed4c9a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/18904 A gentle ping: @sethah @jkbradley --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18904 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80480/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80480/testReport)** for PR 18904 at commit [`35d1f24`](https://github.com/apache/spark/commit/35d1f244f918bd8ea7fe7fdf10796a64e7a62fc9). * This patch **fails SparkR unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #18904: [SPARK-21624]optimzie RF communicaiton cost
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18904 **[Test build #80480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80480/testReport)** for PR 18904 at commit [`35d1f24`](https://github.com/apache/spark/commit/35d1f244f918bd8ea7fe7fdf10796a64e7a62fc9). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
