[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4350#discussion_r24267242 --- Diff: docs/running-on-yarn.md --- @@ -105,6 +105,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes /td /tr tr + tdcodespark.executor.instances/code/td + td2/td + td +The number of executors. Don't set this when dynamic allocation is enabled as they are not compatible. --- End diff -- This is only used in YARN. I will add this when I merge --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/4350#discussion_r24267256 --- Diff: docs/running-on-yarn.md --- @@ -105,6 +105,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes /td /tr tr + tdcodespark.executor.instances/code/td + td2/td + td +The number of executors. Don't set this when dynamic allocation is enabled as they are not compatible. --- End diff -- oh wait this is the YARN page. LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72785278 retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72785441 [Test build #26713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26713/consoleFull) for PR 4350 at commit [`4c3913a`](https://github.com/apache/spark/commit/4c3913add23b39e4c5a5120d8a56917f972e1b4b). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72789745 [Test build #26713 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26713/consoleFull) for PR 4350 at commit [`4c3913a`](https://github.com/apache/spark/commit/4c3913add23b39e4c5a5120d8a56917f972e1b4b). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72789756 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26713/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72778202 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26699/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72778196 [Test build #26699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26699/consoleFull) for PR 4350 at commit [`4c3913a`](https://github.com/apache/spark/commit/4c3913add23b39e4c5a5120d8a56917f972e1b4b). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72799695 Seems like the UT has broken, but I think it is unralted with this patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4335#issuecomment-72642208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26652/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4335#issuecomment-72642199 [Test build #26652 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26652/consoleFull) for PR 4335 at commit [`46c40c4`](https://github.com/apache/spark/commit/46c40c4ddc72b45d6f070b54e5e8e85b68ee0add). * This patch **passes all tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user sryza commented on a diff in the pull request: https://github.com/apache/spark/pull/4335#discussion_r24022023 --- Diff: docs/running-on-yarn.md --- @@ -105,6 +105,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes /td /tr tr + tdcodespark.executor.instances/code/td + td2/td + td +The number of executors. --- End diff -- Can this clarify that spark.executor.instances is ignored when using dynamic allocation? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/4335 [SPARK-2945][YARN][Doc]add doc for spark.executor.instances https://issues.apache.org/jira/browse/SPARK-2945 `spark.executor.instances` works. As this JIRA recommended, we should add docs for it. You can merge this pull request into a Git repository by running: $ git pull https://github.com/WangTaoTheTonic/spark SPARK-2945 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/4335.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4335 commit 46c40c4ddc72b45d6f070b54e5e8e85b68ee0add Author: WangTaoTheTonic barneystin...@aliyun.com Date: 2015-02-03T09:28:56Z add doc for spark.executor.instances --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4335#issuecomment-72633373 [Test build #26652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26652/consoleFull) for PR 4335 at commit [`46c40c4`](https://github.com/apache/spark/commit/46c40c4ddc72b45d6f070b54e5e8e85b68ee0add). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user WangTaoTheTonic closed the pull request at: https://github.com/apache/spark/pull/4335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/4350 [SPARK-2945][YARN][Doc]add doc for spark.executor.instances https://issues.apache.org/jira/browse/SPARK-2945 spark.executor.instances works. As this JIRA recommended, we should add docs for this common config. You can merge this pull request into a Git repository by running: $ git pull https://github.com/WangTaoTheTonic/spark SPARK-2945 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/4350.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4350 commit 5fa9c46967b5427bd586403b2fd935f80f29ec78 Author: WangTaoTheTonic wangtao...@huawei.com Date: 2015-02-03T09:32:14Z add doc for spark.executor.instances commit 4c3913add23b39e4c5a5120d8a56917f972e1b4b Author: WangTaoTheTonic wangtao...@huawei.com Date: 2015-02-04T01:19:00Z not compatible with dynamic allocation --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/4335#issuecomment-72772960 Sorry for deleting the branch by mistake. The new PR is here: https://github.com/apache/spark/pull/4350 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72773202 [Test build #26699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26699/consoleFull) for PR 4350 at commit [`4c3913a`](https://github.com/apache/spark/commit/4c3913add23b39e4c5a5120d8a56917f972e1b4b). * This patch merges cleanly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-2945][YARN][Doc]add doc for spark.execu...
Github user WangTaoTheTonic commented on the pull request: https://github.com/apache/spark/pull/4350#issuecomment-72773151 @sryza Added docs for situation that dynamic allocation is enabled. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
