[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/23260 Ok, got it. @vanzin or @squito or others would be better able to evaluate. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/23260 @srowen For now executor log url is **static** in Spark, which forces Node Manager to be alive even after application is finished, in order to provide executor log in SHS. This situation can be happen when decommission happens a bit frequently, or when end users want a kind of elasticity against YARN cluster (not only decommissioning nodes, but also elasticity on YARN cluster itself - YARN has cluster id for RM which classifies the cluster which can be leveraged when dealing with multiple YARN clusters.) There's also similar change applied on Hadoop side. https://github.com/apache/hadoop/commit/5fe1dbf1959976d0dc5a8e614dd74836cfbee04c We are experimenting central log service which resolves above situation. At least the log url for centralized log service can't be same URL as NM webapp, we have to get flexibility of executor log URL. Hope it explains the rationalization well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user srowen commented on the issue: https://github.com/apache/spark/pull/23260 If you're on YARN, this feels like something you would manage via YARN and its cluster management options. Is there a specific use case here, that this has to happen in Spark? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23260 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23260 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99857/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23260 **[Test build #99857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99857/testReport)** for PR 23260 at commit [`65cc6a3`](https://github.com/apache/spark/commit/65cc6a32729cccba340f66c766c7255be4d7f356). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23260 **[Test build #99857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99857/testReport)** for PR 23260 at commit [`65cc6a3`](https://github.com/apache/spark/commit/65cc6a32729cccba340f66c766c7255be4d7f356). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23260 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #23260: [SPARK-26311][YARN] New feature: custom log URL for stdo...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23260 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
