[jira] [Commented] (AMBARI-17489) Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition
[ https://issues.apache.org/jira/browse/AMBARI-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590859#comment-15590859 ] Hudson commented on AMBARI-17489: - FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #174 (See [https://builds.apache.org/job/Ambari-branch-2.5/174/]) AMBARI-17489. Remove spark.yarn.max.executor.failures configuration in (smohanty: [http://git-wip-us.apache.org/repos/asf?p=ambari.git=commit=6d3ad5adf0020bc3dc3d872d6fea7595adcbd16c]) * (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml * (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml * (edit) ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml * (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml * (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml > Remove spark.yarn.max.executor.failures configuration in Spark Ambari > definition > > > Key: AMBARI-17489 > URL: https://issues.apache.org/jira/browse/AMBARI-17489 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.4.0 >Reporter: Saisai Shao >Assignee: Saisai Shao > Fix For: 2.4.2 > > Attachments: AMBARI-17489-branch-2.4.patch > > > Currently spark.yarn.max.executor.failures should be configured in Ambari web > UI, which makes it inflexible to change, actually the default default in > Spark is more reasonable than here as "3", so propose to remove this > configuration starting from Spark 1.6.0. > CC [~zjffdu], please suggest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-17489) Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition
[ https://issues.apache.org/jira/browse/AMBARI-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590615#comment-15590615 ] Saisai Shao commented on AMBARI-17489: -- [~sumitmohanty], I've rebased this patch to branch-2.4 and 2.5 (https://reviews.apache.org/r/49484/), looks like branch 2.5 can be auto merged, so one patch is enough. Please review. Thanks a lot. > Remove spark.yarn.max.executor.failures configuration in Spark Ambari > definition > > > Key: AMBARI-17489 > URL: https://issues.apache.org/jira/browse/AMBARI-17489 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.4.0 >Reporter: Saisai Shao >Assignee: Saisai Shao > Fix For: 2.4.2 > > > Currently spark.yarn.max.executor.failures should be configured in Ambari web > UI, which makes it inflexible to change, actually the default default in > Spark is more reasonable than here as "3", so propose to remove this > configuration starting from Spark 1.6.0. > CC [~zjffdu], please suggest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-17489) Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition
[ https://issues.apache.org/jira/browse/AMBARI-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590303#comment-15590303 ] Sumit Mohanty commented on AMBARI-17489: [~jerryshao] can you rebase the patch for branch-2.4? > Remove spark.yarn.max.executor.failures configuration in Spark Ambari > definition > > > Key: AMBARI-17489 > URL: https://issues.apache.org/jira/browse/AMBARI-17489 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.4.0 >Reporter: Saisai Shao >Assignee: Saisai Shao > Fix For: 2.4.2 > > > Currently spark.yarn.max.executor.failures should be configured in Ambari web > UI, which makes it inflexible to change, actually the default default in > Spark is more reasonable than here as "3", so propose to remove this > configuration starting from Spark 1.6.0. > CC [~zjffdu], please suggest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AMBARI-17489) Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition
[ https://issues.apache.org/jira/browse/AMBARI-17489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359286#comment-15359286 ] Jeff Zhang commented on AMBARI-17489: - Correct, the default value is numExecutors * 2, with minimum of 3, we should use the default value. > Remove spark.yarn.max.executor.failures configuration in Spark Ambari > definition > > > Key: AMBARI-17489 > URL: https://issues.apache.org/jira/browse/AMBARI-17489 > Project: Ambari > Issue Type: Bug > Components: stacks >Affects Versions: 2.4.0 >Reporter: Saisai Shao >Assignee: Saisai Shao > > Currently spark.yarn.max.executor.failures should be configured in Ambari web > UI, which makes it inflexible to change, actually the default default in > Spark is more reasonable than here as "3", so propose to remove this > configuration starting from Spark 1.6.0. > CC [~zjffdu], please suggest. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
