[
https://issues.apache.org/jira/browse/YARN-10123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Ahuja updated YARN-10123:
-----------------------------------
Description:
A "stop" on a YARN application fails with the below error:
{code}
# yarn app -stop application_1581294743321_0002 -appTypes SPARK
20/02/10 06:24:27 INFO client.RMProxy: Connecting to ResourceManager at
c3224-node2.squadron.support.hortonworks.com/172.25.34.128:8050
20/02/10 06:24:27 INFO client.AHSProxy: Connecting to Application History
server at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:10200
Exception in thread "main" java.lang.IllegalArgumentException: App admin client
class name not specified for type SPARK
at
org.apache.hadoop.yarn.client.api.AppAdminClient.createAppAdminClient(AppAdminClient.java:76)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:579)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:123)
{code}
>From
>https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L76,
> it seems that this is because user does not have the setting:
{code}
yarn.application.admin.client.class.SPARK
{code}
set up in the client configuration.
However, even if this setting is present, we still need to have an
implementation available for the application type. From my internal discussions
- Jobs don't have a notion of stop / resume functionality at YARN level. If
some apps like Spark need it, it has to be implemented at those framework's
level.
Therefore, the above error message is a bit misleading in that, even if
"yarn.application.admin.client.class.SPARK" is supplied (or for that matter -
yarn.application.admin.client.class.MAPREDUCE), if there is no implementation
actually available underneath to handle the stop/start functionality then, we
will fail again, albeit with a different error here:
https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L85.
As such, maybe this error message can be improved to say something like:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: App admin client
class name not specified for type SPARK. Please ensure the App admin client
class actually exists within SPARK to handle this functionality.
{code}
or something similar.
Further, documentation around "-stop" and "-start" options will need to be
improved here ->
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#application_or_app
as it does not mention anything about having an implementation at the
framework level for the YARN stop/start command to succeed.
was:
A "stop" on a YARN application fails with the below error:
{code}
# yarn app -stop application_1581294743321_0002 -appTypes SPARK
20/02/10 06:24:27 INFO client.RMProxy: Connecting to ResourceManager at
c3224-node2.squadron.support.hortonworks.com/172.25.34.128:8050
20/02/10 06:24:27 INFO client.AHSProxy: Connecting to Application History
server at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:10200
Exception in thread "main" java.lang.IllegalArgumentException: App admin client
class name not specified for type SPARK
at
org.apache.hadoop.yarn.client.api.AppAdminClient.createAppAdminClient(AppAdminClient.java:76)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:579)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:123)
{code}
>From
>https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L76,
> it seems that this is because user does not have the setting:
{code}
yarn.application.admin.client.class.SPARK set up in the client configuration.
{code}
However, even if this setting is present, we still need to have an
implementation available for the application type. From my internal discussions
- Jobs don't have a notion of stop / resume functionality at YARN level. If
some apps like Spark need it, it has to be implemented at those framework's
level.
Therefore, the above error message is a bit misleading in that, even if
"yarn.application.admin.client.class.SPARK" is supplied (or for that matter -
yarn.application.admin.client.class.MAPREDUCE), if there is no implementation
actually available underneath to handle the stop/start functionality then, we
will fail again, albeit with a different error here:
https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L85.
As such, maybe this error message can be improved to say something like:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: App admin client
class name not specified for type SPARK. Please ensure the App admin client
class actually exists within SPARK to handle this functionality.
{code}
or something similar.
Further, documentation around "-stop" and "-start" options will need to be
improved here ->
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#application_or_app
as it does not mention anything about having an implementation at the
framework level for the YARN stop/start command to succeed.
> Error message around yarn app -stop/start can be improved to highlight that
> an implementation at framework level is needed for the stop/start
> functionality to work
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-10123
> URL: https://issues.apache.org/jira/browse/YARN-10123
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: client, documentation
> Affects Versions: 3.2.1
> Reporter: Siddharth Ahuja
> Assignee: Siddharth Ahuja
> Priority: Minor
>
> A "stop" on a YARN application fails with the below error:
> {code}
> # yarn app -stop application_1581294743321_0002 -appTypes SPARK
> 20/02/10 06:24:27 INFO client.RMProxy: Connecting to ResourceManager at
> c3224-node2.squadron.support.hortonworks.com/172.25.34.128:8050
> 20/02/10 06:24:27 INFO client.AHSProxy: Connecting to Application History
> server at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:10200
> Exception in thread "main" java.lang.IllegalArgumentException: App admin
> client class name not specified for type SPARK
> at
> org.apache.hadoop.yarn.client.api.AppAdminClient.createAppAdminClient(AppAdminClient.java:76)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:579)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at
> org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:123)
> {code}
> From
> https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L76,
> it seems that this is because user does not have the setting:
> {code}
> yarn.application.admin.client.class.SPARK
> {code}
> set up in the client configuration.
> However, even if this setting is present, we still need to have an
> implementation available for the application type. From my internal
> discussions - Jobs don't have a notion of stop / resume functionality at YARN
> level. If some apps like Spark need it, it has to be implemented at those
> framework's level.
> Therefore, the above error message is a bit misleading in that, even if
> "yarn.application.admin.client.class.SPARK" is supplied (or for that matter -
> yarn.application.admin.client.class.MAPREDUCE), if there is no implementation
> actually available underneath to handle the stop/start functionality then, we
> will fail again, albeit with a different error here:
> https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L85.
> As such, maybe this error message can be improved to say something like:
> {code}
> Exception in thread "main" java.lang.IllegalArgumentException: App admin
> client class name not specified for type SPARK. Please ensure the App admin
> client class actually exists within SPARK to handle this functionality.
> {code}
> or something similar.
> Further, documentation around "-stop" and "-start" options will need to be
> improved here ->
> https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#application_or_app
> as it does not mention anything about having an implementation at the
> framework level for the YARN stop/start command to succeed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
