Siddharth Ahuja created YARN-10123:
--------------------------------------
Summary: Error message around yarn app -stop/start can be improved
to highlight that an implementation at framework level is needed for the
stop/start functionality to work
Key: YARN-10123
URL: https://issues.apache.org/jira/browse/YARN-10123
Project: Hadoop YARN
Issue Type: Improvement
Components: client, documentation
Affects Versions: 3.2.1
Reporter: Siddharth Ahuja
A "stop" on a YARN application fails with the below error:
{code}
# yarn app -stop application_1581294743321_0002 -appTypes SPARK
20/02/10 06:24:27 INFO client.RMProxy: Connecting to ResourceManager at
c3224-node2.squadron.support.hortonworks.com/172.25.34.128:8050
20/02/10 06:24:27 INFO client.AHSProxy: Connecting to Application History
server at c3224-node2.squadron.support.hortonworks.com/172.25.34.128:10200
Exception in thread "main" java.lang.IllegalArgumentException: App admin client
class name not specified for type SPARK
at
org.apache.hadoop.yarn.client.api.AppAdminClient.createAppAdminClient(AppAdminClient.java:76)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:579)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at
org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:123)
{code}
>From
>https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L76,
> it seems that this is because user does not have the setting:
{code}
yarn.application.admin.client.class.SPARK set up in the client configuration.
{code}
However, even if this setting is present, we still need to have an
implementation available for the application type. From my internal discussions
- Jobs don't have a notion of stop / resume functionality at YARN level. If
some apps like Spark need it, it has to be implemented at those framework's
level.
Therefore, the above error message is a bit misleading in that, even if
"yarn.application.admin.client.class.SPARK" is supplied (or for that matter -
yarn.application.admin.client.class.MAPREDUCE), if there is no implementation
actually available underneath to handle the stop/start functionality then, we
will fail again, albeit with a different error here:
https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/AppAdminClient.java#L85.
As such, maybe this error message can be improved to say something like:
{code}
Exception in thread "main" java.lang.IllegalArgumentException: App admin client
class name not specified for type SPARK, is there an implementation even
available for this framework?
{code}
or something similar.
Further, documentation around "-stop" and "-start" options will need to be
improved here ->
https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnCommands.html#application_or_app
as it does not mention anything about having an implementation at the
framework level for the YARN stop/start command to succeed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
