Dennis Huo created SPARK-9782:
---------------------------------
Summary: Add support for YARN application tags running Spark on
YARN
Key: SPARK-9782
URL: https://issues.apache.org/jira/browse/SPARK-9782
Project: Spark
Issue Type: Improvement
Components: YARN
Affects Versions: 1.4.1
Reporter: Dennis Huo
https://issues.apache.org/jira/browse/YARN-1390 originally added the new
“Application Tags” feature to YARN to help track the sources of applications
among many possible YARN clients.
https://issues.apache.org/jira/browse/YARN-1399 improved on this to allow a set
of tags to be applied, and for comparison,
https://issues.apache.org/jira/browse/MAPREDUCE-5699 added support for
MapReduce to easily propagate tags through to YARN via Configuration settings.
Since the ApplicationSubmissionContext.setApplicationTags method was only added
in Hadoop 2.4+, Spark support will invoke the method via reflection the same
way other such version-specific methods are called in elsewhere in the YARN
client. Since the usage of tags is generally not critical to the functionality
of older YARN setups, it should be safe to handle NoSuchMethodException with
just a logWarning.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
