Joep Rottinghuis created YARN-5378:
--------------------------------------
Summary: Accomodate app-id->cluster mapping
Key: YARN-5378
URL: https://issues.apache.org/jira/browse/YARN-5378
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Joep Rottinghuis
Assignee: Joep Rottinghuis
In discussion with [~sjlee0], [~vrushalic], [~subru], and [~curino] a use-case
came up to be able to map from application-id to cluster-id in context of
federation for Yarn.
What happens is that a "random" cluster in the federation is asked to generate
an app-id and then potentially a different cluster can be the "home" cluster
for the AM. Furthermore, tasks can then run in yet other clusters.
In order to be able to pull up the logical home cluster on which the
application ran, there needs to be a mapping from application-id to cluster-id.
This mapping is available in the federated Yarn case only during the active
live of the application.
A similar situation is common in our larger production environment. Somebody
will complain about a slow job, some failure or whatever. If we're lucky we
have an application-id. When we ask the user which cluster they ran on, they'll
typically answer with the machine from where they launched the job (many users
are unaware of the underlying physical clusters). This leaves us to spelunk
through various RM ui's to find a matching epoch in the application ID.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
