[
https://issues.apache.org/jira/browse/YARN-5378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sangjin Lee updated YARN-5378:
------------------------------
Attachment: YARN-5378-YARN-5355.02.patch
Posted patch v.2 that clarifies the encoding behavior of the column
prefix/helper.
Thanks for the review [~jrottinghuis]. As you mentioned, a slight behavior
change with this patch is that the cluster id will not be escaped for the
qualifier separator ("\!"). However, it is still safe as we always join and
split only 2 tokens with column prefixes, thus the presence of the qualifier
separator in the qualifier doesn't pose issues. And this is consistent
throughout the code base.
> Accomodate app-id->cluster mapping
> ----------------------------------
>
> Key: YARN-5378
> URL: https://issues.apache.org/jira/browse/YARN-5378
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Joep Rottinghuis
> Assignee: Sangjin Lee
> Labels: yarn-5355-merge-blocker
> Attachments: YARN-5378-YARN-5355.01.patch,
> YARN-5378-YARN-5355.02.patch
>
>
> In discussion with [~sjlee0], [~vrushalic], [~subru], and [~curino] a
> use-case came up to be able to map from application-id to cluster-id in
> context of federation for Yarn.
> What happens is that a "random" cluster in the federation is asked to
> generate an app-id and then potentially a different cluster can be the "home"
> cluster for the AM. Furthermore, tasks can then run in yet other clusters.
> In order to be able to pull up the logical home cluster on which the
> application ran, there needs to be a mapping from application-id to
> cluster-id. This mapping is available in the federated Yarn case only during
> the active live of the application.
> A similar situation is common in our larger production environment. Somebody
> will complain about a slow job, some failure or whatever. If we're lucky we
> have an application-id. When we ask the user which cluster they ran on,
> they'll typically answer with the machine from where they launched the job
> (many users are unaware of the underlying physical clusters). This leaves us
> to spelunk through various RM ui's to find a matching epoch in the
> application ID.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
