[
https://issues.apache.org/jira/browse/YARN-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15524485#comment-15524485
]
Li Lu commented on YARN-3981:
-----------------------------
Continuing this thread. We may want to limit the active connections to the
HBase backend, thus having collectors for clients on each node manager may be a
little bit too aggressive. For normal use cases, a small subset of NMs running
collectors would be enough?
Another issue is that the RM also needs to discover available client collectors
in the system. It can then dispatch those collectors to clients. The collector
service discovery code for now couples app level collectors with their
application id. We need to expand them to a more general collector_id,
collector_data mapping to support collectors for clients. I'm opening a new
JIRA to trace this.
> support timeline clients not associated with an application
> -----------------------------------------------------------
>
> Key: YARN-3981
> URL: https://issues.apache.org/jira/browse/YARN-3981
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Rohith Sharma K S
> Labels: YARN-5355
>
> In the current v.2 design, all timeline writes must belong in a
> flow/application context (cluster + user + flow + flow run + application).
> But there are use cases that require writing data outside the context of an
> application. One such example is a higher level client (e.g. tez client or
> hive/oozie/cascading client) writing flow-level data that spans multiple
> applications. We need to find a way to support them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
