[
https://issues.apache.org/jira/browse/YARN-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533782#comment-15533782
]
Vrushali C commented on YARN-3981:
----------------------------------
Can you give an example of what information is to be written at the flow level?
Is it at the flow level or at flow run level? Put another way, is this
information going to be stored each time say a hive script is run or is it to
be written just the very first time it is ever run? The attributes of a flow
run like start time or end time are determined by the coprocessor automatically
so those need not be written specially.
If we need to write information that belongs to a particular flow run but not
tied to a specific application with it, we should write this to the "flow run"
table, not the "entity table".
Implementation detail note: the coprocessor is setup for this flow run table,
so a little more attention needed here to ensure we set/do not the right cell
tags.
In order to determine where to have writer processes running and how many and
how often, how to discover those etc, I think it will be helpful to know what
kind of information is to be written.
> support timeline clients not associated with an application
> -----------------------------------------------------------
>
> Key: YARN-3981
> URL: https://issues.apache.org/jira/browse/YARN-3981
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Rohith Sharma K S
> Labels: YARN-5355
>
> In the current v.2 design, all timeline writes must belong in a
> flow/application context (cluster + user + flow + flow run + application).
> But there are use cases that require writing data outside the context of an
> application. One such example is a higher level client (e.g. tez client or
> hive/oozie/cascading client) writing flow-level data that spans multiple
> applications. We need to find a way to support them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
