Zhijie Shen commented on YARN-3391:

bq. do you know of any such config or context parameter that can be set so that 
we can pick up the flow name from there for all yarn applications?

I propose to use app id to generate flow name, such that every orphan app will 
be put in a unique default flow. Otherwise, if we use the job name, for 
example,  all the wordcout jobs will belong to one flow then by default.

bq. Using a run id of 1 will mean everything will fall into this bucket if no 
one sets the run id.

It won't, because each orphan app will have a unique default flow.

Compare to multiple runs under the same flow as the default, I think the more 
appropriate way is that each application should have a unique flow name. Under 
one flow, there's only one default version "1", and under the version, there's 
only one run 1.

> Clearly define flow ID/ flow run / flow version in API and storage
> ------------------------------------------------------------------
>                 Key: YARN-3391
>                 URL: https://issues.apache.org/jira/browse/YARN-3391
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
> To continue the discussion in YARN-3040, let's figure out the best way to 
> describe the flow.
> Some key issues that we need to conclude on:
> - How do we include the flow version in the context so that it gets passed 
> into the collector and to the storage eventually?
> - Flow run id should be a number as opposed to a generic string?
> - Default behavior for the flow run id if it is missing (i.e. client did not 
> set it)
> - How do we handle flow attributes in case of nested levels of flows?

This message was sent by Atlassian JIRA

Reply via email to