Zhijie Shen commented on YARN-2928:

bq. A single tez application can run multiple different Hive queries submitted 
by different users.

In this use case, who is the user of the TEZ application? This may affect the 
data mode and the parent-child relationship (cluster->user->flow->flow 

bq. Where does the current implementation's "otherInfo" and "primaryFilters" 
fit in? 

"metadata" aims to store the same thing as "otherInfo", but I didn't want to be 
called "otherinfo" because it's no longer the other info than "primaryFilters". 
When making the new schema, I'm looking for the option to have the entity 
indexed, but don't need to explicitly specify what is the "primaryFilters", 
which makes trouble and bugs when updating the entity before.

bq. What are the main differences between meta-data and configuration? 

It may be combined, as I consider both are key-value pairs, but I distinguish 
them explicitly for better usage. Or is there any special access pattern for 

bq. If there is a hierarchy of objects, will there be support to listen to or 
retrieve all events for a given tree by providing a root node?

We may probably run adhoc query to get the events of all applications of a 

bq. What use are events? Will there be a "streaming" API available to listen to 
all events based on some search criteria?
bq. In certain cases, it might be required to mine a specific job's data by 
exporting contents out of ATS.

They sound to be interesting features, but we may not able to accommodate them 
within Hadoop 2.8 timeline.

> Application Timeline Server (ATS) next gen: phase 1
> ---------------------------------------------------
>                 Key: YARN-2928
>                 URL: https://issues.apache.org/jira/browse/YARN-2928
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Priority: Critical
>         Attachments: ATSv2.rev1.pdf, ATSv2.rev2.pdf, Data model proposal 
> v1.pdf
> We have the application timeline server implemented in yarn per YARN-1530 and 
> YARN-321. Although it is a great feature, we have recognized several critical 
> issues and features that need to be addressed.
> This JIRA proposes the design and implementation changes to address those. 
> This is phase 1 of this effort.

This message was sent by Atlassian JIRA

Reply via email to