[
https://issues.apache.org/jira/browse/YARN-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310056#comment-14310056
]
Zhijie Shen commented on YARN-2928:
-----------------------------------
bq. A single tez application can run multiple different Hive queries submitted
by different users.
In this use case, who is the user of the TEZ application? This may affect the
data mode and the parent-child relationship (cluster->user->flow->flow
run->application).
bq. Where does the current implementation's "otherInfo" and "primaryFilters"
fit in?
"metadata" aims to store the same thing as "otherInfo", but I didn't want to be
called "otherinfo" because it's no longer the other info than "primaryFilters".
When making the new schema, I'm looking for the option to have the entity
indexed, but don't need to explicitly specify what is the "primaryFilters",
which makes trouble and bugs when updating the entity before.
bq. What are the main differences between meta-data and configuration?
It may be combined, as I consider both are key-value pairs, but I distinguish
them explicitly for better usage. Or is there any special access pattern for
config?
bq. If there is a hierarchy of objects, will there be support to listen to or
retrieve all events for a given tree by providing a root node?
We may probably run adhoc query to get the events of all applications of a
workflow.
bq. What use are events? Will there be a "streaming" API available to listen to
all events based on some search criteria?
bq. In certain cases, it might be required to mine a specific job's data by
exporting contents out of ATS.
They sound to be interesting features, but we may not able to accommodate them
within Hadoop 2.8 timeline.
> Application Timeline Server (ATS) next gen: phase 1
> ---------------------------------------------------
>
> Key: YARN-2928
> URL: https://issues.apache.org/jira/browse/YARN-2928
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: timelineserver
> Reporter: Sangjin Lee
> Priority: Critical
> Attachments: ATSv2.rev1.pdf, ATSv2.rev2.pdf, Data model proposal
> v1.pdf
>
>
> We have the application timeline server implemented in yarn per YARN-1530 and
> YARN-321. Although it is a great feature, we have recognized several critical
> issues and features that need to be addressed.
> This JIRA proposes the design and implementation changes to address those.
> This is phase 1 of this effort.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
