[
https://issues.apache.org/jira/browse/YARN-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838255#comment-15838255
]
Rohith Sharma K S commented on YARN-3588:
-----------------------------------------
As we are discussed about this in the weekly call, few folks asked the
question of what makes entity uniqueness? I believe this is left to the
framework publishers as they will be publishing the entities. Lets say, DAG
entities and MR_JOB_ID entities will never be unique across the
application/user/flow/flow-run/clusters. I think we should build a indexing
table for <EntityType, EntityId> that helps a lot while reading entities for
given EntityType. And need not to index all the entities. may be publisher can
input that this entity should be indexed.
I was looking few existing long running applications where it generates tons of
entities which will be stored in ATSv2. All entities are not important but
still they will keep it for future reference. Only set of entity types are
important which are regularly queried. So, these important entities can be
indexed on user demand.
> Timeline entity uniqueness
> --------------------------
>
> Key: YARN-3588
> URL: https://issues.apache.org/jira/browse/YARN-3588
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Zhijie Shen
> Assignee: Vrushali C
> Labels: YARN-5355
>
> In YARN-3051, we have some discussion about how to uniquely identify an
> entity. Sangjin and some other folks propose to only uniquely identify an
> entity by <type, id> in the scope of a single app. This is different from
> entity uniqueness in ATSv1, where <type, id> can globally identify an entity.
> This is going to affect the way of fetching a single entity, and raise the
> compatibility issue. Let's continue our discussion here to unblock YARN-3051.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
