Sangjin Lee commented on YARN-3051:

My major concern about this proposal is compatibility. Previously in v1, 
timeline entity is globally unique, such that when fetching a single entity 
before, users only need to provide <entity type, entity id>. <app id, entity 
type, entity id> is required to locate one entity, and theoretically <null, 
entity type, entity id> will refer to multiple entities. It probably makes 
difficult to be compatible to existing use cases.

To hash out that point, existing use cases which previously assumed that entity 
id was globally unique would continue to generate entity id's that are globally 
unique, right? Since existing use cases (w/o modification) would stick to 
globally unique entity id's in practice, redefining the uniqueness requirement 
to be in the scope of application should not impact existing use cases. Entity 
id's that are generated to be unique globally would trivially be unique within 
the application scope. The point here is that since this is in the direction of 
relaxing uniqueness, stricter use cases (existing use cases) should not be 
impacted. Let me know your thoughts.

IMO, stating that the entity id's are unique within the scope of applications 
is not an invitation for frameworks to generate tons of redundant entity id's. 
Frameworks (MR, tez, ...) would likely continue to generate entity id's that 
are practically unique globally anyway. But the part of the timeline service, 
we don't have to have checks for enforcing global uniqueness.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be 
> implemented by multiple backing storage implementations.

This message was sent by Atlassian JIRA

Reply via email to