Li Lu commented on YARN-3051:

Hi [~varun_saxena], thanks for the update! Some of my quick thoughts for 
# I just realized in this JIRA we are creating "backing storage read interface 
for ATS readers", but not the user facing ATS reader APIs. I believe these two 
topics are different: in this JIRA we're "wiring up" the storage systems, but 
in ATS reader APIs, we need to deal with user requirements. This said, I think 
the main design goal here is to provide a small set of generic interfaces so 
that we can easily connect them to our writers. We may want to have some brief 
ideas of the potential user facing features (as [~zjshen] mentioned in a 
previous comment), but I'm not sure if we need to implement them before we make 
a concrete design for the storage read interface. 
# If my understanding in point 1 is right, then perhaps we do not need to quite 
worry about the huge list of nulls. Of course, on code level we may want to to 
some cosmetic fixes, but since those interfaces are not user facing, making 
them more general may be more important I think?
# I still think when doing the v2 interface design, it is fine, if not even 
beneficial, to start from scratch rather than thinking about the existing v1 
design. If we're not implementing some v1 features as first-class in v2 storage 
implementations, maybe we can simply leave them out from the interfaces to 
storage level? (I assume we'll have an intermediate layer to do the wire up 
between our user facing reader APIs and the storage interfaces. )
# bq. Now from backing storage implementation viewpoint, would it make more 
sense to let these query params be passed as strings or do datatype conversion ?
I've got no strong preference on this. Leaving them as a generic type (like 
string) gives the storage layer more freedom to interpret the data, but the 
readers need to ensure they understand the types by themselves. 

BTW, could you please briefly skim through the list of Jenkins warnings and see 
if they're critical? Thanks! 

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, 
> YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch, 
> YARN-3051.wip.02.YARN-2928.patch, YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be 
> implemented by multiple backing storage implementations.

This message was sent by Atlassian JIRA

Reply via email to