Zhijie Shen commented on YARN-3051:

Hi Varun, thanks for updating the patch. I have only one remaining issue about 
this patch:

According to 
 It seems that we have chosen clusterId + appId to globally find a unique flow 
run. I think here we should do it similar by adding clusterId, which 's 
mandatory field. /cc [~sjlee0].

Some other improvement that are required in the future to improve robustness 
and performance. Let's make sure we have a jira to improve the reader later.

1. Maybe we want to cache the mapping instead of reading it from the file for 
every query.
2. limit should be push down into the for loop. It's unnecessary that if we 
want to just retrieve 10 entities, we will have to go through 1000 qualified 
candidates and finally pick the top 10.
3. We'd better avoid hard code "/" as the path separator, and we should use 
FileSystem interface to operate the files, such that the impl can also work 
with HDFS.

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, 
> YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch, 
> YARN-3051-YARN-2928.05.patch, YARN-3051-YARN-2928.06.patch, 
> YARN-3051-YARN-2928.07.patch, YARN-3051-YARN-2928.08.patch, 
> YARN-3051.Reader_API.patch, YARN-3051.Reader_API_1.patch, 
> YARN-3051.Reader_API_2.patch, YARN-3051.Reader_API_3.patch, 
> YARN-3051.Reader_API_4.patch, YARN-3051.wip.02.YARN-2928.patch, 
> YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be 
> implemented by multiple backing storage implementations.

This message was sent by Atlassian JIRA

Reply via email to