[ 
https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611196#comment-14611196
 ] 

Varun Saxena commented on YARN-3051:
------------------------------------

Thanks for the review [~zjshen].

bq. 1. We should compare the objects directly instead of converting them to 
String first.
Correct.

bq. 2. No one is writing the mapping into APP_FLOW_MAPPING_FILE in the current 
code base? 
Yes nobody had handled mapping between app to flow in writer code path. So came 
up with this solution. We can write CSV as well.
Will write the code for writer as well when I combine the FS Writer and reader 
classes.
Regarding, searching for user/flowId/flowRunId separately, you mean store them 
in separate files ?

bq. 3. Can we prevent introducing the test oriented configurations into 
YarnConfiguration?
You mean the config about fs storage root directory 
(TIMELINE_SERVICE_STORAGE_DIR_ROOT) ? This is used by writer as well and is 
expected to be a configuration hence moved it to YarnConfiguration. We do not 
want it as a configuration ? The config name I plan to change but that would 
require change in writer too.

bq. We can do some optimization for the file implementation, such putting 
created time and modified time into file name to quickly filter these files 
without reading them, merging the entities and overwriting the file to prevent 
merging again for each query. But that's not critical here, we can do it later.
These are good suggestions. First one even I had thought but then getting in a 
working patch took priority. Anyways will handle when I merge FS reader and 
writer.


> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, 
> YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch, 
> YARN-3051-YARN-2928.05.patch, YARN-3051-YARN-2928.06.patch, 
> YARN-3051.Reader_API.patch, YARN-3051.Reader_API_1.patch, 
> YARN-3051.Reader_API_2.patch, YARN-3051.Reader_API_3.patch, 
> YARN-3051.Reader_API_4.patch, YARN-3051.wip.02.YARN-2928.patch, 
> YARN-3051.wip.patch, YARN-3051_temp.patch
>
>
> Per design in YARN-2928, create backing storage read interface that can be 
> implemented by multiple backing storage implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to