[
https://issues.apache.org/jira/browse/YARN-3051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611196#comment-14611196
]
Varun Saxena commented on YARN-3051:
------------------------------------
Thanks for the review [~zjshen].
bq. 1. We should compare the objects directly instead of converting them to
String first.
Correct.
bq. 2. No one is writing the mapping into APP_FLOW_MAPPING_FILE in the current
code base?
Yes nobody had handled mapping between app to flow in writer code path. So came
up with this solution. We can write CSV as well.
Will write the code for writer as well when I combine the FS Writer and reader
classes.
Regarding, searching for user/flowId/flowRunId separately, you mean store them
in separate files ?
bq. 3. Can we prevent introducing the test oriented configurations into
YarnConfiguration?
You mean the config about fs storage root directory
(TIMELINE_SERVICE_STORAGE_DIR_ROOT) ? This is used by writer as well and is
expected to be a configuration hence moved it to YarnConfiguration. We do not
want it as a configuration ? The config name I plan to change but that would
require change in writer too.
bq. We can do some optimization for the file implementation, such putting
created time and modified time into file name to quickly filter these files
without reading them, merging the entities and overwriting the file to prevent
merging again for each query. But that's not critical here, we can do it later.
These are good suggestions. First one even I had thought but then getting in a
working patch took priority. Anyways will handle when I merge FS reader and
writer.
> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>
> Key: YARN-3051
> URL: https://issues.apache.org/jira/browse/YARN-3051
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Varun Saxena
> Attachments: YARN-3051-YARN-2928.003.patch,
> YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch,
> YARN-3051-YARN-2928.05.patch, YARN-3051-YARN-2928.06.patch,
> YARN-3051.Reader_API.patch, YARN-3051.Reader_API_1.patch,
> YARN-3051.Reader_API_2.patch, YARN-3051.Reader_API_3.patch,
> YARN-3051.Reader_API_4.patch, YARN-3051.wip.02.YARN-2928.patch,
> YARN-3051.wip.patch, YARN-3051_temp.patch
>
>
> Per design in YARN-2928, create backing storage read interface that can be
> implemented by multiple backing storage implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
