[
https://issues.apache.org/jira/browse/YARN-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254914#comment-15254914
]
Sangjin Lee commented on YARN-3150:
-----------------------------------
[~gtCarrera9]:
bq. For the list of timeline v2 configs, maybe we'd like to distinguish the
configs that we adopt from existing ATS v1.x configs and the newly introduced
configs? We may want to stress the overridden configs.
I introduced a column that marks whether the config is new in v.2 as opposed to
existing. See if that format works.
bq. Maybe we'd like to have a few more sentences about the timeline schema
creator? There are some "hidden" functions that might be interesting.
I did add a sentence about skipping existing tables. I didn't document the rest
of the options as I thought those options are mostly geared towards us (TS v.2
developers) rather than general developers/users. Let me know your thoughts.
bq. We may want to clarify the meaning of "system metrics" and "container
metrics" in the document. When readers have some v1 background, it may be
helpful to distinguish a few wordings in the document: "system metrics" vs.
"application history data" in AHS, "container metrics" vs. the old "public
container metrics" option in v1.
I tried to clean up the terminology. I am mostly using "system metrics" to
refer to YARN-generated metrics. "Container metrics" are not entirely accurate
as we are aggregating them to be at the app level, flow level, etc.
While we're at it, I did notice one of the config properties was not described
correctly. {{yarn.rm.system-metrics-publisher.emit-container-events}} is about
RM publisher emitting container *events*, not *metrics*. I corrected the
description and related variable/method names. cc [~Naganarasimha]
bq. We may want to explicitly mention in the "Publishing application specific
data" section that this section is mainly for YARN application programmers, but
not for cluster operators.
Added a sentence.
bq. Note the programmers that the return value of v2 APIs are changed to void?
Good point. Added a couple of sentences.
bq. Maybe we can be more precise about the "reasonable defaults" for flow
contexts?
Done.
bq. We need separate docs for the REST APIs in the future. Right now the REST
API doc is just a simple reference.
I changed a word there to say "informal". Yes, this is not a complete REST API
description. I'm not quite sure if we're at a point where we can generate a
complete reference for that yet. So that will have to wait a little...
I also added some more about the high level architecture and a diagram.
> [Documentation] Documenting the timeline service v2
> ---------------------------------------------------
>
> Key: YARN-3150
> URL: https://issues.apache.org/jira/browse/YARN-3150
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Zhijie Shen
> Assignee: Sangjin Lee
> Labels: yarn-2928-1st-milestone
> Attachments: TimelineServiceV2.html, YARN-3150-YARN-2928.01.patch,
> YARN-3150-YARN-2928.02.patch
>
>
> Let's make sure we will have a document to describe what's new in TS v2, the
> APIs, the client libs and so on. We should do better around documentation in
> v2 than v1.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
