[
https://issues.apache.org/jira/browse/YARN-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999670#comment-14999670
]
Vinod Kumar Vavilapalli commented on YARN-4183:
-----------------------------------------------
Getting the obvious out of the way: It's a mess.
h4. How things worked before this JIRA
- RM uses {{generic-application-history.enabled}} to activate
RMApplicationHistoryWriter (RM sending app events to the
now-dead-but-kept-for-compat APPLICATION_HISTORY_STORE)
- RM uses {{yarn.timeline-service.enabled}} +
{{yarn.resourcemanager.system-metrics-publisher.enabled}} to write
app/app-attempt/container events to Timeline Service
- YarnClient uses {{generic-application-history.enabled}} to talk to the
_history_ server irrespective of where the historic data gets stored
- TimelineClient (embedded inside YarnClient) uses
{{yarn.timeline-service.enabled}} to get tokens and populate during
app-submission.
h4. Quick general context
- Nobody is expected to use RMApplicationHistoryWriter
- {{yarn.timeline-service.generic-application-history.enabled}} is also
supposed to be dead for all purposes. But it is today used beyond the RM using
it to activate RMApplicationHistoryWriter
- SystemMetricsPublisher only writes events to TimelineService (v1, v1.5)
Given the above, I can't but conclude that the existing configuration is not
modeled correctly.
h4. The right thing to do
- Make SystemMetricsPublisher only respect
{{yarn.resourcemanager.system-metrics-publisher.enabled}}
- Leave {{yarn.timeline-service.generic-application-history.enabled}} as a
dead property only to activate RMApplicationHistoryWriter.
- We can leave {{yarn.timeline-service.generic-application-history.enabled}}
to also activate client -> RM for historical data or make RM always proxy these
calls for the client
- There should be an explicit {{yarn.timeline-service.version}} which tells
YarnClient to get tokens or not - yes for non-present version (default), v1, v2
but no for v1.5.
- We should also use the same property in the new API calls proposed for V1.5
YARN-4233 / V2 YARN-2928, lest the users think they can call any API
independent of what is supported on server side. The version field has
semantics on both client *and* server side at the same time - it's picking a
solution end-to-end.
h4. Immediate step
All of this needs more work, so unless I hear strongly otherwise I am going to
revert this patch in the interest of 2.7.2's progress.
/cc [~hitesh] [~gtCarrera9] [~xgong] [~sjlee0]
> Enabling generic application history forces every job to get a timeline
> service delegation token
> ------------------------------------------------------------------------------------------------
>
> Key: YARN-4183
> URL: https://issues.apache.org/jira/browse/YARN-4183
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.1
> Reporter: Mit Desai
> Assignee: Mit Desai
> Fix For: 3.0.0, 2.8.0, 2.7.2
>
> Attachments: YARN-4183.1.patch
>
>
> When enabling just the Generic History Server and not the timeline server,
> the system metrics publisher will not publish the events to the timeline
> store as it checks if the timeline server and system metrics publisher are
> enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force
> every yarn application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if
> application history server is enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
