[
https://issues.apache.org/jira/browse/YARN-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007543#comment-15007543
]
Naganarasimha G R commented on YARN-4183:
-----------------------------------------
Hi [~sjlee0],
Thanks for sharing your thoughts
bq. If the answer is "yarn.resourcemanager.system-metrics-publisher.enabled
should not be set to true if the timeline service is disabled", then it only
makes it clear that yarn.resourcemanager.system-metrics-publisher.enabled=true
implies yarn.timeline-service.enabled=true. Then we should check it explicitly.
Thoughts?
{{yarn.timeline-service.enabled}} doesn't imply that timeline service is
running as we can enable it and still if the timeline server is not running
then we face the same problem. We can configure
{{yarn.timeline-service.enabled}} to be false and still start the AHS /
timelineserver, IIUC Its not used any where in the AHS / timelineserver while
starting up. Hence i interpreted it to be a client side config which tries to
indicate the yarnclient that i am trying to use the timelineserver. My initial
thoughts about this configuration was similar to your approach but it will have
flaws because just by another configuration we cannot guarantee that timeline
service is running in the cluster.
May be we can try to fail the RM startup if SMP is not able to connect to
Timelineserver,but IIUC [~jeagles] and [~jlowe] in other ATS 1.5 jira were
informing that cluster should run though the timelineserver is not running
hence its not desirable to fail the RM startup. Max we can do is to update the
document that *"yarn.resourcemanager.system-metrics-publisher.enabled"*
requires Timelineservice to be running
bq. but the way I view it is that it should act as a "master switch" for the
timeline service; i.e. the highest level switch that toggles the feature on and
off on all sides
This implies that when start the timeline service daemon then we need to check
for {{yarn.timeline-service.enabled}} and if false we need to get it down ?
but in none of the other daemons we have it in that way, so will that be ok ?
Also if the configurations used for the timelineserver has it but not for the
other daemons then again we face issue.
bq. Also, consider the fact that the system metrics publisher may not be the
only server-side component that interacts with the timeline service. There may
be others and there will be more with the timeline service v.2 (e.g. NM
collector service, etc.).
I agree to fact that we will be having lot of additions in the future versions,
but we will be having further configurations like ATS version and on top of
this having one more configuration like {{yarn.timeline-service.enabled}} will
it be of use ?.
These are my views but if we still want to go ahead with
{{yarn.timeline-service.enabled}} then we might need to come up with a *new
configuration* to indicate that client wants to use the timeline server hence
create the timeline client and the timelineserver delegation tokens. If as used
in the current approach then we will meet the issues as mentioned by
[~jeagles]. i.e. server configurations are copied to all clients and if
timeline server is enabled then delegation tokens is created. So each client
would require to explicitly reset the {{yarn.timeline-service.enabled}}
configuration to false if they don't want to use it.
> Enabling generic application history forces every job to get a timeline
> service delegation token
> ------------------------------------------------------------------------------------------------
>
> Key: YARN-4183
> URL: https://issues.apache.org/jira/browse/YARN-4183
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.1
> Reporter: Mit Desai
> Assignee: Mit Desai
> Attachments: YARN-4183.1.patch
>
>
> When enabling just the Generic History Server and not the timeline server,
> the system metrics publisher will not publish the events to the timeline
> store as it checks if the timeline server and system metrics publisher are
> enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force
> every yarn application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if
> application history server is enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
