Naganarasimha G R commented on YARN-4183:

Hi [~sjlee0],
Thanks for sharing your thoughts
bq. If the answer is "yarn.resourcemanager.system-metrics-publisher.enabled 
should not be set to true if the timeline service is disabled", then it only 
makes it clear that yarn.resourcemanager.system-metrics-publisher.enabled=true 
implies yarn.timeline-service.enabled=true. Then we should check it explicitly. 
{{yarn.timeline-service.enabled}} doesn't imply that timeline service is 
running as we can enable it and still if the timeline server is not running 
then we face the same problem. We can configure 
{{yarn.timeline-service.enabled}} to be false and still start the AHS / 
timelineserver, IIUC Its not used any where in the AHS / timelineserver while 
starting up. Hence i interpreted it to be a client side config which tries to 
indicate the yarnclient that i am trying to use the timelineserver. My initial 
thoughts about this configuration was similar to your approach but it will have 
flaws because just by another configuration we cannot guarantee that timeline 
service is running in the cluster. 
May be we can try to fail the RM startup if SMP is not able to connect to 
Timelineserver,but IIUC  [~jeagles] and [~jlowe] in other ATS 1.5 jira were 
informing that cluster should run though the timelineserver is not running 
hence its not desirable to fail the RM startup. Max we can do is to update the 
document that *"yarn.resourcemanager.system-metrics-publisher.enabled"* 
requires Timelineservice to be running 

bq. but the way I view it is that it should act as a "master switch" for the 
timeline service; i.e. the highest level switch that toggles the feature on and 
off on all sides
This implies that when start the timeline service daemon then we need to check 
for  {{yarn.timeline-service.enabled}} and if false we need to get it down ? 
but in none of the other daemons we have it in that way, so will that be ok ? 
Also if the configurations used for the timelineserver has it but not for the 
other daemons then again we face issue.

bq. Also, consider the fact that the system metrics publisher may not be the 
only server-side component that interacts with the timeline service. There may 
be others and there will be more with the timeline service v.2 (e.g. NM 
collector service, etc.).
I agree to fact that we will be having lot of additions in the future versions, 
but we will be having further configurations like ATS version and on top of 
this having one more configuration like {{yarn.timeline-service.enabled}} will 
it be of use ?.

These are my views but if we still want to go ahead with  
{{yarn.timeline-service.enabled}} then we might need to come up with a *new 
configuration* to indicate that client wants to use the timeline server hence 
create the timeline client and the timelineserver delegation tokens. If as used 
in the current approach then we will meet the issues as mentioned by 
[~jeagles]. i.e. server configurations are copied to all clients and if 
timeline server is enabled then delegation tokens is created. So each client 
would require to explicitly reset the {{yarn.timeline-service.enabled}} 
configuration to false if they don't want to use it.

> Enabling generic application history forces every job to get a timeline 
> service delegation token
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-4183
>                 URL: https://issues.apache.org/jira/browse/YARN-4183
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>            Reporter: Mit Desai
>            Assignee: Mit Desai
>         Attachments: YARN-4183.1.patch
> When enabling just the Generic History Server and not the timeline server, 
> the system metrics publisher will not publish the events to the timeline 
> store as it checks if the timeline server and system metrics publisher are 
> enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force 
> every yarn application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if 
> application history server is enabled.

This message was sent by Atlassian JIRA

Reply via email to