Sangjin Lee commented on YARN-4183:

bq. May be we can try to fail the RM startup if SMP is not able to connect to 
Timelineserver,but IIUC Jonathan Eagles and Jason Lowe in other ATS 1.5 jira 
were informing that cluster should run though the timelineserver is not running 
hence its not desirable to fail the RM startup. 

I'm *not* suggesting that we fail the RM if the SMP cannot connect to the 
timeline server. I'm suggesting we can *disable* the SMP (as opposed to having 
continuous failures or failing the RM) if we know the timeline service is 
disabled. That's why the config helps, as it is the clearly indicated intent 
for running the cluster.

As you said, it is possible that yarn.timeline-service.enabled is set to false 
even if the timeline service is up and vice versa. But I think we should assume 
a reasonable use case here where it is fair to expect the timeline service to 
be there if yarn.timeline-service.enabled is true. If that config is true but 
the timeline service is not up, then I think it is acceptable to see continuous 
timeline service failures (this is the existing behavior btw).

bq. if we still want to go ahead with yarn.timeline-service.enabled then we 
might need to come up with a new configuration to indicate that client wants to 
use the timeline server hence create the timeline client and the timelineserver 
delegation tokens.

Yes, I agree. It's implied that we would need a different config/mechanism for 
disabling getting the delegation token. I would advocate having a separate 
client-side config. Whether the server has enabled the timeline service and 
whether a particular client/app will use it are separate concerns, and separate 
configs should drive them. The current state of tez and MR is a good example. 
MR should be able to say I won't use the timeline service even if it is 
available. Any MR code that uses the timeline service should basically check 
both configs; i.e. the timeline service should be enabled and its config to use 
the timeline service should be true.

This might be an idealistic argument on my part, but I think doing something 
along that line would lead to cleaner separation of concerns and larger degrees 
of freedom. My 2 cents.

> Enabling generic application history forces every job to get a timeline 
> service delegation token
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-4183
>                 URL: https://issues.apache.org/jira/browse/YARN-4183
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>            Reporter: Mit Desai
>            Assignee: Mit Desai
>         Attachments: YARN-4183.1.patch
> When enabling just the Generic History Server and not the timeline server, 
> the system metrics publisher will not publish the events to the timeline 
> store as it checks if the timeline server and system metrics publisher are 
> enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force 
> every yarn application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if 
> application history server is enabled.

This message was sent by Atlassian JIRA

Reply via email to