Sangjin Lee commented on YARN-4183:

This is also somewhat relevant to YARN-4356.

I'd like to point out an interesting possibility raised in another JIRA by 
[~jrottinghuis]. With v.2 and especially early on with v.2, it would be rather 
useful to be able to enable both v.1 (or v.1.5) and v.2. That would provide a 
useful verification and comparison environment with a single cluster. The way 
it's being discussed right now, it sounds like the version would be a single 
value (mutually exclusive). Wouldn't it be good to have a possibility to be 
able to enable more than one version? Thoughts?

Although the documentation appears to suggest the use is primarily for clients, 
I believe it strongly implies it applies to the server-side too. After all, 
even if it is indicated to the client that the timeline service is enabled, it 
would be no use obviously (and actually worse than not indicating) if the 
timeline service was not running. I think it's a completely compatible 
interpretation it also means strongly that the server-side can interpret this 
as an instruction to enable all things timeline service.

bq. if the timelineserver daemon is started it directly starts the 
timelinestore without checking for the configuration 

The timelineserver daemon might be able to do it, but the issue is with any 
other server-side component (e.g. RM, etc.) that needs to see if the timeline 
server should be supported/used. IMO it should be checked by everything 
(server-side and client-side) to see if the timeline service is and should be 

I also think ideally each framework (client) should define whether they will 
use the timeline service. Just because the timeline service is enabled doesn't 
mean they will use it (like MR today). Ideally the framework should have its 
own config to use it. Any code of theirs to use the timeline service should be 
predicated on *both* properties being true.

Regarding versions, as long as clients can discover that the version they want 
to use is included in the enabled versions (see above) and if their config is 
also enabled, it should be able to write/read (provided the APIs exist of 


bq. IIUC, the newly proposed yarn.timeline-service.version supports a sanity 
check mechanism: each API should check if the current running ATS's version is 
equal to or higher than it's required version.

I'm not entirely sure if this is feasible. "Higher" doesn't necessarily mean it 
will support all versions at and below its own. For example, I don't think we 
can make a guarantee that v.2 will be able to support all queries for v.1 and 
v.1.5. What's more appropriate is *an exact match*. IMO we should not make 
guarantees about lower version compatibility as it's going to be very 
challenging to pull off and constraining for the newer implementation.

> Enabling generic application history forces every job to get a timeline 
> service delegation token
> ------------------------------------------------------------------------------------------------
>                 Key: YARN-4183
>                 URL: https://issues.apache.org/jira/browse/YARN-4183
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.1
>            Reporter: Mit Desai
>            Assignee: Mit Desai
>         Attachments: YARN-4183.1.patch
> When enabling just the Generic History Server and not the timeline server, 
> the system metrics publisher will not publish the events to the timeline 
> store as it checks if the timeline server and system metrics publisher are 
> enabled before creating a timeline client.
> To make it work, if the timeline service flag is turned on, it will force 
> every yarn application to get a delegation token.
> Instead of checking if timeline service is enabled, we should be checking if 
> application history server is enabled.

This message was sent by Atlassian JIRA

Reply via email to