Junping Du commented on YARN-4265:

Thanks [~gtCarrera9] for reply and updating the patch. 
I am still reviewing the new patch but have some quick response below:
bq.  Plugins are provided by third-party applications such as Tez. Right now we 
cannot assume which exact entity group plugin the user is using, therefore we 
have to conservatively leave this config as empty. 
Thanks for clarification on this. Can we provide a default one (could be 
simple) for built-in YARN applications, i.e. distributed shell? This can be 
helpful to demonstrate how to use it for applications other than Tez. Or other 
apps have to refer Tez code on how to use it. IMO, this is not a very good 
practice for YARN.

bq. we're using fine-grained locking for each cached item in the reader cache, 
and cache refresh only happens infrequently (~10 secs by default), so maybe 
we'd like to stabilize the whole synchronization story before fine tune this 
Sounds like a plan. Looks like we still need to make effort on checking the 
whole synchronization mechanism. Isn't it?

> Provide new timeline plugin storage to support fine-grained entity caching
> --------------------------------------------------------------------------
>                 Key: YARN-4265
>                 URL: https://issues.apache.org/jira/browse/YARN-4265
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-4265-trunk.001.patch, YARN-4265-trunk.002.patch, 
> YARN-4265.YARN-4234.001.patch, YARN-4265.YARN-4234.002.patch
> To support the newly proposed APIs in YARN-4234, we need to create a new 
> plugin timeline store. The store may have similar behavior as the 
> EntityFileTimelineStore proposed in YARN-3942, but cache date in cache id 
> granularity, instead of application id granularity. Let's have this storage 
> as a standalone one, instead of updating EntityFileTimelineStore, to keep the 
> existing store (EntityFileTimelineStore) stable. 

This message was sent by Atlassian JIRA

Reply via email to