Li Lu updated YARN-4265:
    Attachment: YARN-4265.YARN-4234.001.patch

I refactored my code and added some unit tests. The current patch only depends 
on YARN-4234. I addressed comments raised by [~jlowe] in the previous round of 
review, with two pending actions that I think we can address in separate JIRAs:
# Build a "fall-back" plugin to have the same behavior of YARN-3942. 
# Make caching storage pluggable. 

For the second item, right now my patch does _not_ depend on YARN-4219. So I'm 
setting the v1.5 plugin storage to use the memory storage system as the 
"caching" storage, similar to YARN-3942.

I'm adding the v1.5 plugin storage (EntityGroupFSTimelineStore) in a module 
called hadoop-yarn-server-timeline-pluginstorage. This is slightly different to 
YARN-3942. We need a separate module because the new v1.5 storage depends on 
yarn-client, but we don't want ats server depends on yarn-client (The v1.5 
storage is not a purely server side storage.). I'm naming it as "pluginstorage" 
because I'm considering to put the leveldb caching storage into this extension 
as well. 

> Provide new timeline plugin storage to support fine-grained entity caching
> --------------------------------------------------------------------------
>                 Key: YARN-4265
>                 URL: https://issues.apache.org/jira/browse/YARN-4265
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-4265-trunk.poc_001.patch, 
> YARN-4265.YARN-4234.001.patch
> To support the newly proposed APIs in YARN-4234, we need to create a new 
> plugin timeline store. The store may have similar behavior as the 
> EntityFileTimelineStore proposed in YARN-3942, but cache date in cache id 
> granularity, instead of application id granularity. Let's have this storage 
> as a standalone one, instead of updating EntityFileTimelineStore, to keep the 
> existing store (EntityFileTimelineStore) stable. 

This message was sent by Atlassian JIRA

Reply via email to