Sangjin Lee commented on YARN-3051:

Varun, Li and I recently have a offline discussion. In general, we agreed on 
focusing on storage-oriented interface (raw data query) together with a FS 
implementation of it on this jira, but spinning off change about the 
user-oriented interface, web front wire up, and single reader daemon setup and 
dealing with them separately. The rationale is to roll out the reader interface 
fast, and we can work the HBase/Phoenix implement and web front wireup on a 
commonly agreed interface in parallel. How do you think about the plan?
Agreed with the approach. I would go so far as focusing on the raw data reader 
part first and get that done and get to the aggregated reader later. Thoughts?

Context is useful. Instead of creating a new one, maybe we can reuse the 
existing Context, which hosts more content than reader needs. So we just need 
to let reader put/get the required information to/from it.
It should be fine, as long as it is clear we don't need to fill in all the info 
for the read path.

+1 LGTM, but I think it's for the query of searching a set of qualified 
entities, right. For fetching a single entity, the query may look like 
(context) + (entity identifier) + (contents to retrieve)
Yes, I agree. One can think of the entity id is a special form of a "predicate" 
still. I'm not married to exactly one API; just the need to use a more 
coarse-grained approach.

Another issue I want to raise is that after our performance evaluation, we 
agreed on using HBase for raw data and Phoenix for aggregated data. It implies 
that we need to use HBase to implement the APIs for the raw entities, while use 
Phoenix to implement the APIs for the aggregated data.
We discussed this offline. We can have a couple of different approaches for 
this. We could either have separate reader APIs for raw data and (time-based) 
aggregated data. Or we could hide the separation behind a facade reader 
implementation that dispatches calls to a HBase reader impl for raw data and 
those to a phoenix impl for aggregated data. Either way, it should be pretty 

> [Storage abstraction] Create backing storage read interface for ATS readers
> ---------------------------------------------------------------------------
>                 Key: YARN-3051
>                 URL: https://issues.apache.org/jira/browse/YARN-3051
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>         Attachments: YARN-3051-YARN-2928.003.patch, 
> YARN-3051-YARN-2928.03.patch, YARN-3051-YARN-2928.04.patch, 
> YARN-3051.wip.02.YARN-2928.patch, YARN-3051.wip.patch, YARN-3051_temp.patch
> Per design in YARN-2928, create backing storage read interface that can be 
> implemented by multiple backing storage implementations.

This message was sent by Atlassian JIRA

Reply via email to