Varun Saxena commented on YARN-4224:

1. I understand we would like to have plural forms for listing (like /flows, 
/apps) and singular forms for detail (like /flow/{uid}). But then, why do we 
need both /runs/{uid} and /run/{uid}?
The same question also applies to apps. 
You mean just have one endpoint and based on delimiters in UID, decide whether 
to fetch single entity or multiple ?

For entities, we require both UID and type. Why type is not a part of UID 
(which means UID is not sufficient to identify an entity)?
Because, the query before query for entities, in which we return UID, will be 
query for apps. This query is from application table where we do not have 
entity related information. So I cannot send all the possible entity types for 
an entity or a list of entities in the response. Hence when we query list of 
entities, it is within the scope of app UID and entity type. Hence entity type 
has to be specified.

Or, are you planning to support operations like "list all entities in a given 
entity type"?
Yes, if we try to query all entity types and related entities, this would 
require scanning quite a bit of the entity table which can grow quite big. And 
from UI, I envisage only queries for APP_ATTEMPT and CONTAINER so we would know 
the entity type.

If it is the latter, then do we want to consider put type into query parameters 
on end point entities?
Normally in REST, mandatory params are kept as part of path. I expect entity 
type to be a mandatory param. I am assuming we do not want to support queries 
like get entities for all possible entity types. Thoughts ?

bq. For flows, why we' re not including an UID endpoint to locate one flow? 
This poses a challenge when we'd like to list all flow runs within one flow 
(or, do we have any other end points to do this work? ). 
When we query flows, we return all possible flow runs with it. So query for a 
single flow is not going to give any new information.
Moreover, runs endpoint will do it for you i.e. fetch all flowruns for a flow.

3. Seems like there is no full path to locate one entity from the cluster, 
user, flow, run, app, entity type, and entity id. Are we omitting this endpoint 
There is. The {{\/entity\/{uid}\/}} endpoint. I hope this is what your question 

As a side note, in this patch there are 3 types of "shortcuts" in the URL: omit 
the cluster id (with default cluster id), omit user id (with default user id) 
and directly access app id. I'm OK with direct accessing app ids (with cluster 
id), but do we want to omit the other two? Comments are more then welcome.
Can you elaborate a bit on this ?
We have 3 endpoints. One with cluster id, one without cluster id(default 
cluster from config is taken) and one with UID. For apps, the UID endpoint will 
contain flow context information.

bq. I'm also debating with myself on this. Right now I'm leaning towards to 
make the UIDs transparent to the storage layer.
I am fine either ways. I can see pros and cons for both.
Only concern I see is that if we are fetching a lot of entities and specify a 
high limit, we need to iterate over all the entities again to fill the UID.
If it is a mere 50-100 entities then should be negligible difference but what 
if its very high.
Another thing we need to ponder is that whether we need to support pagination 
for UIs' and if it would be possible to support it. Because then we will have 
to store some contextual information in reader. Or we can send some info back 
in response and continue from there for next pagination request. That is handle 
pagination by ourselves. Not sure if we can do this before 1st milestone though.

> Change the ATSv2 reader side REST interface to conform to current REST APIs' 
> in YARN
> ------------------------------------------------------------------------------------
>                 Key: YARN-4224
>                 URL: https://issues.apache.org/jira/browse/YARN-4224
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4224-YARN-2928.01.patch, 
> YARN-4224-feature-YARN-2928.wip.02.patch

This message was sent by Atlassian JIRA

Reply via email to