[ 
https://issues.apache.org/jira/browse/YARN-4224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059420#comment-15059420
 ] 

Sangjin Lee commented on YARN-4224:
-----------------------------------

I'm a little late to this thread, and have just started to catch up on the 
discussion, so I might be off-base. But let me ask some questions and also 
chime in on some points.

I actually thought that [Li's original 
proposal|https://issues.apache.org/jira/browse/YARN-4224?focusedCommentId=15052865&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15052865]
 (2nd approach in that comment) was quite good. Although it can be a little 
verbose, it's strongly resource-based and very consistent. Also, I liked the 
fact that some parts of the path may be omitted (e.g. users) if it can be 
inferred from other information (which we already support). With that *single* 
rule, it can describe just about any URL here. It can't be any clearer than 
that.

As for the proposal for creating a single token by concatenating cluster, user, 
and the flow name, is the goal basically to avoid multiple levels in the URL 
path (to aid the UI implementation)? Then what about the flow run id? For 
example, let's consider querying for all apps in a flow run. With Li's original 
proposal, it would be

{noformat}
/ws/v2/timeline/clusters/yarn_cluster/users/admin/flows/hive_flow/runs/123/apps
{noformat}
(I swapped cluster and user)

With the concatenation proposal, it would become

{noformat}
/ws/v2/timeline/yarn_cluster_admin_hive_flow/runs/123/apps
{noformat}

(As Varun pointed out, there is the sticky issue of escaping the concatenation 
character, but let's set that aside for the moment) We still have "/runs/123" 
appended after that UID. Is that going to be fine with the UI implementation? 
Or does that also need to be concatenated so that we have something like

{noformat}
/ws/v2/timeline/yarn_cluster_admin_hive_flow_123/apps
{noformat}
?

I think this has a potential of making things a lot more complicated than is 
needed. I don’t think it’s easy or desirable to flatten everything into a 
single token at all times. Also, how about cases where some part of the 
information can be omitted (e.g. user, flow name, flow run, etc.)? Then how 
should we form the UID? Would we require user/UI to always specify all the 
parts? It may not be always feasible.

Also, I'm not sure if I like the idea of having an end point just to return the 
UID given the bits. It would make the workflow a lot more complicated (the 
client needs this handshake before it can start querying), and I'm not sure 
what we gain by hiding that part into the server. If we were to do this, I 
think we might as well make this a public piece of information so any user or 
UI can compose it quickly. That would make things a whole lot easier to 
implement on both sides.

How about the following proposal? Can we adopt Li's original proposal and also 
support the UID-based pattern to aid the UI? The UID can be considered more 
like a short-hand notation, but would *complement (not replace)* the basic 
REST-style pattern. But we should clearly spell out under which condition such 
concatenated UIDs are supported in order to eliminate any ambiguity (e.g. only 
"cluster+user+flowname", or "entity_type+entity_id" too). It shouldn't be too 
difficult for the server to support both modes, and we would retain most of the 
simplicity that Li's first proposal has yet be able to facilitate the UI 
implementation.

What do you think?

> Change the ATSv2 reader side REST interface to conform to current REST APIs' 
> in YARN
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-4224
>                 URL: https://issues.apache.org/jira/browse/YARN-4224
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-4224-YARN-2928.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to