Zhijie Shen commented on YARN-2513:

[~jeagles], thanks for working on this feature, which seems to be interesting. 
The benefit I can see with integrating the per-framework UI widget into the 
timeline server is to relieve the framework from deploying its own web server, 
and to prevent additional data move from the timeline server to the framework 
web server, and finally to the user.

To enable the web plugin, so far I can think of the following things that it's 
good to address.

1. Security: there're three potential issues. a) Do we want to use Hadoop http 
authentication to protect the pre-framework web plugin? b) We don't page-level 
or url matching access authorization, such that, for example, UserA can only 
access the web pages of it's authorized web plugin. c) Currently, everything 
inside timeline server belongs to YARN, such that we don't limit its access to 
the resources internally. Web plugin is hosted by the timeline server on behalf 
of the framework, and it should only have the access to the resources granted 
to it. For example, Tez webUI should only have access to the tez metrics in the 
timeline store.

2. Isolation: for some reason, the web plugin is going to crash, we should make 
sure it is not going to affect other components in the timeline server and 
other web plugins. Moreover, if multiple web plugins are hosted in the timeline 
server, we need to take care of their competing for the web server resources, 
preventing starvation.

3. Scalability: now everything is hosted in a single web server container. 
Hosting framework UIs will drive more traffic to the timeline server. We may 
want to scale up web server instances to handle users' requests. In addition, 
it's good think of whether we want distribute the workload according the 
functions: some of them serves raw REST APIs, others serve web UI of framework 
1, 2, 3 and the remaining ones serve web UI of framework 4, 5, 6.

> Host framework UIs in YARN for use with the ATS
> -----------------------------------------------
>                 Key: YARN-2513
>                 URL: https://issues.apache.org/jira/browse/YARN-2513
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Jonathan Eagles
>            Assignee: Jonathan Eagles
>         Attachments: YARN-2513-v1.patch
> Allow for pluggable UIs as described by TEZ-8. Yarn can provide the 
> infrastructure to host java script and possible java UIs.

This message was sent by Atlassian JIRA

Reply via email to