Vinod Kumar Vavilapalli created YARN-5079:
---------------------------------------------
Summary: [Umbrella] Native YARN framework layer for services and
beyond
Key: YARN-5079
URL: https://issues.apache.org/jira/browse/YARN-5079
Project: Hadoop YARN
Issue Type: New Feature
Reporter: Vinod Kumar Vavilapalli
Assignee: Vinod Kumar Vavilapalli
(See overview doc at YARN-4692, modifying and copy-pasting some of the relevant
pieces and sub-section 3.3.1 to track the specific sub-item.)
(This is a companion to YARN-4793 in our effort to simplify the entire story,
but focusing on APIs)
So far, YARN by design has restricted itself to having a very low-level API
that can support any type of application. Frameworks like Apache Hadoop
MapReduce, Apache Tez, Apache Spark, Apache REEF, Apache Twill, Apache Helix
and others ended up exposing higher level APIs that end-users can directly
leverage to build their applications on top of YARN. On the services side,
Apache Slider has done something similar.
With our current attention on making services first-class and simplified, it's
time to take a fresh look at how we can make Apache Hadoop YARN support
services well out of the box. Beyond the functionality that I outlined in the
previous sections in the doc on how NodeManagers can be enhanced to help
services, the biggest missing piece is the framework itself. There is a lot of
very important functionality that a services' framework can own together with
YARN in executing services end-to-end.
In this JIRA I propose we look at having a native Apache Hadoop framework for
running services natively on YARN.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
