Modified: incubator/reef/site/faq.html URL: http://svn.apache.org/viewvc/incubator/reef/site/faq.html?rev=1700380&r1=1700379&r2=1700380&view=diff ============================================================================== --- incubator/reef/site/faq.html (original) +++ incubator/reef/site/faq.html Tue Sep 1 02:08:00 2015 @@ -1,496 +1,496 @@ -<!DOCTYPE html> -<!-- - | Generated by Apache Maven Doxia at 2015-08-20 - | Rendered using Apache Maven Fluido Skin 1.4 ---> -<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> - <head> - <meta charset="UTF-8" /> - <meta name="viewport" content="width=device-width, initial-scale=1.0" /> - <meta name="Date-Revision-yyyymmdd" content="20150820" /> - <meta http-equiv="Content-Language" content="en" /> - <title>Apache REEF - FAQ</title> - <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" /> - <link rel="stylesheet" href="./css/site.css" /> - <link rel="stylesheet" href="./css/print.css" media="print" /> - - - <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script> - - - -<script src="js/release.js" type="text/javascript"></script> - - </head> - <body class="topBarEnabled"> - - - - - - <div id="topbar" class="navbar navbar-fixed-top "> - <div class="navbar-inner"> - <div class="container-fluid"> - <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar"> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - <span class="icon-bar"></span> - </a> - - <ul class="nav"> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache REEF <b class="caret"></b></a> - <ul class="dropdown-menu"> - - <li> <a href="index.html" title="Overview">Overview</a> -</li> - - <li> <a href="faq.html" title="FAQ">FAQ</a> -</li> - - <li> <a href="license.html" title="License">License</a> -</li> - - <li> <a href="downloads.html" title="Downloads">Downloads</a> -</li> - </ul> - </li> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> - <ul class="dropdown-menu"> - - <li> <a href="introduction.html" title="Introduction to REEF">Introduction to REEF</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Tutorials"; title="REEF Tutorial">REEF Tutorial</a> -</li> - - <li> <a href="talks.html" title="Talks">Talks</a> -</li> - - <li> <a href="glossary.html" title="Glossary">Glossary</a> -</li> - - <li> <a href="tang.html" title="Tang">Tang</a> -</li> - - <li> <a href="wake.html" title="Wake">Wake</a> -</li> - - <li> <a href="apidocs/0.10.0-incubating/index.html" title="0.10.0-incubating API">0.10.0-incubating API</a> -</li> - - <li> <a href="apidocs/0.11.0-incubating/index.html" title="0.11.0-incubating API">0.11.0-incubating API</a> -</li> - - <li> <a href="apidocs/0.12.0-incubating/index.html" title="0.12.0-incubating API">0.12.0-incubating API</a> -</li> - </ul> - </li> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown">Contribution <b class="caret"></b></a> - <ul class="dropdown-menu"> - - <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Contributing"; title="Contributing">Contributing</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Committer+Guide"; title="Committer Guide">Committer Guide</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Coding+Guidelines"; title="Coding Guidelines">Coding Guidelines</a> -</li> - </ul> - </li> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a> - <ul class="dropdown-menu"> - - <li> <a href="team.html" title="Team">Team</a> -</li> - - <li> <a href="mailing-list.html" title="Mailing List">Mailing List</a> -</li> - - <li> <a href="https://issues.apache.org/jira/browse/REEF"; title="Issue Tracker">Issue Tracker</a> -</li> - - <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Powered+by+REEF"; title="Powered By">Powered By</a> -</li> - </ul> - </li> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> - <ul class="dropdown-menu"> - - <li> <a href="http://www.apache.org/foundation/"; title="Apache Software Foundation">Apache Software Foundation</a> -</li> - - <li> <a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a> -</li> - - <li> <a href="http://incubator.apache.org/"; title="Apache Incubator">Apache Incubator</a> -</li> - - <li> <a href="http://www.apache.org/licenses/LICENSE-2.0.html"; title="Apache License">Apache License</a> -</li> - - <li> <a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsorship">Sponsorship</a> -</li> - - <li> <a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a> -</li> - </ul> - </li> - </ul> - - - - <ul class="nav pull-right"> - <li class="dropdown"> - <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a> - <ul class="dropdown-menu"> - <li> <a href="https://github.com/apache/incubator-reef"; title="Apache REEF GitHub">Apache REEF GitHub</a> -</li> - <li> <a href="http://www.apache.org"; title="Apache">Apache</a> -</li> - <li> <a href="http://incubator.apache.org/"; title="Apache Incubator">Apache Incubator</a> -</li> - </ul> - </li> - </ul> - - </div> - - </div> - </div> - </div> - - <div class="container-fluid"> - <div id="banner"> - <div class="pull-left"> - <a href="./" id="bannerLeft" title="Apache REEF"> - <img src="ApacheREEF_logo_no_margin_small.png" alt="Apache REEF"/> - </a> - </div> - <div class="pull-right"> <a href="http://incubator.apache.org/"; id="bannerRight"> - <img src="ApacheIncubator.png" /> - </a> - </div> - <div class="clear"><hr/></div> - </div> - - <div id="breadcrumbs"> - <ul class="breadcrumb"> - - - <li class=""> - <a href="index.html" title="Apache REEF"> - Apache REEF</a> - <span class="divider">|</span> - </li> - <li class="active ">FAQ</li> - - - - - </ul> - </div> - - - <div class="row-fluid"> - <div id="leftColumn" class="span2"> - <div class="well sidebar-nav"> - - - <ul class="nav nav-list"> - <li class="nav-header">Apache REEF</li> - - <li> - - <a href="index.html" title="Overview"> - <span class="none"></span> - Overview</a> - </li> - - <li class="active"> - - <a href="#"><span class="none"></span>FAQ</a> - </li> - - <li> - - <a href="license.html" title="License"> - <span class="none"></span> - License</a> - </li> - - <li> - - <a href="downloads.html" title="Downloads"> - <span class="none"></span> - Downloads</a> - </li> - <li class="nav-header">Documentation</li> - - <li> - - <a href="introduction.html" title="Introduction to REEF"> - <span class="none"></span> - Introduction to REEF</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/REEF/Tutorials"; class="externalLink" title="REEF Tutorial"> - <span class="none"></span> - REEF Tutorial</a> - </li> - - <li> - - <a href="talks.html" title="Talks"> - <span class="none"></span> - Talks</a> - </li> - - <li> - - <a href="glossary.html" title="Glossary"> - <span class="none"></span> - Glossary</a> - </li> - - <li> - - <a href="tang.html" title="Tang"> - <span class="none"></span> - Tang</a> - </li> - - <li> - - <a href="wake.html" title="Wake"> - <span class="none"></span> - Wake</a> - </li> - - <li> - - <a href="apidocs/0.10.0-incubating/index.html" title="0.10.0-incubating API"> - <span class="none"></span> - 0.10.0-incubating API</a> - </li> - - <li> - - <a href="apidocs/0.11.0-incubating/index.html" title="0.11.0-incubating API"> - <span class="none"></span> - 0.11.0-incubating API</a> - </li> - - <li> - - <a href="apidocs/0.12.0-incubating/index.html" title="0.12.0-incubating API"> - <span class="none"></span> - 0.12.0-incubating API</a> - </li> - <li class="nav-header">Contribution</li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/REEF/Contributing"; class="externalLink" title="Contributing"> - <span class="none"></span> - Contributing</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/REEF/Committer+Guide"; class="externalLink" title="Committer Guide"> - <span class="none"></span> - Committer Guide</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/REEF/Coding+Guidelines"; class="externalLink" title="Coding Guidelines"> - <span class="none"></span> - Coding Guidelines</a> - </li> - <li class="nav-header">Community</li> - - <li> - - <a href="team.html" title="Team"> - <span class="none"></span> - Team</a> - </li> - - <li> - - <a href="mailing-list.html" title="Mailing List"> - <span class="none"></span> - Mailing List</a> - </li> - - <li> - - <a href="https://issues.apache.org/jira/browse/REEF"; class="externalLink" title="Issue Tracker"> - <span class="none"></span> - Issue Tracker</a> - </li> - - <li> - - <a href="https://cwiki.apache.org/confluence/display/REEF/Powered+by+REEF"; class="externalLink" title="Powered By"> - <span class="none"></span> - Powered By</a> - </li> - <li class="nav-header">ASF</li> - - <li> - - <a href="http://www.apache.org/foundation/"; class="externalLink" title="Apache Software Foundation"> - <span class="none"></span> - Apache Software Foundation</a> - </li> - - <li> - - <a href="http://www.apache.org/foundation/how-it-works.html"; class="externalLink" title="How Apache Works"> - <span class="none"></span> - How Apache Works</a> - </li> - - <li> - - <a href="http://incubator.apache.org/"; class="externalLink" title="Apache Incubator"> - <span class="none"></span> - Apache Incubator</a> - </li> - - <li> - - <a href="http://www.apache.org/licenses/LICENSE-2.0.html"; class="externalLink" title="Apache License"> - <span class="none"></span> - Apache License</a> - </li> - - <li> - - <a href="http://www.apache.org/foundation/sponsorship.html"; class="externalLink" title="Sponsorship"> - <span class="none"></span> - Sponsorship</a> - </li> - - <li> - - <a href="http://www.apache.org/foundation/thanks.html"; class="externalLink" title="Thanks"> - <span class="none"></span> - Thanks</a> - </li> - </ul> - - - - <hr /> - - <div id="poweredBy"> - <div class="clear"></div> - <div class="clear"></div> - <div class="clear"></div> - <div class="clear"></div> - <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> - <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> - </a> - </div> - </div> - </div> - - - <div id="bodyColumn" class="span10" > - - <!-- Licensed to the Apache Software Foundation (ASF) under one -or more contributor license agreements. See the NOTICE file -distributed with this work for additional information -regarding copyright ownership. The ASF licenses this file -to you under the Apache License, Version 2.0 (the -"License"); you may not use this file except in compliance -with the License. You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, -software distributed under the License is distributed on an -"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -KIND, either express or implied. See the License for the -specific language governing permissions and limitations -under the License. --><h1>FAQ</h1> - -<ol style="list-style-type: decimal"> - -<li><a href="#who">Who is REEF for?</a></li> - -<li><a href="#why">Why did it come about?</a></li> - -<li><a href="#tang">What is Tang?</a></li> - -<li><a href="#wake">What is Wake?</a></li> - -<li><a href="#how">How can I get started?</a></li> -</ol> -<div class="section"> -<div class="section"> -<h3><a name="who"></a>Who is REEF for?<a name="Who_is_REEF_for"></a></h3> -<p>REEF is for developers of data processing systems on cloud computing platforms that provide fine-grained resource allocations. REEF provides system authors with a centralized (pluggable) control flow that embeds a user-defined system controller called the Job Driver. The interfaces associated with the Job Driver are event driven; events signal resource allocations and failures, various states associated with task executions and communication channels, alarms based on wall-clock or logical time, and so on. REEF also aims to package a variety of data-processing libraries (e.g., high-bandwidth shuffle, relational operators, low-latency group communication, etc.) in a reusable form. Authors of big data systems and toolkits can leverage REEF to immediately begin development of their application specific data flow, while reusing packaged libraries where they make sense.</p> -<hr /></div> -<div class="section"> -<h3><a name="why"></a>Why did it come about?<a name="Why_did_it_come_about"></a></h3> -<p>Traditional data-processing systems are built around a single programming model (like SQL or MapReduce) and a runtime (query) engine. These systems assume full ownership over the machine resources used to execute compiled queries. For example, Hadoop (version one) supports the MapReduce programming model, which is used to express jobs that execute a map step followed by an optional reduce step. Each step is carried out by some number of parallel tasks. The Hadoop runtime is built on a single master (the JobTracker) that schedules map and reduce tasks on a set of workers (TaskTrackers) that expose fixed-sized task “slots”. This design leads to three key problems in Hadoop: </p> - -<ol style="list-style-type: decimal"> - -<li>The resources tied to a TaskTracker are provisioned for MapReduce only.</li> - -<li>Clients must speak some form of MapReduce in order to make use of cluster resources, and in turn, gain compute access to the data that lives there.</li> - -<li>Poor cluster utilization, especially in the case of idle resources (slots) due to straggler tasks.</li> -</ol> -<p>With YARN (Hadoop version two), resource management has been decoupled from the MapReduce programming model in Hadoop, freeing cluster resources from slotted formats, and opening the door to programming frameworks beyond MapReduce. It is well understood that while enticingly simple and fault-tolerant, the MapReduce model is not ideal for many applications, especially iterative or recursive workloads like machine learning and graph processing, and those that tremendously benefit from main memory (as opposed to disk based) computation. A variety of big data systems stem from this insight: Microsoft’s Dryad, Apache Spark, Google’s Pregel, CMU’s GraphLab and UCI’s AsterixDB, to name a few. Each of these systems add unique capabilities, but form islands around key functionalities, making it hard to share both data and compute resources between them. YARN, and related resource managers, move us one step closer toward a unified Big Data system stack. The goal of REEF is to provide the next level of detail in this layering.</p> -<hr /></div> -<div class="section"> -<h3><a name="tang"></a>What is Tang?<a name="What_is_Tang"></a></h3> -<p>Tang is a dependency injection Framework co-developed with REEF. It has extensive documentation which can be found <a href="tang.html">here</a>.</p> -<hr /></div> -<div class="section"> -<h3><a name="wake"></a>What is Wake?<a name="What_is_Wake"></a></h3> -<p>Please refer to <a href="wake.html">this</a> section.</p> -<hr /></div> -<div class="section"> -<h3><a name="how"></a>How can I get started?<a name="How_can_I_get_started"></a></h3> -<p>Check out the <a class="externalLink" href="https://cwiki.apache.org/confluence/display/REEF/Tutorials";>REEF Tutorial</a> and the <a class="externalLink" href="https://cwiki.apache.org/confluence/display/REEF/Contributing";>Contributing</a> page and join the community!</p></div></div> - </div> - </div> - </div> - - <hr/> - - <footer> - <div class="container-fluid"> - <div class="row-fluid"> - -<div class="container-fluid"> - <div class="row-fluid"> - <a href="http://www.apache.org";>Apache Software Foundation</a> - </div> -</div> - -<div class="row span12">Apache REEF, REEF, Apache, the Apache feather logo, and the Apache REEF logo are trademarks - of The Apache Software Foundation. All other marks mentioned may be trademarks or registered - trademarks of their respective owners.</div> - </div> - - - </div> - </footer> - </body> -</html> \ No newline at end of file +<!DOCTYPE html> +<!-- + | Generated by Apache Maven Doxia at 2015-09-01 + | Rendered using Apache Maven Fluido Skin 1.4 +--> +<html xmlns="http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> + <head> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1.0" /> + <meta name="Date-Revision-yyyymmdd" content="20150901" /> + <meta http-equiv="Content-Language" content="en" /> + <title>Apache REEF - FAQ</title> + <link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" /> + <link rel="stylesheet" href="./css/site.css" /> + <link rel="stylesheet" href="./css/print.css" media="print" /> + + + <script type="text/javascript" src="./js/apache-maven-fluido-1.4.min.js"></script> + + + +<script src="js/release.js" type="text/javascript"></script> + + </head> + <body class="topBarEnabled"> + + + + + + <div id="topbar" class="navbar navbar-fixed-top "> + <div class="navbar-inner"> + <div class="container-fluid"> + <a data-target=".nav-collapse" data-toggle="collapse" class="btn btn-navbar"> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </a> + + <ul class="nav"> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">Apache REEF <b class="caret"></b></a> + <ul class="dropdown-menu"> + + <li> <a href="index.html" title="Overview">Overview</a> +</li> + + <li> <a href="faq.html" title="FAQ">FAQ</a> +</li> + + <li> <a href="license.html" title="License">License</a> +</li> + + <li> <a href="downloads.html" title="Downloads">Downloads</a> +</li> + </ul> + </li> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">Documentation <b class="caret"></b></a> + <ul class="dropdown-menu"> + + <li> <a href="introduction.html" title="Introduction to REEF">Introduction to REEF</a> +</li> + + <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Tutorials"; title="REEF Tutorial">REEF Tutorial</a> +</li> + + <li> <a href="talks.html" title="Talks">Talks</a> +</li> + + <li> <a href="glossary.html" title="Glossary">Glossary</a> +</li> + + <li> <a href="tang.html" title="Tang">Tang</a> +</li> + + <li> <a href="wake.html" title="Wake">Wake</a> +</li> + + <li> <a href="apidocs/0.10.0-incubating/index.html" title="0.10.0-incubating API">0.10.0-incubating API</a> +</li> + + <li> <a href="apidocs/0.11.0-incubating/index.html" title="0.11.0-incubating API">0.11.0-incubating API</a> +</li> + + <li> <a href="apidocs/0.12.0-incubating/index.html" title="0.12.0-incubating API">0.12.0-incubating API</a> +</li> + </ul> + </li> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">Contribution <b class="caret"></b></a> + <ul class="dropdown-menu"> + + <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Contributing"; title="Contributing">Contributing</a> +</li> + + <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Committer+Guide"; title="Committer Guide">Committer Guide</a> +</li> + + <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Coding+Guidelines"; title="Coding Guidelines">Coding Guidelines</a> +</li> + </ul> + </li> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">Community <b class="caret"></b></a> + <ul class="dropdown-menu"> + + <li> <a href="team.html" title="Team">Team</a> +</li> + + <li> <a href="mailing-list.html" title="Mailing List">Mailing List</a> +</li> + + <li> <a href="https://issues.apache.org/jira/browse/REEF"; title="Issue Tracker">Issue Tracker</a> +</li> + + <li> <a href="https://cwiki.apache.org/confluence/display/REEF/Powered+by+REEF"; title="Powered By">Powered By</a> +</li> + </ul> + </li> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">ASF <b class="caret"></b></a> + <ul class="dropdown-menu"> + + <li> <a href="http://www.apache.org/foundation/"; title="Apache Software Foundation">Apache Software Foundation</a> +</li> + + <li> <a href="http://www.apache.org/foundation/how-it-works.html"; title="How Apache Works">How Apache Works</a> +</li> + + <li> <a href="http://incubator.apache.org/"; title="Apache Incubator">Apache Incubator</a> +</li> + + <li> <a href="http://www.apache.org/licenses/LICENSE-2.0.html"; title="Apache License">Apache License</a> +</li> + + <li> <a href="http://www.apache.org/foundation/sponsorship.html"; title="Sponsorship">Sponsorship</a> +</li> + + <li> <a href="http://www.apache.org/foundation/thanks.html"; title="Thanks">Thanks</a> +</li> + </ul> + </li> + </ul> + + + + <ul class="nav pull-right"> + <li class="dropdown"> + <a href="#" class="dropdown-toggle" data-toggle="dropdown">External Links <b class="caret"></b></a> + <ul class="dropdown-menu"> + <li> <a href="https://github.com/apache/incubator-reef"; title="Apache REEF GitHub">Apache REEF GitHub</a> +</li> + <li> <a href="http://www.apache.org"; title="Apache">Apache</a> +</li> + <li> <a href="http://incubator.apache.org/"; title="Apache Incubator">Apache Incubator</a> +</li> + </ul> + </li> + </ul> + + </div> + + </div> + </div> + </div> + + <div class="container-fluid"> + <div id="banner"> + <div class="pull-left"> + <a href="./" id="bannerLeft" title="Apache REEF"> + <img src="ApacheREEF_logo_no_margin_small.png" alt="Apache REEF"/> + </a> + </div> + <div class="pull-right"> <a href="http://incubator.apache.org/"; id="bannerRight"> + <img src="ApacheIncubator.png" /> + </a> + </div> + <div class="clear"><hr/></div> + </div> + + <div id="breadcrumbs"> + <ul class="breadcrumb"> + + + <li class=""> + <a href="index.html" title="Apache REEF"> + Apache REEF</a> + <span class="divider">|</span> + </li> + <li class="active ">FAQ</li> + + + + + </ul> + </div> + + + <div class="row-fluid"> + <div id="leftColumn" class="span2"> + <div class="well sidebar-nav"> + + + <ul class="nav nav-list"> + <li class="nav-header">Apache REEF</li> + + <li> + + <a href="index.html" title="Overview"> + <span class="none"></span> + Overview</a> + </li> + + <li class="active"> + + <a href="#"><span class="none"></span>FAQ</a> + </li> + + <li> + + <a href="license.html" title="License"> + <span class="none"></span> + License</a> + </li> + + <li> + + <a href="downloads.html" title="Downloads"> + <span class="none"></span> + Downloads</a> + </li> + <li class="nav-header">Documentation</li> + + <li> + + <a href="introduction.html" title="Introduction to REEF"> + <span class="none"></span> + Introduction to REEF</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/REEF/Tutorials"; class="externalLink" title="REEF Tutorial"> + <span class="none"></span> + REEF Tutorial</a> + </li> + + <li> + + <a href="talks.html" title="Talks"> + <span class="none"></span> + Talks</a> + </li> + + <li> + + <a href="glossary.html" title="Glossary"> + <span class="none"></span> + Glossary</a> + </li> + + <li> + + <a href="tang.html" title="Tang"> + <span class="none"></span> + Tang</a> + </li> + + <li> + + <a href="wake.html" title="Wake"> + <span class="none"></span> + Wake</a> + </li> + + <li> + + <a href="apidocs/0.10.0-incubating/index.html" title="0.10.0-incubating API"> + <span class="none"></span> + 0.10.0-incubating API</a> + </li> + + <li> + + <a href="apidocs/0.11.0-incubating/index.html" title="0.11.0-incubating API"> + <span class="none"></span> + 0.11.0-incubating API</a> + </li> + + <li> + + <a href="apidocs/0.12.0-incubating/index.html" title="0.12.0-incubating API"> + <span class="none"></span> + 0.12.0-incubating API</a> + </li> + <li class="nav-header">Contribution</li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/REEF/Contributing"; class="externalLink" title="Contributing"> + <span class="none"></span> + Contributing</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/REEF/Committer+Guide"; class="externalLink" title="Committer Guide"> + <span class="none"></span> + Committer Guide</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/REEF/Coding+Guidelines"; class="externalLink" title="Coding Guidelines"> + <span class="none"></span> + Coding Guidelines</a> + </li> + <li class="nav-header">Community</li> + + <li> + + <a href="team.html" title="Team"> + <span class="none"></span> + Team</a> + </li> + + <li> + + <a href="mailing-list.html" title="Mailing List"> + <span class="none"></span> + Mailing List</a> + </li> + + <li> + + <a href="https://issues.apache.org/jira/browse/REEF"; class="externalLink" title="Issue Tracker"> + <span class="none"></span> + Issue Tracker</a> + </li> + + <li> + + <a href="https://cwiki.apache.org/confluence/display/REEF/Powered+by+REEF"; class="externalLink" title="Powered By"> + <span class="none"></span> + Powered By</a> + </li> + <li class="nav-header">ASF</li> + + <li> + + <a href="http://www.apache.org/foundation/"; class="externalLink" title="Apache Software Foundation"> + <span class="none"></span> + Apache Software Foundation</a> + </li> + + <li> + + <a href="http://www.apache.org/foundation/how-it-works.html"; class="externalLink" title="How Apache Works"> + <span class="none"></span> + How Apache Works</a> + </li> + + <li> + + <a href="http://incubator.apache.org/"; class="externalLink" title="Apache Incubator"> + <span class="none"></span> + Apache Incubator</a> + </li> + + <li> + + <a href="http://www.apache.org/licenses/LICENSE-2.0.html"; class="externalLink" title="Apache License"> + <span class="none"></span> + Apache License</a> + </li> + + <li> + + <a href="http://www.apache.org/foundation/sponsorship.html"; class="externalLink" title="Sponsorship"> + <span class="none"></span> + Sponsorship</a> + </li> + + <li> + + <a href="http://www.apache.org/foundation/thanks.html"; class="externalLink" title="Thanks"> + <span class="none"></span> + Thanks</a> + </li> + </ul> + + + + <hr /> + + <div id="poweredBy"> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <div class="clear"></div> + <a href="http://maven.apache.org/"; title="Built by Maven" class="poweredBy"> + <img class="builtBy" alt="Built by Maven" src="./images/logos/maven-feather.png" /> + </a> + </div> + </div> + </div> + + + <div id="bodyColumn" class="span10" > + + <!-- Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. --><h1>FAQ</h1> + +<ol style="list-style-type: decimal"> + +<li><a href="#who">Who is REEF for?</a></li> + +<li><a href="#why">Why did it come about?</a></li> + +<li><a href="#tang">What is Tang?</a></li> + +<li><a href="#wake">What is Wake?</a></li> + +<li><a href="#how">How can I get started?</a></li> +</ol> +<div class="section"> +<div class="section"> +<h3><a name="who"></a>Who is REEF for?<a name="Who_is_REEF_for"></a></h3> +<p>REEF is for developers of data processing systems on cloud computing platforms that provide fine-grained resource allocations. REEF provides system authors with a centralized (pluggable) control flow that embeds a user-defined system controller called the Job Driver. The interfaces associated with the Job Driver are event driven; events signal resource allocations and failures, various states associated with task executions and communication channels, alarms based on wall-clock or logical time, and so on. REEF also aims to package a variety of data-processing libraries (e.g., high-bandwidth shuffle, relational operators, low-latency group communication, etc.) in a reusable form. Authors of big data systems and toolkits can leverage REEF to immediately begin development of their application specific data flow, while reusing packaged libraries where they make sense.</p> +<hr /></div> +<div class="section"> +<h3><a name="why"></a>Why did it come about?<a name="Why_did_it_come_about"></a></h3> +<p>Traditional data-processing systems are built around a single programming model (like SQL or MapReduce) and a runtime (query) engine. These systems assume full ownership over the machine resources used to execute compiled queries. For example, Hadoop (version one) supports the MapReduce programming model, which is used to express jobs that execute a map step followed by an optional reduce step. Each step is carried out by some number of parallel tasks. The Hadoop runtime is built on a single master (the JobTracker) that schedules map and reduce tasks on a set of workers (TaskTrackers) that expose fixed-sized task “slots”. This design leads to three key problems in Hadoop: </p> + +<ol style="list-style-type: decimal"> + +<li>The resources tied to a TaskTracker are provisioned for MapReduce only.</li> + +<li>Clients must speak some form of MapReduce in order to make use of cluster resources, and in turn, gain compute access to the data that lives there.</li> + +<li>Poor cluster utilization, especially in the case of idle resources (slots) due to straggler tasks.</li> +</ol> +<p>With YARN (Hadoop version two), resource management has been decoupled from the MapReduce programming model in Hadoop, freeing cluster resources from slotted formats, and opening the door to programming frameworks beyond MapReduce. It is well understood that while enticingly simple and fault-tolerant, the MapReduce model is not ideal for many applications, especially iterative or recursive workloads like machine learning and graph processing, and those that tremendously benefit from main memory (as opposed to disk based) computation. A variety of big data systems stem from this insight: Microsoft’s Dryad, Apache Spark, Google’s Pregel, CMU’s GraphLab and UCI’s AsterixDB, to name a few. Each of these systems add unique capabilities, but form islands around key functionalities, making it hard to share both data and compute resources between them. YARN, and related resource managers, move us one step closer toward a unified Big Data system stack. The goal of REEF is to provide the next level of detail in this layering.</p> +<hr /></div> +<div class="section"> +<h3><a name="tang"></a>What is Tang?<a name="What_is_Tang"></a></h3> +<p>Tang is a dependency injection Framework co-developed with REEF. It has extensive documentation which can be found <a href="tang.html">here</a>.</p> +<hr /></div> +<div class="section"> +<h3><a name="wake"></a>What is Wake?<a name="What_is_Wake"></a></h3> +<p>Please refer to <a href="wake.html">this</a> section.</p> +<hr /></div> +<div class="section"> +<h3><a name="how"></a>How can I get started?<a name="How_can_I_get_started"></a></h3> +<p>Check out the <a class="externalLink" href="https://cwiki.apache.org/confluence/display/REEF/Tutorials";>REEF Tutorial</a> and the <a class="externalLink" href="https://cwiki.apache.org/confluence/display/REEF/Contributing";>Contributing</a> page and join the community!</p></div></div> + </div> + </div> + </div> + + <hr/> + + <footer> + <div class="container-fluid"> + <div class="row-fluid"> + +<div class="container-fluid"> + <div class="row-fluid"> + <a href="http://www.apache.org";>Apache Software Foundation</a> + </div> +</div> + +<div class="row span12">Apache REEF, REEF, Apache, the Apache feather logo, and the Apache REEF logo are trademarks + of The Apache Software Foundation. All other marks mentioned may be trademarks or registered + trademarks of their respective owners.</div> + </div> + + + </div> + </footer> + </body> +</html>
