Repository: incubator-apex-site
Updated Branches:
  refs/heads/asf-site 36d67a699 -> 65c080a85


from 7424ddbc50f4f316200207952b4d09913e5a0f8c


Project: http://git-wip-us.apache.org/repos/asf/incubator-apex-site/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-apex-site/commit/cd8ef1cb
Tree: http://git-wip-us.apache.org/repos/asf/incubator-apex-site/tree/cd8ef1cb
Diff: http://git-wip-us.apache.org/repos/asf/incubator-apex-site/diff/cd8ef1cb

Branch: refs/heads/asf-site
Commit: cd8ef1cb2d24ac5b9772870c5c65069c26fd7bed
Parents: a773a78
Author: sashadt <[email protected]>
Authored: Sat Jan 2 11:33:59 2016 -0800
Committer: sashadt <[email protected]>
Committed: Sat Jan 2 11:33:59 2016 -0800

----------------------------------------------------------------------
 content/contributing.html |   6 +
 content/docs.html         |   6 +-
 content/roadmap.html      | 523 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 533 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/cd8ef1cb/content/contributing.html
----------------------------------------------------------------------
diff --git a/content/contributing.html b/content/contributing.html
index 3be0197..2bd67e9 100644
--- a/content/contributing.html
+++ b/content/contributing.html
@@ -118,6 +118,12 @@
 </li>
 <li>Once your feature is complete, submit the pull request on github against 
<code>devel-3</code>.</li>
 <li>If you want specific people to review your pull request, use the 
<code>@</code> notation in Github comments to mention that user, and request 
that he/she reviews your changes.</li>
+<li>Check the status of the pull request and ensure the Travis CI build is 
successful. If not, inspect the linked build log for details.<ul>
+<li>If build fails due to license headers, follow instructions above.</li>
+<li>If build fails due to code style violations, run <code>mvn 
checkstyle:check -Dcheckstyle.console=true</code> and correct those issues that 
were introduced with your changes. </li>
+</ul>
+</li>
+<li>Add changes after the PR was opened to the same branch, Travis CI will 
detect changes and build automatically. To force the CI run, close and re-open 
the PR.</li>
 <li><p>After all review is complete, combine all new commits into one squashed 
commit except when there are multiple contributors, and include the Jira number 
in the commit message. There are several ways to squash commits, but <a 
href="https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#Squashing-Commits";>here
 is one explanation from git-scm.com</a> and a simple example is illustrated 
below:</p>
 <p>If tracking upstream/devel-3 then run <code>git rebase -i</code>. Else run 
<code>git rebase -i upstream/devel-3</code>.<br>This command opens the text 
editor which lists the multiple commits:</p>
 <pre><code>pick 67cd79b change1

http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/cd8ef1cb/content/docs.html
----------------------------------------------------------------------
diff --git a/content/docs.html b/content/docs.html
index ec58547..bf08595 100644
--- a/content/docs.html
+++ b/content/docs.html
@@ -76,8 +76,6 @@
 <div class="container">
   
   <h1 id="documentation">Documentation</h1>
-<p>New documentation will be coming soon!</p>
-<p>Here are a few resources to get started with Apex:</p>
 <h3 id="getting-started">Getting Started</h3>
 <ul>
 <li><a href="https://youtu.be/LwRWBudOjg4";>Building Your First Apache Apex 
Application (video)</a></li>
@@ -94,6 +92,10 @@
 <li><a href="https://www.datatorrent.com/docs/apidocs/";>JavaDoc</a></li>
 <li><a 
href="https://www.datatorrent.com/blog-apex-performance-benchmark/";>Benchmarks 
compare between 2.0 and 3.0</a></li>
 </ul>
+<h3 id="roadmap">Roadmap</h3>
+<ul>
+<li><a href="roadmap.html">Apex Roadmap</a> comprises key features planned for 
the future releases</li>
+</ul>
 
 </div>
 

http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/cd8ef1cb/content/roadmap.html
----------------------------------------------------------------------
diff --git a/content/roadmap.html b/content/roadmap.html
new file mode 100644
index 0000000..ef38760
--- /dev/null
+++ b/content/roadmap.html
@@ -0,0 +1,523 @@
+<html lang="en"><head>
+    
+    <meta charset="utf-8">
+    <meta http-equiv="X-UA-Compatible" content="IE=edge">
+    <meta name="viewport" content="width=device-width, initial-scale=1">
+    <!-- The above 3 meta tags *must* come first in the head; any other head 
content must come *after* these tags -->
+    <meta name="description" content="Apex is an enterprise grade native YARN 
big data-in-motion platform that unifies stream processing as well as batch 
processing.">
+    <meta name="author" content="Apache Software Foundation">
+    <link rel="icon" href="favicon.ico">
+
+    <title>Apache Apex (Incubating)</title>
+
+    <!-- Main Stylesheet -->
+    <link href="css/main.css" rel="stylesheet">
+
+  </head>
+
+  <body>
+    <nav class="navbar navbar-default navbar-static-top" id="main-nav">
+      <div class="container">
+
+      <div class="navbar-header">
+        <button type="button" class="navbar-toggle collapsed" 
data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" 
aria-expanded="false">
+          <span class="sr-only">Toggle navigation</span>
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+          <span class="icon-bar"></span>
+        </button>
+        <a class="navbar-brand" href="/">
+          <img src="images/apex-logo.svg" class="logo" alt="Apache Apex Logo">
+          Apache Apex
+          <small>(incubating)</small>
+        </a>
+      </div>
+
+      <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1">
+        <ul class="nav navbar-right navbar-nav">
+          <li class="nav-item">
+            <a class="nav-link " href="/">Home</a>
+          </li>
+          <li class="nav-item">
+            <a class="nav-link " href="/announcements.html">Announcements</a>
+          </li>
+          <li class="nav-item">
+            <a class="nav-link " href="/community.html">Community</a>
+          </li>
+          <li class="nav-item">
+            <a class="nav-link " href="/docs.html">Docs</a>
+          </li>
+          <li class="nav-item">
+            <a href="#" data-toggle="dropdown" class="dropdown-toggle 
nav-link">Source<b class="caret"></b></a>
+             <ul class="dropdown-menu">
+              <li><a 
href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-core.git";>Apex 
Core (ASF)</a></li>
+              <li><a href="https://github.com/apache/incubator-apex-core";>Apex 
Core (Github Mirror)</a></li>
+              <li><a 
href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-malhar.git";>Apex 
Malhar (ASF)</a></li>
+              <li><a 
href="https://github.com/apache/incubator-apex-malhar";>Apex Malhar (Github 
Mirror)</a></li>
+            </ul>
+          </li>
+          <li class="nav-item">
+            <a href="#" data-toggle="dropdown" class="dropdown-toggle 
nav-link">Apache<b class="caret"></b></a>
+             <ul class="dropdown-menu">
+              <li><a 
href="http://incubator.apache.org/projects/apex.html";>Status Page</a></li>
+              <li><a 
href="http://www.apache.org/foundation/how-it-works.html";>Apache 
Foundation</a></li>
+              <li><a href="http://www.apache.org/licenses/";>Apache 
License</a></li>
+              <li><a 
href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li>
+              <li><a 
href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li>
+            </ul>
+          </li>
+          <li class="nav-item">
+            <a class="nav-link btn btn-success" 
href="/downloads.html">Download</a>
+          </li>
+        </ul>
+        
+      </div>
+    </nav>
+<div class="container">
+  
+  <h1>Apex Roadmap</h1>
+  
+  <!-- APEX CORE ROADMAP -->
+  <h2>Core</h2>
+  <table class="table table-bordered table-striped">
+    <thead>
+      <tr>
+        <th scope="col">JIRA</th>
+        <th scope="col">Summary</th>
+        <th scope="col">Version</th>
+      </tr>
+    </thead>
+    <tbody>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-3";>APEXCORE-3</a>
+        </td>
+        <td title="Apex should have an operator API that lets the operator 
generate DAG during launch time. This will mean the following
+
+- Logical DAG will have one operator. This is the operator that will generate 
a DAG underneath
+- Physical plan will have the DAG generated by the operator
+- Execution plan will mimic physical plan + container location etc.
+
+For example lets say we have three operators in a DAG (app) A-&gt;B-&gt;C
+
+B during launch time generates a DAG B1-&gt;B2-&gt;B3, then the physical plan 
will be
+
+A-&gt;B1-&gt;B2-&gt;B3-&gt;C
+
+This should work irrespective of number of ports, etc. A typical flattening. 
The operators inside of B (B1, B2, B3) should have properties and attributes 
just as any. Users should be able to access these at run time and compile time. 
B itself should support properties and attributes that B1, B2, B3 can inherit 
from.
+
+This is a very critical feature as it will open up users to plug-in their own 
engines and still take up complete operability support from Apex engine.">
+          Ability for an operator to populate DAG at launch time
+        </td>
+        <td>
+    
+
+            <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE/fixforversion/12333950";>3.3.0</a>&nbsp;
+
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-10";>APEXCORE-10</a>
+        </td>
+        <td title="The issue happens on cloud which provides virtual cores 
with software like Xen underneath. In effect if CPU intensive operators land up 
on same node we have a resource bottleneck,
+
+Need to create an attribute that does the following
+- Operators A &amp; B should not be on same node
+- Stram should use this attribute to try to get containers on different node
+
+It is understood that the user is making an explicit choice to use NIC instead 
of stream local optimization">
+          Enable non-affinity of operators per node (not containers)
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-60";>APEXCORE-60</a>
+        </td>
+        <td title="We would like to support iterative processing by 
introducing cycles in the graph (known as DAG now, but no longer if we support 
iterative processing).
+
+Initial idea is as follow:
+{noformat}
+     |----|
+     v    |
+A -&gt; B -&gt; C -&gt; D
+^         |
+|---------|
+{noformat} 
+
+C has two separate backward streams to A and B.  The input ports of A and B 
that C connects to will have a special attribute on how many window IDs ahead 
the incoming windows should be treated as, and A and B will be responsible for 
the initial data for such input ports.
+
+Another idea is to have C advance the window ID on its output ports and have C 
generate the initial data on its output ports to A and B.">
+          Iterative processing support
+        </td>
+        <td>
+    
+
+            <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE/fixforversion/12333950";>3.3.0</a>&nbsp;
+
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-119";>APEXCORE-119</a>
+        </td>
+        <td title="This JIRA Proposes support for a new type of distributed 
operator. Currently when an operator is partitioned there is no platform 
supported mechanism through which partitions can talk to each other. A 
Distributed operator would have an easy to use platform supported mechanism 
through which operators in a partitioning can exchange information with each 
other. Eventually Distributed operators would support running plain old single 
threaded java code transparently across partitions.
+
+In summary the goals would be to do the following:
+
+1 - provide a platform supported fault tolerant mechanism through which 
operators in a partitioning can talk to each other.
+2 - provide a platform supported way to run plain old single threaded java 
code accross all the partitions of a Distributed operator
+
+The benefits of implementing this would be huge:
+
+1 - Using distributed operators we could support large in memory fault 
tolerant data structures (graphs, maps, arrays) in a fault tolerant way. Like 
Spark&#x27;s RDD&#x27;s but better.
+2 - Plain old java code could be used to access and manipulate the data 
structures, without the user having the learn complex API&#x27;s like with 
Spark.
+
+An implementation proposal and presentation are coming soon.">
+          Add Support For A New Type Of (Distributed) Operator
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-163";>APEXCORE-163</a>
+        </td>
+        <td title="Apex support modification of operator properties at runtime 
but the current implemenations has the following shortcomings.
+
+1. Property is not set across all partitions on the same window as individual 
partitions can be on different windows when property change is initiated from 
client resulting in inconsistency of data for those windows. I am being 
generous using the word inconsistent.
+2. Sometimes properties need to be set on more than one logical operators at 
the same time to achieve the change the user is seeking. Today they will be two 
separate changes happening on two different windows again resulting in 
inconsistent data for some windows. These would need to happen as a single 
transaction.
+3. If there is an operator failure before a committed checkpoint after an 
operator property is dynamically changed the operator will restart with the old 
property and the change will not be re-applied.
+
+Tim and myself did some brainstorming and we have a proposal to overcome these 
shortcomings. The main problem in all the above cases is that the property 
changes are happening out-of-band of data flow and hence independent of 
windowing. The proposal is to bring the property change request into the 
in-band dataflow so that they are handled consistently with windowing and 
handled distributively.
+
+The idea is to inject a special property change tuple containing the property 
changes and the identification information of the operator&#x27;s they affect 
into the dataflow at the input operator. The tuple will be injected at window 
boundary after end window and before begin window and as this tuple flows 
through the DAG the intended operators properties will be modifed. They will 
all be modified consistently at the same window. The tuple can contain more 
than one property changes for more than one logical operators and the change 
will be applied consistently to the different logical operators at the same 
window. In case of failure the replay of tuples will ensure that the property 
change gets reapplied at the correct window.">
+          Dynamic application property changes
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-202";>APEXCORE-202</a>
+        </td>
+        <td title="Apache Samoa[https://samoa.incubator.apache.org/] is an 
abstraction of a collections of streaming machine learning Algorithm. By far, 
it has integration with Samza, Storm and flink, It is a good start point for 
Apex to support streaming ML.">
+          Integration with Samoa
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-231";>APEXCORE-231</a>
+        </td>
+        <td title="The Apex engine supports many platform level attributes 
like operator memory, application window count, container jvm options etc. 
Today these can only be set at application launch time and cannot be changed 
once the application is running.
+
+This issue is to add the ability to change the attributes dynamically even as 
the application is running. The mechanics of an user requesting the attribute 
change can be similar to how a user requests property change via the command 
line client.
+
+Since each attribute is different the actual backend implementation to affect 
the changes will most likely be custom handling for different attributes but 
during the implementation process  hopefully some common themes emerge and some 
amount of reuse possible.">
+          Ability to configure attributes dynamically
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-232";>APEXCORE-232</a>
+        </td>
+        <td title="There are scenarios when new processing code needs to be 
added to an already running application. There are two scenarios.
+
+a. A bug is discovered in an operator and an existing operator in the running 
DAG needs to be replaced. The platform supports shutting down and resuming an 
application which could be use as a first cut way to do this but there are a 
couple of drawbacks.
+       i. This only works when the input source has memory, if it doesn&#x27;t 
the messages received during the time the application is down are lost.
+      ii. Depending on the complexity and state of the application it may take 
some time for this entire process and the application to get back to running 
state and this delay may not be acceptable for the downstream components that 
depend on the output of this application.
+
+b. A new operator needs to be added to the DAG to take data from an existing 
operator and do some additional processing. Today this is supported as long as 
the code for the operator is already in the application libraries. Often this 
will not be the case as users will not know what the operator will be 
beforehand when the application is originally launched.">
+          Ability to add new processing code to the DAG
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-233";>APEXCORE-233</a>
+        </td>
+        <td title="There are scenarios where the same object instance needs to 
be specified for two attributes. Example is partitioner and stats listener, for 
partitioners that need to affect partitoning based on operator stats the same 
instance needs to be both. This is not possible to specify using a property 
file today as it will create two separate instances and can only be done in 
Java code today. The issue is to request adding this feature.">
+          Ability to specify single instance objects in configuration
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-234";>APEXCORE-234</a>
+        </td>
+        <td title="The current property file specification follows the hadoop 
configuration file format and this has led to some drawbacks. 
+    a. The names for the properties and attributes are verbose in the 
configuration file. 
+    b. When there are nested properties in operators the syntax deviates from 
the bean specification because it introduces some specific keywords in the 
specification like .prop and ,attr.
+
+There will already be some changes afoot based on the following
+   a. When adding ability to specify single instance attributes 
(https://malhar.atlassian.net/browse/APEXCORE-233) implementing it in the 
current syntax may not be possible or lead to very unwieldy syntax.
+   b. There are also other ideas such as one from David to have the ability to 
specify global application level attributes which possible require rethinking 
the current syntax.
+
+Users have also asked for an easier and more consistent way to specify these 
properties.  This issue is to track the ideas and progress of these changes.">
+          Investigate other ways to specify properties in property files
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-235";>APEXCORE-235</a>
+        </td>
+        <td title="Apex can be used for real-time and batch processing as it 
stands, but there are some aspects of batch processing that can be better 
supported through explicit constructs. This ticket can serve as umbrella for 
various features.">
+          Explicit support for batch processing
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-289";>APEXCORE-289</a>
+        </td>
+        <td title="We should support encrypted streams in a DAG for Apex.
+Basically there will be 2 ways user can configure the streams for encryption:
+1) App wide attributes- Using which all the stream in the DAG will have 
encrypted channel.
+2) Stream based attribute - Using this user can set a certain stream to flow 
over encrypted channel.
+
+Encrypted for the streams should done at Network/Buffer Server levels.">
+          Encrypted Streams in Apex DAG
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-293";>APEXCORE-293</a>
+        </td>
+        <td title="">
+          Add core and malhar documentation to project web site
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://issues.apache.org/jira/browse/APEXCORE-295";>APEXCORE-295</a>
+        </td>
+        <td title="Flink streaming is compatible with Apache Storm interfaces 
and therefore allows reusing code that was implemented for Storm.
+Details can be found here.
+https://ci.apache.org/projects/flink/flink-docs-master/apis/storm_compatibility.html
+This jira item can contain tasks for providing similar support in Apex">
+          Running a Storm topology on Apex.
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+    </tbody>
+  </table>
+
+  <!-- APEX MALHAR ROADMAP -->
+  <h2>Malhar</h2>
+  <table class="table table-bordered table-striped">
+    <thead>
+      <tr>
+        <th scope="col">JIRA</th>
+        <th scope="col">Summary</th>
+        <th scope="col">Version</th>
+      </tr>
+    </thead>
+    <tbody>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1720";>MLHR-1720</a>
+        </td>
+        <td title="">
+          Development of Inner Join Operator
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1811";>MLHR-1811</a>
+        </td>
+        <td title="Add new condition for non-equality join predicate (for 
example, user.zipcode != authzn.zipcode)">
+          Add Non-Equality Join Condition
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1818";>MLHR-1818</a>
+        </td>
+        <td title="Once we have ability to code generate, we should take a 
look at integrating Calcite into Apex. The operator that enables populate DAG, 
should use Calcite to generate the DAG, given a SQL query.">
+          Create a Calcite operator to enable SQL commands to be run
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1843";>MLHR-1843</a>
+        </td>
+        <td title="[~andyp] I am assigning this to you cause you are the one 
who first said it. So either you lead it or find a willing lead to get this 
task to completion.
+
+The problem with contrib and library modules of malhar is that a ton of 
dependencies are prescribed as optional. The motive behind it was that the 
users of these libraries are given an opportunity to keep the size of the 
dependency-included packages to bare minimum. It  comes at a cost that the 
dependency now has to be manually figured out. This is a complete misuse of the 
optional dependency, IMO. It defeats the purpose of maven having dependency 
management as one of the biggest features of it.
+
+So keep things sane - the proposed compromise is that we start creating 
smaller discreet packages for discrete technologies. ">
+          Split Malhar Library and Malhar Contrib package into baby packages
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1873";>MLHR-1873</a>
+        </td>
+        <td title="We need a Scalable Cache component in Malhar. Following are 
some of the key features of the cache
+
+1. Cache has limited size and is backed by a persistent store.
+2. When there is a cache miss, the data is loaded from backup store.
+3. To minimize misses, a range of keys can be loaded.
+4. Ability to purge key/values
+5. Writing to the backup store should be optimized.
+6. It should provide support for fault-tolerance.">
+          Create a fault-tolerant/scalable cache component backed by a 
persistent store
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1904";>MLHR-1904</a>
+        </td>
+        <td title="">
+          Rewrite kafka input operator to use 0.9.0 new consumer
+        </td>
+        <td>
+    
+
+            <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR/fixforversion/12000";>3.3.0</a>&nbsp;
+
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1938";>MLHR-1938</a>
+        </td>
+        <td title="Currently Apex engine provides operator checkpointing in 
Hdfs ( with Hdfs backed StorageAgents i.e. FSStorageAgent &amp; 
AsyncFSStorageAgent )
+As operator check-pointing is critical functionality of Apex streaming 
platform to ensure fault tolerant behavior, platform should also provide 
alternate StorageAgents which will work seamlessly with large applications that 
requires Exactly once semantics.
+HDFS read/write latency is limited and doesn&#x27;t improve beyond certain 
point because of disk io &amp; staging writes. Having alternate strategy to 
this check-pointing in fault tolerant distributed in-memory grid would ensure 
application stability and performance is not impacted by checkpointing
+
+*This feature will add below functionalities*
+* A KeyValue store interface which is used by In-memory checkpointing storage 
agent.
+* Abstract implementation of KeyValue storage agent which can be configured 
with concrete implementation of KeyValue store for checkpointing.
+* Concrete implementation of In memory storage agent for Apache Geode
+
+*This feature depends on below APEX core feature* 
+https://issues.apache.org/jira/browse/APEXCORE-283
+* Interface for storage agent to provide application id
+* Stram client changes to pass applicationId ">
+          Operator checkpointing in distributed in-memory store
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1939";>MLHR-1939</a>
+        </td>
+        <td title="">
+          Stream API
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+      <tr>
+        <td>
+          <a target="_blank" 
href="https://malhar.atlassian.net/browse/MLHR-1942";>MLHR-1942</a>
+        </td>
+        <td title="We would like to contribute the Apache 
Geode(http://geode.incubator.apache.org/) Operator support for Apex.
+It will basically be implementation for writing to geode region.
+This is in continuation with the Operator checkpointing alternative under 
review (MLHR-1938)">
+          Apex Operator for Apache Geode.
+        </td>
+        <td>
+    
+
+        </td>
+      </tr>
+    </tbody>
+  </table>
+
+</div>
+
+  <hr>
+  <div class="container">
+    <footer id="main-footer">
+      <p>
+        Copyright &copy; <span id="copyright-year">2015</span> <a 
href="http://apache.org";>The Apache Software Foundation</a>,
+        Licensed under the Apache License, Version 2.0<br>
+        Apache and the Apache feather logo are trademarks of The Apache 
Software Foundation.<br>
+        <a class="footer-link-img" href="http://apache.org";><img 
src="/images/asf-feather.png" alt="The Apache Software Foundation"></a>
+        <a class="footer-link-img" href="http://incubator.apache.org/";><img 
src="/images/incubator-egg.png" alt="Apache Incubator"></a>
+      </p>
+      <p class="text-muted">
+        <small>Apache Apex is an effort undergoing incubation at The Apache 
Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is 
required of all newly accepted projects until a further review indicates that 
the infrastructure, communications, and decision making process have stabilized 
in a manner consistent with other successful ASF projects. While incubation 
status is not necessarily a reflection of the completeness or stability of the 
code, it does indicate that the project has yet to be fully endorsed by the 
ASF.</small>
+      </p>
+    </footer>
+  </div> <!-- /container -->
+
+  <!-- Placed at the end of the document so the pages load faster -->
+  <script 
src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js";></script>
+  <script src="/js/bootstrap.min.js"></script>
+  <script>
+    $('#copyright-year').text((new Date()).getFullYear());
+  </script>
+
+</body>
+</html>

Reply via email to