Repository: incubator-apex-site Updated Branches: refs/heads/asf-site bb0eb98f1 -> a7225fbbf
http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/04ffbb71/content/content/people.html ---------------------------------------------------------------------- diff --git a/content/content/people.html b/content/content/people.html new file mode 100644 index 0000000..e510b0c --- /dev/null +++ b/content/content/people.html @@ -0,0 +1,331 @@ +<html lang="en"><head> + + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> + <meta name="description" content="Apex is an enterprise grade native YARN big data-in-motion platform that unifies stream processing as well as batch processing."> + <meta name="author" content="Apache Software Foundation"> + <link rel="icon" href="favicon.ico"> + + <title>Apache Apex</title> + + <!-- Main Stylesheet --> + <link href="css/main.css" rel="stylesheet"> + + </head> + + <body> + <nav class="navbar navbar-default navbar-static-top" id="main-nav"> + <div class="container"> + + <div class="navbar-header"> + <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="navbar-brand" href="/"> + <img src="images/apex-logo.svg" class="logo" alt="Apache Apex Logo"> + Apache Apex<span class="trademark">™</span> + </a> + </div> + + <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> + <ul class="nav navbar-right navbar-nav"> + <li class="nav-item"> + <a class="nav-link " href="/">Home</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/announcements.html">Announcements</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/community.html">Community</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/docs.html">Docs</a> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Source<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-core.git";>Apex Core (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-core";>Apex Core (Github Mirror)</a></li> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-malhar.git";>Apex Malhar (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-malhar";>Apex Malhar (Github Mirror)</a></li> + </ul> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Apache<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="http://incubator.apache.org/projects/apex.html";>Status Page</a></li> + <li><a href="http://www.apache.org/foundation/how-it-works.html";>Apache Foundation</a></li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + </ul> + </li> + <li class="nav-item"> + <a class="nav-link btn btn-success" href="/downloads.html">Download</a> + </li> + </ul> + + </div> + </nav> + +<div class="container"> + + <h1 id="people">People</h1> +<p>The community is comprised of the PMC, Committers and Contributors. The PMC (podling project management committee) is a committee of the Apache Software Foundation charged with responsibility for a top level project. Committers have direct access to the source of a project and actively evolve the code-base. Contributors improve the project in various ways such as submission of patches, improving documentation, testing the product and by providing suggestions to the Committers. The number of Contributors to the project is unbounded. Get involved today. All contributions to the project are greatly appreciated. For more details, please see <a href="contributing.html">contributing guidelines</a>.</p> +<h2 id="becoming-a-committer">Becoming a committer</h2> +<p>Community members that make sustained, welcome contributions to the project may be invited to become a committer. Committers are voted in by the PPMC. A committer has a signed Contributor License Agreement (CLA) on file and an apache.org address.</p> +<p>We expect committers to subscribe to the project mailing lists.</p> +<p>A committer will be considered âemeritus/inactiveâ by not contributing in any form to the project for over 1 year. An emeritus committer may request reinstatement of commit access from the PPMC. Such reinstatement is subject to lazy consensus of active PPMC members.</p> +<p>The Podling Project Management Committee (PPMC) is responsible for the oversight of the project and it also decides who to add as a PPMC member. Existing committers may be invited to become a PPMC member after consistent contribution and activity over a period of time and participation in directional and community building discussions.</p> +<h2 id="committers">Committers</h2> +<p>Following is the list of all Apache Apex committers, PPMC members and mentors. A good place to see all contributors and other project stats is <a href="https://www.openhub.net/p/apache_apex";>Open Hub</a></p> +<table> +<thead> +<tr> +<th>Name</th> +<th>Apache ID</th> +<th>Role</th> +</tr> +</thead> +<tbody> +<tr> +<td>Isha Arkatkar</td> +<td>ishark</td> +<td>Committer</td> +</tr> +<tr> +<td>Milind Barve</td> +<td>milindb</td> +<td>Committer</td> +</tr> +<tr> +<td>Bhupesh Chawda</td> +<td>bhupesh</td> +<td>Committer</td> +</tr> +<tr> +<td>Chaitanya Chebolu</td> +<td>chaithu</td> +<td>Committer</td> +</tr> +<tr> +<td>Bright Chen</td> +<td>brightchen</td> +<td>Committer</td> +</tr> +<tr> +<td>Pradeep A. Dalvi</td> +<td>prad</td> +<td>Committer</td> +</tr> +<tr> +<td>Sandeep Deshmukh</td> +<td>sandeep</td> +<td>Committer</td> +</tr> +<tr> +<td>Yogi Devendra</td> +<td>yogidevendra</td> +<td>Committer</td> +</tr> +<tr> +<td>Ted Dunning</td> +<td>tdunning</td> +<td>Mentor</td> +</tr> +<tr> +<td>Timothy Farkas</td> +<td>timothyfarkas</td> +<td>Committer</td> +</tr> +<tr> +<td>Ilya Ganelin</td> +<td>ilganeli</td> +<td>PPMC</td> +</tr> +<tr> +<td>Alan Gates</td> +<td>gates</td> +<td>Mentor</td> +</tr> +<tr> +<td>P. Taylor Goetz</td> +<td>ptgoetz</td> +<td>Mentor</td> +</tr> +<tr> +<td>Tushar Gosavi</td> +<td>tushar</td> +<td>Committer</td> +</tr> +<tr> +<td>Priyanka Gugale</td> +<td>priyag</td> +<td>Committer</td> +</tr> +<tr> +<td>Gaurav Gupta</td> +<td>gauravgopi123</td> +<td>PPMC</td> +</tr> +<tr> +<td>Sandesh Hegde</td> +<td>sandesh</td> +<td>Committer</td> +</tr> +<tr> +<td>Siyuan Hua</td> +<td>hsy541</td> +<td>Committer</td> +</tr> +<tr> +<td>Pramod Immaneni</td> +<td>pramod</td> +<td>PPMC</td> +</tr> +<tr> +<td>Amol Kekre</td> +<td>akekre</td> +<td>PPMC</td> +</tr> +<tr> +<td>Chinmay Kolhatkar</td> +<td>chinmay</td> +<td>Committer</td> +</tr> +<tr> +<td>Anuj Lal</td> +<td>anujlal</td> +<td>Committer</td> +</tr> +<tr> +<td>Dongming Liang</td> +<td>d9liang</td> +<td>Committer</td> +</tr> +<tr> +<td>Justin Mclean</td> +<td>jmclean</td> +<td>Mentor</td> +</tr> +<tr> +<td>Adi Mishra</td> +<td>apmishra</td> +<td>Committer</td> +</tr> +<tr> +<td>Chetan Narsude</td> +<td>chetan</td> +<td>PPMC</td> +</tr> +<tr> +<td>Chris Nauroth</td> +<td>cnauroth</td> +<td>Mentor</td> +</tr> +<tr> +<td>Sasha Parfenov</td> +<td>sashap</td> +<td>Committer</td> +</tr> +<tr> +<td>Andy Perlitch</td> +<td>andyp</td> +<td>Committer</td> +</tr> +<tr> +<td>Shubham Pathak</td> +<td>shubham</td> +<td>Committer</td> +</tr> +<tr> +<td>Ashwin Putta</td> +<td>ashwinchandrap</td> +<td>Committer</td> +</tr> +<tr> +<td>Munagala Ramanath</td> +<td>ramapex</td> +<td>Committer</td> +</tr> +<tr> +<td>Vlad Rozov</td> +<td>vrozov</td> +<td>PPMC</td> +</tr> +<tr> +<td>Hitesh Shah</td> +<td>hitesh</td> +<td>Mentor</td> +</tr> +<tr> +<td>Atri Sharma</td> +<td>atri</td> +<td>Committer</td> +</tr> +<tr> +<td>Chandni Singh</td> +<td>csingh</td> +<td>Committer</td> +</tr> +<tr> +<td>Ashish Tadose</td> +<td>ashishtadose</td> +<td>Committer</td> +</tr> +<tr> +<td>Aniruddha Thombare</td> +<td>aniruddha</td> +<td>Committer</td> +</tr> +<tr> +<td>Jessica Wang</td> +<td>jessicawang</td> +<td>Committer</td> +</tr> +<tr> +<td>Thomas Weise</td> +<td>thw</td> +<td>PPMC</td> +</tr> +<tr> +<td>David Yan</td> +<td>davidyan</td> +<td>PPMC</td> +</tr> +<tr> +<td>Brennon York</td> +<td>boyork</td> +<td>PPMC</td> +</tr> +</tbody> +</table> + +</div> + + <hr> + <div class="container"> + <footer id="main-footer"> + <p> + Copyright © <span id="copyright-year">2015</span> <a href="http://apache.org";>The Apache Software Foundation</a>, + Licensed under the Apache License, Version 2.0<br> + Apache and the Apache feather logo are trademarks of The Apache Software Foundation.<br> + <a class="footer-link-img" href="http://apache.org";><img src="/images/asf_logo.svg" alt="The Apache Software Foundation"></a> + </p> + </footer> + </div> <!-- /container --> + + <!-- Placed at the end of the document so the pages load faster --> + <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js";></script> + <script src="/js/bootstrap.min.js"></script> + <script> + $('#copyright-year').text((new Date()).getFullYear()); + </script> + +</body> +</html> http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/04ffbb71/content/content/release.html ---------------------------------------------------------------------- diff --git a/content/content/release.html b/content/content/release.html new file mode 100644 index 0000000..3f33e57 --- /dev/null +++ b/content/content/release.html @@ -0,0 +1,228 @@ +<html lang="en"><head> + + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> + <meta name="description" content="Apex is an enterprise grade native YARN big data-in-motion platform that unifies stream processing as well as batch processing."> + <meta name="author" content="Apache Software Foundation"> + <link rel="icon" href="favicon.ico"> + + <title>Apache Apex</title> + + <!-- Main Stylesheet --> + <link href="css/main.css" rel="stylesheet"> + + </head> + + <body> + <nav class="navbar navbar-default navbar-static-top" id="main-nav"> + <div class="container"> + + <div class="navbar-header"> + <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="navbar-brand" href="/"> + <img src="images/apex-logo.svg" class="logo" alt="Apache Apex Logo"> + Apache Apex<span class="trademark">™</span> + </a> + </div> + + <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> + <ul class="nav navbar-right navbar-nav"> + <li class="nav-item"> + <a class="nav-link " href="/">Home</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/announcements.html">Announcements</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/community.html">Community</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/docs.html">Docs</a> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Source<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-core.git";>Apex Core (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-core";>Apex Core (Github Mirror)</a></li> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-malhar.git";>Apex Malhar (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-malhar";>Apex Malhar (Github Mirror)</a></li> + </ul> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Apache<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="http://incubator.apache.org/projects/apex.html";>Status Page</a></li> + <li><a href="http://www.apache.org/foundation/how-it-works.html";>Apache Foundation</a></li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + </ul> + </li> + <li class="nav-item"> + <a class="nav-link btn btn-success" href="/downloads.html">Download</a> + </li> + </ul> + + </div> + </nav> + +<div class="container"> + + <h1 id="how-to-release-apache-apex">How to release Apache Apex</h1> +<p>For general information on ASF releases, see: <a href="http://www.apache.org/dev/release.html";>http://www.apache.org/dev/release.html</a></p> +<h2 id="creating-release-branch">Creating Release Branch</h2> +<p>If this is a minor release (X.Y.0), start with creating a new branch. Example for 3.2.0:</p> +<pre><code class="lang-bash">git checkout master && git pull +git checkout -b release-3.2 master +</code></pre> +<p>Replace version in master branch:</p> +<pre><code>git checkout master +git grep -l "3.2.0-SNAPSHOT" +</code></pre><p>For informational purpose, this should yield the list of files that needs the version number replaced to X.(Y+1).0 next version. Note that the replacement step is different between the repositories due to an open issue. See:</p> +<p><a href="https://issues.apache.org/jira/browse/APEXCORE-34";>https://issues.apache.org/jira/browse/APEXCORE-34</a></p> +<p>For -core:</p> +<pre><code class="lang-bash">dv=3.2.0-SNAPSHOT +rv=3.3.0-SNAPSHOT +for a in `git grep -l "${dv}"`; do echo $a; sed -i 's/'"${dv}"'/'"${rv}"'/g' $a; done +</code></pre> +<p>For -malhar:</p> +<pre><code>mvn versions:set -DnewVersion=${rv} -Pall-modules +</code></pre><p>Commit and push the change:</p> +<pre><code>git commit -am "Preparing for 3.3.0 development" +git push apache master +</code></pre><h2 id="preparing-release-candidate">Preparing Release Candidate</h2> +<h3 id="add-missing-since-tags">Add missing @since tags</h3> +<p>For Java classes added since the last release, the @since tags need to be added. The javadoc plugin inserts missing tags, but does not play well with the license header when no class level documentation block is present. This is tracked as</p> +<p><a href="https://issues.apache.org/jira/browse/APEXCORE-183";>https://issues.apache.org/jira/browse/APEXCORE-183</a></p> +<p>It also removes the custom @tags doclet tag when the existing JavaDoc is malformed, <strong>do not use this to make changes in Malhar</strong>. Until these problems are resolved, use the following Ruby script to do the replacement: <a href="https://issues.apache.org/jira/secure/attachment/12781158/add-since.rb";>https://issues.apache.org/jira/secure/attachment/12781158/add-since.rb</a></p> +<pre><code>ruby ~/add-since.rb `pwd` -s 3.2.0 +</code></pre><h3 id="update-changelog-from-jira">Update CHANGELOG from JIRA</h3> +<p>Navigate to the unreleased version, example:</p> +<p><a href="https://issues.apache.org/jira/browse/APEXMALHAR/fixforversion/12334589";>https://issues.apache.org/jira/browse/APEXMALHAR/fixforversion/12334589</a></p> +<p>Obtain the release notes (text mode):</p> +<p><a href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334589&styleName=Text&projectId=12318824";>https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12334589&styleName=Text&projectId=12318824</a></p> +<p>Shorten any wrapping and overly long titles to fit width. Copy the report and insert new release section into <code>CHANGELOG.md</code> with the release date set to 72 hours ahead to reflect the time for the vote.</p> +<p>Create shortlink for the JIRA release notes on <a href="http://s.apache.org/";>http://s.apache.org/</a> for use in the VOTE thread. +Example: <a href="http://s.apache.org/8uT";>http://s.apache.org/8uT</a></p> +<p>Commit tags and change log:</p> +<pre><code>git commit -am "Add @since tags and update change log for release 3.2.0" +</code></pre><h3 id="update-version-number-for-rc">Update version number for RC</h3> +<pre><code>dv=3.2.0-SNAPSHOT +rv=3.2.0 +</code></pre><p>As mentioned earlier, use the following for -core releases:</p> +<pre><code>for a in `git grep -l "${dv}"`; do echo $a; sed -i 's/'"${dv}"'/'"${rv}"'/g' $a; done +</code></pre><p>And this for -malhar releases:</p> +<pre><code>mvn versions:set -Pall-modules -DnewVersion=${rv} +</code></pre><p>Commit version change:</p> +<pre><code>git commit -am "Preparing to release ${rv}-RC1" +git tag -a "v${rv}-RC1" -m "Release ${rv}-RC1" +</code></pre><p>Push to fork (as temporary branch), open pull request, wait for Travis CI build to succeed. Then push the tag.</p> +<pre><code>git push apache "v${rv}-RC1" +</code></pre><p>The only difference between release branch and tag is this final version number change. The branch stays at <code>-SNAPSHOT</code> version.</p> +<h2 id="build-and-deploy-release-candidate">Build and Deploy Release Candidate</h2> +<p>Prerequisites:</p> +<ul> +<li>GPG key needs to be in <a href="https://dist.apache.org/repos/dist/release/incubator/apex/KEYS";>https://dist.apache.org/repos/dist/release/incubator/apex/KEYS</a></li> +<li>Credentials for <code>apache.staging.https</code> server distribution management in <code>~/.m2/settings.xml</code></li> +<li>Tag pushed to ASF git</li> +</ul> +<p>Build and deploy release candidate from RC tag:</p> +<pre><code>git checkout "v${rv}-RC1" +git clean -d -f +mvn clean apache-rat:check deploy -Papache-release -Pall-modules -DskipTests +</code></pre><p>Confirm no archives are included in source release (rat:check reports them in target/rat.txt but does not fail the build):</p> +<pre><code>unzip -l target/*-source-release.zip | grep -e ".zip\|.jar" +</code></pre><p>Log on to <a href="https://repository.apache.org";>https://repository.apache.org</a> and look for Staging Repositories. "Close" the newly created orgapacheapex-xxxx staging repository to obtain the temporary URL, note it down for the VOTE thread.</p> +<p>Example URL: <a href="https://repository.apache.org/content/repositories/orgapacheapex-1000/";>https://repository.apache.org/content/repositories/orgapacheapex-1000/</a></p> +<p>Copy files to distribution dir and create signatures and checksums. +(Note this is per policy to stage these files outside of the Maven repository, otherwise everything below would happen automatically as defined in the parent POM.)</p> +<p>For -core releases:</p> +<pre><code>RNAME=apache-apex-core-${rv} +</code></pre><p>For -malhar releases:</p> +<pre><code>RNAME=apache-apex-malhar-${rv} +</code></pre><pre><code class="lang-bash">cd target +md5sum ${RNAME}-source-release.tar.gz > ${RNAME}-source-release.tar.gz.md5 +md5sum ${RNAME}-source-release.zip > ${RNAME}-source-release.zip.md5 +shasum -a 512 ${RNAME}-source-release.tar.gz > ${RNAME}-source-release.tar.gz.sha +shasum -a 512 ${RNAME}-source-release.zip > ${RNAME}-source-release.zip.sha +gpg --yes --armor --output ${RNAME}-source-release.tar.gz.asc --detach-sig ${RNAME}-source-release.tar.gz +gpg --yes --armor --output ${RNAME}-source-release.zip.asc --detach-sig ${RNAME}-source-release.zip +</code></pre> +<p>Check files into the dist staging area:</p> +<pre><code class="lang-bash">mkdir svn-dist && cp *-source-* svn-dist/ +svn import svn-dist https://dist.apache.org/repos/dist/dev/incubator/apex/${RNAME}-RC1 -m "Apache Apex v${rv}-RC1" +</code></pre> +<h2 id="voting">Voting</h2> +<p>Vote call sample: +<a href="http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAKJfLDPr3CBCfstQJWjchG-ZEYw5P%2Bwv5jN0tfy3EL%2BU%3DBUQgQ%40mail.gmail.com%3E";>http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAKJfLDPr3CBCfstQJWjchG-ZEYw5P%2Bwv5jN0tfy3EL%2BU%3DBUQgQ%40mail.gmail.com%3E</a></p> +<p>Vote result: +<a href="http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAKJfLDNQzMN4zcuTHosU%2BCepF38A_2VL03GOYSc2%3DxxV-9iqMw%40mail.gmail.com%3E";>http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAKJfLDNQzMN4zcuTHosU%2BCepF38A_2VL03GOYSc2%3DxxV-9iqMw%40mail.gmail.com%3E</a></p> +<p>If the vote is not successful, a new RC needs to be built and new vote called. Once the PMC vote passes, proceed with promoting and announcing the release.</p> +<h2 id="promote-release">Promote Release</h2> +<p>Release Nexus staging repository: <a href="http://central.sonatype.org/pages/releasing-the-deployment.html#close-and-drop-or-release-your-staging-repository";>http://central.sonatype.org/pages/releasing-the-deployment.html#close-and-drop-or-release-your-staging-repository</a></p> +<p>Move source release from dist staging to release folder:</p> +<pre><code>rv=3.2.0 +RNAME=apache-apex-core-${rv} +svn mv https://dist.apache.org/repos/dist/dev/incubator/apex/${RNAME}-RC1 https://dist.apache.org/repos/dist/release/incubator/apex/${RNAME} -m "Release ${RNAME}" +</code></pre><h3 id="jira">JIRA</h3> +<p>Close release and all associated tickets (use bulk change workflow transition and turn off notification at bottom of page). +Create version number X.Y.Z+1 for next release</p> +<h3 id="git">git</h3> +<p>Create final release tag:</p> +<pre><code class="lang-bash">rv=3.2.0 +git tag -a "v${rv}" -m "Release ${rv}" "v${rv}-RC2" +git push apache "v${rv}" +</code></pre> +<p>Bump patch version number in release branch (X.Y.Z+1 - otherwise same as when creating new release branch):</p> +<pre><code class="lang-bash">git checkout release-3.2 +dv=3.2.0-SNAPSHOT +rv=3.2.1-SNAPSHOT +for a in `git grep -l "${dv}"`; do echo $a; sed -i 's/'"${dv}"'/'"${rv}"'/g' $a; done +</code></pre> +<p>If there are new artifacts published to Maven repositories consider enabling semantic versioning check for the newly +published libraries.</p> +<p>Commit all changes and push them to the remote git repository:</p> +<pre><code class="lang-bash">git commit -am "Preparing for 3.2.1 development" +git push apache +</code></pre> +<p>Merge <code>@since</code> tag and change log changes to <code>master</code>.</p> +<h2 id="announce-release">Announce Release</h2> +<p>For minor or major release, publish the documentation to the web site prior to updating download page (which will automatically link the documentation). See <a href="https://github.com/apache/incubator-apex-core/tree/master/docs#deployment";>https://github.com/apache/incubator-apex-core/tree/master/docs#deployment</a></p> +<p>Update the download page to reflect the new release: <a href="https://github.com/apache/incubator-apex-site#updating-downloads-page";>https://github.com/apache/incubator-apex-site#updating-downloads-page</a></p> +<p>Send the announcement email, example: +<a href="http://mail-archives.apache.org/mod_mbox/www-announce/201605.mbox/%3CCA%2B5xAo1ZYso6azUBJOkpVtJqM%3DAnJFr_RtjKk9_VusBwgYNS8A%40mail.gmail.com%3E";>http://mail-archives.apache.org/mod_mbox/www-announce/201605.mbox/%3CCA%2B5xAo1ZYso6azUBJOkpVtJqM%3DAnJFr_RtjKk9_VusBwgYNS8A%40mail.gmail.com%3E</a></p> +<h2 id="removing-old-releases">Removing old Releases</h2> +<p>As part of publishing new releases, please determine whether old releases should be deleted. See <a href="http://www.apache.org/dev/release.html#when-to-archive";>release archiving policy</a> for details why. </p> +<p>With a new patch release, the previous patch release can be removed. For example, once 3.3.1 patch is released, we no longer need to have 3.3.0 on the download page. </p> +<p>Once a release branch is no longer supported, we can also remove the last release in that line. For example once <code>release-3.1</code> branch is EOL, releases 3.1.1 (or whatever the latest patch was) can be removed from downloads. </p> + +</div> + + <hr> + <div class="container"> + <footer id="main-footer"> + <p> + Copyright © <span id="copyright-year">2015</span> <a href="http://apache.org";>The Apache Software Foundation</a>, + Licensed under the Apache License, Version 2.0<br> + Apache and the Apache feather logo are trademarks of The Apache Software Foundation.<br> + <a class="footer-link-img" href="http://apache.org";><img src="/images/asf_logo.svg" alt="The Apache Software Foundation"></a> + </p> + </footer> + </div> <!-- /container --> + + <!-- Placed at the end of the document so the pages load faster --> + <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js";></script> + <script src="/js/bootstrap.min.js"></script> + <script> + $('#copyright-year').text((new Date()).getFullYear()); + </script> + +</body> +</html> http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/04ffbb71/content/content/roadmap.html ---------------------------------------------------------------------- diff --git a/content/content/roadmap.html b/content/content/roadmap.html new file mode 100644 index 0000000..1eec430 --- /dev/null +++ b/content/content/roadmap.html @@ -0,0 +1,497 @@ +<html lang="en"><head> + + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> + <meta name="description" content="Apex is an enterprise grade native YARN big data-in-motion platform that unifies stream processing as well as batch processing."> + <meta name="author" content="Apache Software Foundation"> + <link rel="icon" href="favicon.ico"> + + <title>Apache Apex</title> + + <!-- Main Stylesheet --> + <link href="css/main.css" rel="stylesheet"> + + </head> + + <body> + <nav class="navbar navbar-default navbar-static-top" id="main-nav"> + <div class="container"> + + <div class="navbar-header"> + <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="navbar-brand" href="/"> + <img src="images/apex-logo.svg" class="logo" alt="Apache Apex Logo"> + Apache Apex<span class="trademark">™</span> + </a> + </div> + + <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> + <ul class="nav navbar-right navbar-nav"> + <li class="nav-item"> + <a class="nav-link " href="/">Home</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/announcements.html">Announcements</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/community.html">Community</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/docs.html">Docs</a> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Source<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-core.git";>Apex Core (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-core";>Apex Core (Github Mirror)</a></li> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-malhar.git";>Apex Malhar (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-malhar";>Apex Malhar (Github Mirror)</a></li> + </ul> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Apache<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="http://incubator.apache.org/projects/apex.html";>Status Page</a></li> + <li><a href="http://www.apache.org/foundation/how-it-works.html";>Apache Foundation</a></li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + </ul> + </li> + <li class="nav-item"> + <a class="nav-link btn btn-success" href="/downloads.html">Download</a> + </li> + </ul> + + </div> + </nav> + +<div class="container"> + + <h1>Apex Roadmap</h1> + + Summary of key <a target="_blank" href="https://issues.apache.org/jira/issues/?jql=project+in+(APEXCORE,APEXMALHAR)+AND+labels+in+(roadmap)+and+fixVersion+in+(EMPTY,unreleasedVersions())+ORDER+BY+key">JIRAs</a> planned for future releases of Apex Core and Malhar. + + <!-- APEX CORE ROADMAP --> + <h2>Core</h2> + <table class="table table-bordered table-striped"> + <thead> + <tr> + <th scope="col">JIRA</th> + <th scope="col">Summary</th> + <th scope="col">Version</th> + </tr> + </thead> + <tbody> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-119";>APEXCORE-119</a> + </td> + <td title="This JIRA Proposes support for a new type of distributed operator. Currently when an operator is partitioned there is no platform supported mechanism through which partitions can talk to each other. A Distributed operator would have an easy to use platform supported mechanism through which operators in a partitioning can exchange information with each other. Eventually Distributed operators would support running plain old single threaded java code transparently across partitions. + +In summary the goals would be to do the following: + +1 - provide a platform supported fault tolerant mechanism through which operators in a partitioning can talk to each other. +2 - provide a platform supported way to run plain old single threaded java code accross all the partitions of a Distributed operator + +The benefits of implementing this would be huge: + +1 - Using distributed operators we could support large in memory fault tolerant data structures (graphs, maps, arrays) in a fault tolerant way. Like Spark's RDD's but better. +2 - Plain old java code could be used to access and manipulate the data structures, without the user having the learn complex API's like with Spark. + +An implementation proposal and presentation are coming soon."> + Add Support For A New Type Of (Distributed) Operator + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-163";>APEXCORE-163</a> + </td> + <td title="Apex support modification of operator properties at runtime but the current implemenations has the following shortcomings. + +1. Property is not set across all partitions on the same window as individual partitions can be on different windows when property change is initiated from client resulting in inconsistency of data for those windows. I am being generous using the word inconsistent. +2. Sometimes properties need to be set on more than one logical operators at the same time to achieve the change the user is seeking. Today they will be two separate changes happening on two different windows again resulting in inconsistent data for some windows. These would need to happen as a single transaction. +3. If there is an operator failure before a committed checkpoint after an operator property is dynamically changed the operator will restart with the old property and the change will not be re-applied. + +Tim and myself did some brainstorming and we have a proposal to overcome these shortcomings. The main problem in all the above cases is that the property changes are happening out-of-band of data flow and hence independent of windowing. The proposal is to bring the property change request into the in-band dataflow so that they are handled consistently with windowing and handled distributively. + +The idea is to inject a special property change tuple containing the property changes and the identification information of the operator's they affect into the dataflow at the input operator. The tuple will be injected at window boundary after end window and before begin window and as this tuple flows through the DAG the intended operators properties will be modifed. They will all be modified consistently at the same window. The tuple can contain more than one property changes for more than one logical operators and the change will be applied consistently to the different logical operators at the same window. In case of failure the replay of tuples will ensure that the property change gets reapplied at the correct window."> + Dynamic application property changes + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-202";>APEXCORE-202</a> + </td> + <td title="Apache Samoa[https://samoa.incubator.apache.org/] is an abstraction of a collections of streaming machine learning Algorithm. By far, it has integration with Samza, Storm and flink, It is a good start point for Apex to support streaming ML."> + Integration with Samoa + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-231";>APEXCORE-231</a> + </td> + <td title="The Apex engine supports many platform level attributes like operator memory, application window count, container jvm options etc. Today these can only be set at application launch time and cannot be changed once the application is running. + +This issue is to add the ability to change the attributes dynamically even as the application is running. The mechanics of an user requesting the attribute change can be similar to how a user requests property change via the command line client. + +Since each attribute is different the actual backend implementation to affect the changes will most likely be custom handling for different attributes but during the implementation process hopefully some common themes emerge and some amount of reuse possible."> + Ability to configure attributes dynamically + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-232";>APEXCORE-232</a> + </td> + <td title="There are scenarios when new processing code needs to be added to an already running application. There are two scenarios. + +a. A bug is discovered in an operator and an existing operator in the running DAG needs to be replaced. The platform supports shutting down and resuming an application which could be use as a first cut way to do this but there are a couple of drawbacks. + i. This only works when the input source has memory, if it doesn't the messages received during the time the application is down are lost. + ii. Depending on the complexity and state of the application it may take some time for this entire process and the application to get back to running state and this delay may not be acceptable for the downstream components that depend on the output of this application. + +b. A new operator needs to be added to the DAG to take data from an existing operator and do some additional processing. Today this is supported as long as the code for the operator is already in the application libraries. Often this will not be the case as users will not know what the operator will be beforehand when the application is originally launched."> + Ability to add new processing code to the DAG + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-233";>APEXCORE-233</a> + </td> + <td title="There are scenarios where the same object instance needs to be specified for two attributes. Example is partitioner and stats listener, for partitioners that need to affect partitoning based on operator stats the same instance needs to be both. This is not possible to specify using a property file today as it will create two separate instances and can only be done in Java code today. The issue is to request adding this feature."> + Ability to specify single instance objects in configuration + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-234";>APEXCORE-234</a> + </td> + <td title="The current property file specification follows the hadoop configuration file format and this has led to some drawbacks. + a. The names for the properties and attributes are verbose in the configuration file. + b. When there are nested properties in operators the syntax deviates from the bean specification because it introduces some specific keywords in the specification like .prop and ,attr. + +There will already be some changes afoot based on the following + a. When adding ability to specify single instance attributes (https://malhar.atlassian.net/browse/APEXCORE-233) implementing it in the current syntax may not be possible or lead to very unwieldy syntax. + b. There are also other ideas such as one from David to have the ability to specify global application level attributes which possible require rethinking the current syntax. + +Users have also asked for an easier and more consistent way to specify these properties. This issue is to track the ideas and progress of these changes."> + Investigate other ways to specify properties in property files + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-235";>APEXCORE-235</a> + </td> + <td title="Apex can be used for real-time and batch processing as it stands, but there are some aspects of batch processing that can be better supported through explicit constructs. This ticket can serve as umbrella for various features."> + Explicit support for batch processing + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-289";>APEXCORE-289</a> + </td> + <td title="We should support encrypted streams in a DAG for Apex. +Basically there will be 2 ways user can configure the streams for encryption: +1) App wide attributes- Using which all the stream in the DAG will have encrypted channel. +2) Stream based attribute - Using this user can set a certain stream to flow over encrypted channel. + +Encrypted for the streams should done at Network/Buffer Server levels."> + Encrypted Streams in Apex DAG + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-414";>APEXCORE-414</a> + </td> + <td title="Apex core has streaming windows that establish a boundary based on arrival time of events. Many applications require boundaries based on the time of events, which could be a field in the tuple. Some of the operators support this today (time bucketing), but it would be good to provide more generic support for this in the engine itself. "> + Native support for event-time windowing + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXCORE-418";>APEXCORE-418</a> + </td> + <td title="Today Apex has two modes of execution: Embedded mode (everything running in a single JVM) and YARN. There has been a few questions around native support for Mesos. A cursory look suggests that Mesos support can be added by reimplementing the YARN specific portions in the master (AppMasterService, ContainerLauncher) and limited changes to the streaming container driver. + +Mesos has a different model of resource allocation: The master offers resources to the framework while in YARN resources are requested. Apex master needs to implement the "framework scheduler" that is responsible to accept the resources and control the tasks. + +http://mesos.apache.org/documentation/latest/app-framework-development-guide/ + +Tasks are launched through executors, command line and docker executors are provided. + +Apex also requires support to deploy the dependencies to the nodes on which the streaming containers are launched. YARN supports that through resource localization. Mesos supports this through the fetcher, which can copy the resources to the slave node. + +http://mesos.apache.org/documentation/latest/fetcher/ +"> + Support for Mesos + </td> + <td> + + + </td> + </tr> + </tbody> + </table> + + <!-- APEX MALHAR ROADMAP --> + <h2>Malhar</h2> + <table class="table table-bordered table-striped"> + <thead> + <tr> + <th scope="col">JIRA</th> + <th scope="col">Summary</th> + <th scope="col">Version</th> + </tr> + </thead> + <tbody> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1720";>APEXMALHAR-1720</a> + </td> + <td title=""> + Development of Inner Join Operator + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1811";>APEXMALHAR-1811</a> + </td> + <td title="Add new condition for non-equality join predicate (for example, user.zipcode != authzn.zipcode)"> + Add Non-Equality Join Condition + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1818";>APEXMALHAR-1818</a> + </td> + <td title="Once we have ability to generate a subdag, we should take a look at integrating Calcite into Apex. The operator that enables populate DAG, should use Calcite to generate the DAG, given a SQL query."> + Integrate Calcite to support SQL + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1843";>APEXMALHAR-1843</a> + </td> + <td title="[~andyp] I am assigning this to you cause you are the one who first said it. So either you lead it or find a willing lead to get this task to completion. + +The problem with contrib and library modules of malhar is that a ton of dependencies are prescribed as optional. The motive behind it was that the users of these libraries are given an opportunity to keep the size of the dependency-included packages to bare minimum. It comes at a cost that the dependency now has to be manually figured out. This is a complete misuse of the optional dependency, IMO. It defeats the purpose of maven having dependency management as one of the biggest features of it. + +So keep things sane - the proposed compromise is that we start creating smaller discreet packages for discrete technologies."> + Split Malhar Library and Malhar Contrib package into baby packages + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1897";>APEXMALHAR-1897</a> + </td> + <td title="ManagedState is described in the document below: + +https://docs.google.com/document/d/1gRWN9ufKSZSZD0N-pthlhpC9TZ8KwJ6hJlAX6nxl5f8/edit#heading=h.z87ti1fwyt0t";> + Large operator state management + </td> + <td> + + + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR/fixforversion/12334637";>3.4.0</a> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1938";>APEXMALHAR-1938</a> + </td> + <td title="Currently Apex engine provides operator checkpointing in Hdfs ( with Hdfs backed StorageAgents i.e. FSStorageAgent & AsyncFSStorageAgent ) +As operator check-pointing is critical functionality of Apex streaming platform to ensure fault tolerant behavior, platform should also provide alternate StorageAgents which will work seamlessly with large applications that requires Exactly once semantics. +HDFS read/write latency is limited and doesn't improve beyond certain point because of disk io & staging writes. Having alternate strategy to this check-pointing in fault tolerant distributed in-memory grid would ensure application stability and performance is not impacted by checkpointing + +*This feature will add below functionalities* +* A KeyValue store interface which is used by In-memory checkpointing storage agent. +* Abstract implementation of KeyValue storage agent which can be configured with concrete implementation of KeyValue store for checkpointing. +* Concrete implementation of In memory storage agent for Apache Geode + +*This feature depends on below APEX core feature* +https://issues.apache.org/jira/browse/APEXCORE-283 +* Interface for storage agent to provide application id +* Stram client changes to pass applicationId"> + Operator checkpointing in distributed in-memory store + </td> + <td> + + + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR/fixforversion/12334637";>3.4.0</a> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1939";>APEXMALHAR-1939</a> + </td> + <td title=""> + Stream API + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1942";>APEXMALHAR-1942</a> + </td> + <td title="We would like to contribute the Apache Geode(http://geode.incubator.apache.org/) Operator support for Apex. +It will basically be implementation for writing to geode region. +This is in continuation with the Operator checkpointing alternative under review (MLHR-1938)"> + Apex Operator for Apache Geode. + </td> + <td> + + + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR/fixforversion/12334637";>3.4.0</a> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-1999";>APEXMALHAR-1999</a> + </td> + <td title="Flink streaming is compatible with Apache Storm interfaces and therefore allows reusing code that was implemented for Storm. +Details can be found here. +https://ci.apache.org/projects/flink/flink-docs-master/apis/storm_compatibility.html +This jira item can contain tasks for providing similar support in Apex"> + Running a Storm topology on Apex. + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-2026";>APEXMALHAR-2026</a> + </td> + <td title="Add libraryies for spooling datastructures to a key value store. There are several customer use cases which require spooled data structures. + +1 - Some operators like AbstractFileInputOperator have ever growing state. This is an issue because eventually the state of the operator will grow larger than the memory allocated to the operator, which will cause the operator to perpetually fail. However if the operator's datastructures are spooled then the operator will never run out of memory. + +2 - Some users have requested for the ability to maintain a map as well as a list of keys over which to iterate. Most key value stores don't provide this functionality. However, with spooled datastructures this functionality can be provided by maintaining a spooled map and an iterable set of keys. + +3 - Some users have requested building graph databases within APEX. This would require implementing a spooled graph data structure. + +4 - Another use case for spooled data structures is database operators. Database operators need to write data to a data base, but sometimes the database is down. In this case most of the database operators repeatedly fail until the database comes back up. In order to avoid constant failures the database operator need to writes data to a queue when the data base is down, then when the database is up the operator need to take data from the queue and write it to the database. In the case of a database failure this queue will grow larger than the total amount of memory available to the operator, so the queue should be spooled in order to prevent the operator from failing. + +5 - Any operator which needs to maintain a large data structure in memory currently needs to have that data serialized and written out to HDFS with every checkpoint. This is costly when the data structure is large. If the data structure is spooled, then only the changes to the data structure are written out to HDFS instead of the entire data structure. + +6 - Also building an Apex Native database for aggregations requires indices. These indices need to take the form of spooled data structures. + +7 - In the future any operator which needs to maintain a data structure larger than the memory available to it will need to spool the data structure."> + Spill-able Datastructures + </td> + <td> + + + </td> + </tr> + <tr> + <td> + <a target="_blank" href="https://issues.apache.org/jira/browse/APEXMALHAR-2089";>APEXMALHAR-2089</a> + </td> + <td title="Apex should provide a runner for Beam. This ticket is a proxy for BEAM-261 as the implementation should probably live in the Beam repository. +"> + Apache Beam support + </td> + <td> + + + </td> + </tr> + </tbody> + </table> + +</div> + + <hr> + <div class="container"> + <footer id="main-footer"> + <p> + Copyright © <span id="copyright-year">2015</span> <a href="http://apache.org";>The Apache Software Foundation</a>, + Licensed under the Apache License, Version 2.0<br> + Apache and the Apache feather logo are trademarks of The Apache Software Foundation.<br> + <a class="footer-link-img" href="http://apache.org";><img src="/images/asf_logo.svg" alt="The Apache Software Foundation"></a> + </p> + </footer> + </div> <!-- /container --> + + <!-- Placed at the end of the document so the pages load faster --> + <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js";></script> + <script src="/js/bootstrap.min.js"></script> + <script> + $('#copyright-year').text((new Date()).getFullYear()); + </script> + +</body> +</html> http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/04ffbb71/content/content/verification.html ---------------------------------------------------------------------- diff --git a/content/content/verification.html b/content/content/verification.html new file mode 100644 index 0000000..1521820 --- /dev/null +++ b/content/content/verification.html @@ -0,0 +1,176 @@ +<html lang="en"><head> + + <meta charset="utf-8"> + <meta http-equiv="X-UA-Compatible" content="IE=edge"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> + <meta name="description" content="Apex is an enterprise grade native YARN big data-in-motion platform that unifies stream processing as well as batch processing."> + <meta name="author" content="Apache Software Foundation"> + <link rel="icon" href="favicon.ico"> + + <title>Apache Apex</title> + + <!-- Main Stylesheet --> + <link href="css/main.css" rel="stylesheet"> + + </head> + + <body> + <nav class="navbar navbar-default navbar-static-top" id="main-nav"> + <div class="container"> + + <div class="navbar-header"> + <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-example-navbar-collapse-1" aria-expanded="false"> + <span class="sr-only">Toggle navigation</span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + <span class="icon-bar"></span> + </button> + <a class="navbar-brand" href="/"> + <img src="images/apex-logo.svg" class="logo" alt="Apache Apex Logo"> + Apache Apex<span class="trademark">™</span> + </a> + </div> + + <div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"> + <ul class="nav navbar-right navbar-nav"> + <li class="nav-item"> + <a class="nav-link " href="/">Home</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/announcements.html">Announcements</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/community.html">Community</a> + </li> + <li class="nav-item"> + <a class="nav-link " href="/docs.html">Docs</a> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Source<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-core.git";>Apex Core (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-core";>Apex Core (Github Mirror)</a></li> + <li><a href="https://git-wip-us.apache.org/repos/asf?p=incubator-apex-malhar.git";>Apex Malhar (ASF)</a></li> + <li><a href="https://github.com/apache/incubator-apex-malhar";>Apex Malhar (Github Mirror)</a></li> + </ul> + </li> + <li class="nav-item"> + <a href="#" data-toggle="dropdown" class="dropdown-toggle nav-link">Apache<b class="caret"></b></a> + <ul class="dropdown-menu"> + <li><a href="http://incubator.apache.org/projects/apex.html";>Status Page</a></li> + <li><a href="http://www.apache.org/foundation/how-it-works.html";>Apache Foundation</a></li> + <li><a href="http://www.apache.org/licenses/";>Apache License</a></li> + <li><a href="http://www.apache.org/foundation/sponsorship.html";>Sponsorship</a></li> + <li><a href="http://www.apache.org/foundation/thanks.html";>Thanks</a></li> + </ul> + </li> + <li class="nav-item"> + <a class="nav-link btn btn-success" href="/downloads.html">Download</a> + </li> + </ul> + + </div> + </nav> + +<div class="container"> + + <h1 id="how-to-verify-apache-apex-release-candidate-builds">How to verify Apache Apex release candidate builds</h1> +<h2 id="prerequisites">Prerequisites</h2> +<p>Download the KEYS file and import it. Verify that the key just imported is correct.</p> +<pre><code class="lang-bash">wget https://dist.apache.org/repos/dist/release/incubator/apex/KEYS +gpg --import KEYS +gpg --fingerprint <key-id> +</code></pre> +<p>The <key-id> of the release manager can be found out in the <em>PGP key</em> link in the mail. It would be alphanumric number like D920A98Cbb. The above step can be skipped if you have already imported key of the current release manager.</p> +<p>More about key verification at <a href="http://www.apache.org/info/verification.html";>http://www.apache.org/info/verification.html</a></p> +<p>You can also create your own key which is required if you would like to sign the build. This step is optional.</p> +<pre><code class="lang-bash">gpg --gen-key +</code></pre> +<p>Please provide 4096 as keysize while generating the key.</p> +<h2 id="file-integrity-check">File integrity check</h2> +<p>Download all files present in staging directory of the RC build. Staging directory link is shared in VOTE thread of the release candidate. It is recommended that these files are downloaded in a clean directory.</p> +<pre><code class="lang-bash">wget -r -np -nd <staging-area-link>/ +</code></pre> +<p>Note the link should end with "/".</p> +<p>Define the apex release candidate variable. We will set it up <em>apex-3.4.0</em> as an example.</p> +<pre><code class="lang-bash">APEX_RELEASE_CANDIDATE=apex-3.4.0 +</code></pre> +<p>Verify integrity of tar.gz file:</p> +<pre><code class="lang-bash">gpg --verify $APEX_RELEASE_CANDIDATE-source-release.tar.gz.asc +md5sum --check $APEX_RELEASE_CANDIDATE-source-release.tar.gz.md5 +sha512sum --check $APEX_RELEASE_CANDIDATE-source-release.tar.gz.sha +</code></pre> +<p>Verify integrity of .zip file:</p> +<pre><code class="lang-bash">gpg --verify $APEX_RELEASE_CANDIDATE-source-release.zip.asc +md5sum --check $APEX_RELEASE_CANDIDATE-source-release.zip.md5 +sha512sum --check $APEX_RELEASE_CANDIDATE-source-release.zip.sha +</code></pre> +<h2 id="source-code-verification">Source code verification</h2> +<p>You can extract source either using .tar.gz file or .zip file.</p> +<h3 id="using-tar-gz-source">Using .tar.gz source</h3> +<p>Extract source using .tar.gz:</p> +<pre><code class="lang-bash">tar -zxvf $APEX_RELEASE_CANDIDATE-source-release.tar.gz +</code></pre> +<h3 id="using-zip-source">Using .zip source</h3> +<pre><code class="lang-bash">unzip $APEX_RELEASE_CANDIDATE-source-release.zip +</code></pre> +<p>Any of the two commands above will create a directory named after <em>apex release candidate</em>.</p> +<p>Change directory:</p> +<pre><code class="lang-bash">cd $APEX_RELEASE_CANDIDATE +</code></pre> +<h3 id="miscellaneous-checks-part-i-pre-compilation-">Miscellaneous checks: Part I (Pre Compilation)</h3> +<p>Existence of LICENSE, NOTICE, README.md and CHANGELOG.md files. +Please make sure these files are present in the parent folder after extracting the zip or tar. The CHANGELOG.md file should contain change log for the current release.</p> +<p>No unexpected binary files in the sources. +The extracted directory should not contain any binary. Need to test this before compiling the source code.</p> +<pre><code class="lang-bash">find . -type f -name '*.*' | sed 's|.*\.||' | sort -u +</code></pre> +<p>Please make sure no binary extension is listed here. Likely candidates are .jar files. Additionally, for Malhar, demo package files, i.e. .apa files.</p> +<h3 id="check-for-compilation-license-headers-etc-">Check for compilation, license headers, etc.</h3> +<p>For Apex:</p> +<pre><code class="lang-bash">mvn clean apache-rat:check verify -Dlicense.skip=false install +</code></pre> +<p>For Malhar:</p> +<pre><code class="lang-bash">mvn clean apache-rat:check verify -Dlicense.skip=false -Pall-modules install +</code></pre> +<p>Jars should be installed in your maven repository, typically in <em>~/.m2/repository/org/apache/apex/</em> directory</p> +<p>Following step is optional and needs the key generated by you as in prerequisite section given above.</p> +<pre><code class="lang-bash">mvn verify -Papache-release -DskipTests +</code></pre> +<h2 id="launch-demos">Launch demos</h2> +<p>For verification of Apex build, check backward compatibility with respect to Malhar. Locally update <em>apex.core.version</em> in Malhar to point to <em>to-be released</em> version of apex-core and recompile Malhar.</p> +<p>Launch few demos to make sure everything is working fine using <em>apex</em> utitlity. The apex script is present at <apex-core-folder>/engine/src/main/scripts/apex . If apex-core is being verified, then simply use engine/src/main/scripts/apex to launch the script. Otherwise, prefix it with apex-core folder path.</p> +<pre><code class="lang-bash">engine/src/main/scripts/apex +</code></pre> +<p>You will get apex prompt, where demos can be launched.</p> +<pre><code class="lang-bash">apex> launch <demo-apa-file> +</code></pre> +<p>Demo apa files are typically in malhar/demos directory.</p> +<h2 id="miscellaneous-checks-part-ii-post-compilation-">Miscellaneous checks: Part II (Post compilation)</h2> +<p>Correct artifact names: +All the apex artifacts generated in local maven build directory should contain classs jar, sources jars, javadocs jar. All the artifacts are under <em>~/.m2/repository/org/apache/apex/</em> directory on your local machine.</p> + +</div> + + <hr> + <div class="container"> + <footer id="main-footer"> + <p> + Copyright © <span id="copyright-year">2015</span> <a href="http://apache.org";>The Apache Software Foundation</a>, + Licensed under the Apache License, Version 2.0<br> + Apache and the Apache feather logo are trademarks of The Apache Software Foundation.<br> + <a class="footer-link-img" href="http://apache.org";><img src="/images/asf_logo.svg" alt="The Apache Software Foundation"></a> + </p> + </footer> + </div> <!-- /container --> + + <!-- Placed at the end of the document so the pages load faster --> + <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js";></script> + <script src="/js/bootstrap.min.js"></script> + <script> + $('#copyright-year').text((new Date()).getFullYear()); + </script> + +</body> +</html> http://git-wip-us.apache.org/repos/asf/incubator-apex-site/blob/04ffbb71/content/downloads.html ---------------------------------------------------------------------- diff --git a/content/downloads.html b/content/downloads.html index 07a1627..767e8ae 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -204,29 +204,29 @@ <tbody> <tr> <td> - 3.3.1-incubating + 3.4.0 <span class="latest-tag">(latest)</span> </td> <td> - 2016-03-02 + 2016-05-24 </td> <td> - <a href="http://www.apache.org/dyn/closer.lua/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.zip";>malhar-3.3.1-incubating-source-release.zip</a><br> + <a href="http://www.apache.org/dyn/closer.lua/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.zip";>malhar-3.4.0-source-release.zip</a><br> <small> ( - <a href="https://dist.apache.org/repos/dist/release/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.zip.asc";>pgp</a>, - <a href="https://dist.apache.org/repos/dist/release/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.zip.md5";>md5</a>, - <a href="https://dist.apache.org/repos/dist/release/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.zip.sha";>sha1</a> + <a href="https://dist.apache.org/repos/dist/release/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.zip.asc";>pgp</a>, + <a href="https://dist.apache.org/repos/dist/release/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.zip.md5";>md5</a>, + <a href="https://dist.apache.org/repos/dist/release/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.zip.sha";>sha1</a> ) </small> </td> <td> - <a href="http://www.apache.org/dyn/closer.lua/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.tar.gz";>malhar-3.3.1-incubating-source-release.tar.gz</a><br> + <a href="http://www.apache.org/dyn/closer.lua/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.tar.gz";>malhar-3.4.0-source-release.tar.gz</a><br> <small> ( - <a href="https://dist.apache.org/repos/dist/release/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.tar.gz.asc";>pgp</a>, - <a href="https://dist.apache.org/repos/dist/release/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.tar.gz.md5";>md5</a>, - <a href="https://dist.apache.org/repos/dist/release/incubator/apex/malhar/v3.3.1-incubating/malhar-3.3.1-incubating-source-release.tar.gz.sha";>sha1</a> + <a href="https://dist.apache.org/repos/dist/release/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.tar.gz.asc";>pgp</a>, + <a href="https://dist.apache.org/repos/dist/release/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.tar.gz.md5";>md5</a>, + <a href="https://dist.apache.org/repos/dist/release/apex/apache-apex-malhar-3.4.0/apache-apex-malhar-3.4.0-source-release.tar.gz.sha";>sha1</a> ) </small> </td>
