[apex-site] branch master updated: APEXCORE-799.Fix-documentation-for-release-process

2017-12-02 Thread thw 
This is an automated email from the ASF dual-hosted git repository.

thw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/apex-site.git


The following commit(s) were added to refs/heads/master by this push:
 new 56f9ea2  APEXCORE-799.Fix-documentation-for-release-process
56f9ea2 is described below

commit 56f9ea25fb84838af2e3e31e7e7648b743d05300
Author: Ananth 
AuthorDate: Thu Nov 30 22:01:33 2017 +1100

APEXCORE-799.Fix-documentation-for-release-process
---
 src/md/release.md | 24 
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/src/md/release.md b/src/md/release.md
index d257a85..7c403a7 100644
--- a/src/md/release.md
+++ b/src/md/release.md
@@ -173,6 +173,9 @@ Javadoc will be automatically deployed for registered 
branches. For minor releas
 
 
https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/apex.conf
 
+This essentially means that the file above needs to be edited for the release 
number in the array of releases. Ensure that you verify the success of the 
build in the build bot [url](https://ci.apache.org/builders)
+Also note that the build bot is a nightly trigger. Wait for atleast the next 
build trigger to kick in before raising a ticket with the infra team if you see 
that the javadoc is not appearing as expected.
+
 ### User Documentation
 
 The documentation will be generated as static HTML files into the `apex-site` 
repository, [separated by version 
(X.Y)](https://github.com/apache/apex-site/tree/asf-site/docs).
@@ -237,7 +240,7 @@ Note that the vote result email should have the subject 
prefixed with `[RESULT]`
 
 If the vote is not successful, a new RC needs to be built and new vote called. 
Once the PMC vote passes, proceed with promoting and announcing the release.
 
-## Promote Release
+## Promote Release (To be done by a PMC member)
 
 Release Nexus staging repository: 
http://central.sonatype.org/pages/releasing-the-deployment.html#close-and-drop-or-release-your-staging-repository
 
@@ -248,14 +251,17 @@ RNAME=apache-apex-core-${rv}
 svn mv https://dist.apache.org/repos/dist/dev/apex/${RNAME}-RC1 
https://dist.apache.org/repos/dist/release/apex/${RNAME} -m "Release ${RNAME}"
 ```
 
-### JIRA
+### JIRA (To be done by a PMC member)
 
-Close release and all associated tickets (use bulk change workflow transition 
and **turn off notification** at bottom of page to not flood the mailing list). 
-Create version number X.Y.Z+1 for next release (to be done by PMC member).
+The closing of the JIRA item has to be done by a PMC member as the turn off 
notification for bulk changes in JIRA is an option available only for the PMC 
members.
+Request a PMC member to close release and all associated tickets (using bulk 
change workflow transition and **turn off notification** at bottom of page to 
not flood the mailing list). 
+Create version number X.Y.Z+1 for next release (also to be done by PMC member).
 
 ### git
 
 Create final release tag:
+
+Note that the value of RC2 in the example below assumes that the second 
release candidate RC2 is the final release tag. Change this value accordingly.
 ```bash
 rv=3.4.0
 git tag -a "v${rv}" -m "Release ${rv}" "v${rv}-RC2"
@@ -277,6 +283,7 @@ git commit --author "Apex Dev " -am 
"Preparing for 3.4.1 de
 git push apache
 ```
 Cherry-pick `@since` tag and change log changes from release tag to `master`.
+Push the changes to apache
 
 ### Documentation
 
@@ -300,11 +307,20 @@ Update the download page to reflect the new release: 
https://github.com/apache/a
 
 Publish the web site, see [apex-site 
repository](https://github.com/apache/apex-site#contributing) for instructions.
 
+Publishing the site involves running the build.sh script from the master 
branch and performing a git push from the asf-site branch. The local repo is 
automatically switched to the asf-site branch when the build.sh script is 
invoked from the master.
+
 ## Announce Release
 
 Send the announcement email, example:
 
http://mail-archives.apache.org/mod_mbox/www-announce/201605.mbox/%3CCA%2B5xAo1ZYso6azUBJOkpVtJqM%3DAnJFr_RtjKk9_VusBwgYNS8A%40mail.gmail.com%3E
 
+Please include the annou...@apache.org as one of the recipients. Also make 
sure that the email is being sent from the yourem...@apache.org email id. Note 
that the email to annou...@apache.org will bounce back if yourem...@apache.org 
email is not configured for the alias address from which you are sending this 
email.
+The release email must be visible under [announce mailing 
list](https://lists.apache.org/list.html?annou...@apache.org)
+ 
+## Anouncements page
+
+Edit the announcements page to publish the feed about the new release by 
editing the markup file announcements-latest.md under the source tree.
+
 ## Removing old Releases
 
 As part of publishing new releases, please determine whether old releases 
should be deleted. See [release archiving

[apex-core] branch master updated: APEXCORE-800 Disable the disk health checker service for StramMiniClusterTest

2017-12-02 Thread thw 
This is an automated email from the ASF dual-hosted git repository.

thw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/apex-core.git


The following commit(s) were added to refs/heads/master by this push:
 new 646674b  APEXCORE-800 Disable the disk health checker service for 
StramMiniClusterTest
646674b is described below

commit 646674ba9810b60d6e51950f6d12eb8bd5b0894a
Author: Vlad Rozov 
AuthorDate: Thu Nov 30 11:19:43 2017 -0800

APEXCORE-800 Disable the disk health checker service for 
StramMiniClusterTest
---
 .../datatorrent/stram/StramMiniClusterTest.java| 23 ++
 1 file changed, 15 insertions(+), 8 deletions(-)

diff --git 
a/engine/src/test/java/com/datatorrent/stram/StramMiniClusterTest.java 
b/engine/src/test/java/com/datatorrent/stram/StramMiniClusterTest.java
index 0e3c0ab..bbb7c1c 100644
--- a/engine/src/test/java/com/datatorrent/stram/StramMiniClusterTest.java
+++ b/engine/src/test/java/com/datatorrent/stram/StramMiniClusterTest.java
@@ -63,6 +63,7 @@ import org.apache.hadoop.yarn.api.records.ResourceRequest;
 import org.apache.hadoop.yarn.client.api.AMRMClient;
 import org.apache.hadoop.yarn.client.api.AMRMClient.ContainerRequest;
 import org.apache.hadoop.yarn.conf.YarnConfiguration;
+import org.apache.hadoop.yarn.exceptions.YarnException;
 import org.apache.hadoop.yarn.server.MiniYARNCluster;
 import org.apache.hadoop.yarn.server.resourcemanager.ClientRMService;
 import org.apache.hadoop.yarn.util.Records;
@@ -90,6 +91,7 @@ import com.datatorrent.stram.webapp.StramWebServices;
 
 import static java.lang.Thread.sleep;
 import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
 
 /**
  * The purpose of this test is to verify basic streaming application deployment
@@ -126,6 +128,7 @@ public class StramMiniClusterTest
 conf.setInt("yarn.nodemanager.vmem-pmem-ratio", 20); // workaround to 
avoid containers being killed because java allocated too much vmem
 conf.set("yarn.scheduler.capacity.root.queues", "default");
 conf.set("yarn.scheduler.capacity.root.default.capacity", "100");
+conf.setBoolean(YarnConfiguration.NM_DISK_HEALTH_CHECK_ENABLE, false);
 conf.set(YarnConfiguration.NM_ADMIN_USER_ENV, 
String.format("JAVA_HOME=%s,CLASSPATH=%s", System.getProperty("java.home"), 
getTestRuntimeClasspath()));
 conf.set(YarnConfiguration.NM_ENV_WHITELIST, 
YarnConfiguration.DEFAULT_NM_ENV_WHITELIST.replaceAll("JAVA_HOME,*", ""));
 
@@ -165,22 +168,26 @@ public class StramMiniClusterTest
 }
   }
 
-  @Test
-  public void testSetupShutdown() throws Exception
+  private void checkNodeState() throws YarnException
   {
-GetClusterNodesRequest request =
-Records.newRecord(GetClusterNodesRequest.class);
+GetClusterNodesRequest request = 
Records.newRecord(GetClusterNodesRequest.class);
 ClientRMService clientRMService = 
yarnCluster.getResourceManager().getClientRMService();
 GetClusterNodesResponse response = 
clientRMService.getClusterNodes(request);
 List nodeReports = response.getNodeReports();
 LOG.info("{}", nodeReports);
 
 for (NodeReport nr: nodeReports) {
-  LOG.info("Node: {}", nr.getNodeId());
-  LOG.info("Total memory: {}", nr.getCapability());
-  LOG.info("Used memory: {}", nr.getUsed());
-  LOG.info("Number containers: {}", nr.getNumContainers());
+  if (!nr.getNodeState().isUnusable()) {
+return;
+  }
 }
+fail("Yarn Mini cluster should have at least one usable node.");
+  }
+
+  @Test
+  public void testSetupShutdown() throws Exception
+  {
+checkNodeState();
 
 JarHelper jarHelper = new JarHelper();
 LOG.info("engine jar: {}", jarHelper.getJar(StreamingAppMaster.class));

-- 
To stop receiving notification emails like this one, please contact
['"commits@apex.apache.org" '].

[apex-site] branch asf-site updated: from 56f9ea25fb84838af2e3e31e7e7648b743d05300

2017-12-02 Thread thw 
This is an automated email from the ASF dual-hosted git repository.

thw pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/apex-site.git


The following commit(s) were added to refs/heads/asf-site by this push:
 new 460cef1  from 56f9ea25fb84838af2e3e31e7e7648b743d05300
460cef1 is described below

commit 460cef175a55fbc8486aa8464c12e01a7fcd590c
Author: Apex Dev 
AuthorDate: Sat Dec 2 07:31:48 2017 -0800

from 56f9ea25fb84838af2e3e31e7e7648b743d05300
---
 content/downloads.html |  8 
 content/release.html   | 20 +++-
 2 files changed, 19 insertions(+), 9 deletions(-)

diff --git a/content/downloads.html b/content/downloads.html
index e518b34..34b0519 100644
--- a/content/downloads.html
+++ b/content/downloads.html
@@ -110,7 +110,7 @@
   (latest)
 
 
-  2017-05-05
+  2017-05-04
 
 
   http://www.apache.org/dyn/closer.lua/apex/apache-apex-core-3.6.0/apache-apex-core-3.6.0-source-release.zip;>apex-3.6.0-source-release.zip
@@ -143,7 +143,7 @@
   (latest)
 
 
-  2016-12-11
+  2016-12-10
 
 
   http://www.apache.org/dyn/closer.lua/apex/apache-apex-core-3.5.0/apache-apex-core-3.5.0-source-release.zip;>apex-3.5.0-source-release.zip
@@ -192,7 +192,7 @@
   (latest)
 
 
-  2017-11-12
+  2017-11-11
 
 
   http://www.apache.org/dyn/closer.lua/apex/apache-apex-malhar-3.8.0/apache-apex-malhar-3.8.0-source-release.zip;>malhar-3.8.0-source-release.zip
@@ -225,7 +225,7 @@
   (latest)
 
 
-  2017-04-01
+  2017-03-31
 
 
   http://www.apache.org/dyn/closer.lua/apex/apache-apex-malhar-3.7.0/apache-apex-malhar-3.7.0-source-release.zip;>malhar-3.7.0-source-release.zip
diff --git a/content/release.html b/content/release.html
index 0393b0c..060d315 100644
--- a/content/release.html
+++ b/content/release.html
@@ -185,6 +185,8 @@ svn import svn-dist 
https://dist.apache.org/repos/dist/dev/apex/${RNAME}-RC1 -m
 Javadoc
 Javadoc will be automatically deployed for registered branches. For minor 
releases, add the new branch to the buildbot configuration:
 https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/apex.conf;>https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/apex.conf
+This essentially means that the file above needs to be edited for the 
release number in the array of releases. Ensure that you verify the success of 
the build in the build bot https://ci.apache.org/builders;>url
+Also note that the build bot is a nightly trigger. Wait for atleast the next 
build trigger to kick in before raising a ticket with the infra team if you see 
that the javadoc is not appearing as expected.
 User Documentation
 The documentation will be generated as static HTML files into the 
apex-site repository, https://github.com/apache/apex-site/tree/asf-site/docs;>separated by 
version (X.Y).
 Note You need Python 2.7+ and http://www.mkdocs.org/;>mkdocs with patch for issue https://github.com/mkdocs/mkdocs/issues/859;>mkdocs #859 on top of 
the currently available version 0.15.3. After installing mkdocs with pip, run 
the following to obtain this build:
@@ -227,17 +229,19 @@ git push
 https://lists.apache.org/thread.html/2887965dba350e181d06aa02af19d8d47c82eb6f0b47eef8dce14fe8@%3Cdev.apex.apache.org%3E;>https://lists.apache.org/thread.html/2887965dba350e181d06aa02af19d8d47c82eb6f0b47eef8dce14fe8@%3Cdev.apex.apache.org%3E
 Note that the vote result email should have the subject prefixed with 
[RESULT].
 If the vote is not successful, a new RC needs to be built and new vote 
called. Once the PMC vote passes, proceed with promoting and announcing the 
release.
-Promote Release
+Promote Release (To be 
done by a PMC member)
 Release Nexus staging repository: http://central.sonatype.org/pages/releasing-the-deployment.html#close-and-drop-or-release-your-staging-repository;>http://central.sonatype.org/pages/releasing-the-deployment.html#close-and-drop-or-release-your-staging-repository
 Move source release from dist staging to release folder (to be done by PMC 
member):
 rv=3.4.0
 RNAME=apache-apex-core-${rv}
 svn mv https://dist.apache.org/repos/dist/dev/apex/${RNAME}-RC1 
https://dist.apache.org/repos/dist/release/apex/${RNAME} -m Release 
${RNAME}
-JIRA
-Close release and all associated tickets (use bulk change workflow 
transition and turn off notification at bottom of page to not 
flood the mailing list). 
-Create version number X.Y.Z+1 for next release (to be done by PMC member).
+JIRA (To be done by a 
PMC member)
+The closing of the JIRA item has to be done by a PMC member as the turn off 
notification for bulk changes in JIRA is an option available only for the PMC 
members.
+Request a PMC member to close release and all