better markdown
Project: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/commit/c5d54a9b Tree: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/tree/c5d54a9b Diff: http://git-wip-us.apache.org/repos/asf/incubator-streams-master/diff/c5d54a9b Branch: refs/heads/master Commit: c5d54a9bcf7266d16180fec336964b343650feee Parents: 88b9c36 Author: Steve Blackmon @steveblackmon <[email protected]> Authored: Tue Mar 1 15:45:32 2016 -0600 Committer: Steve Blackmon @steveblackmon <[email protected]> Committed: Tue Mar 1 15:45:32 2016 -0600 ---------------------------------------------------------------------- pom.xml | 14 +- src/site/markdown/dependency-info.md | 2 +- src/site/markdown/downloads.md | 4 +- src/site/markdown/faq.md | 24 +-- src/site/markdown/release-setup.md | 89 ++++---- src/site/markdown/release.md | 345 ++++++++++++++---------------- 6 files changed, 226 insertions(+), 252 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/c5d54a9b/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 3eca798..af2ff94 100644 --- a/pom.xml +++ b/pom.xml @@ -36,7 +36,8 @@ <inceptionYear>2012</inceptionYear> - <url>http://people.apache.org/sblackmon/${project.version}/${project.artifactId}</url> + <!-- <url>http://people.apache.org/sblackmon/${project.version}/${project.artifactId}</url> --> + <url>http://streams.incubator.apache.org/site/${project.version}/${project.artifactId}</url> <licenses> <license> @@ -236,9 +237,13 @@ <url>${snapshot.repository.url}</url> </snapshotRepository> <site> - <id>site.streams.sblackmon.home</id> - <url>scp://people.apache.org/home/sblackmon/public_html/0.3-incubating-SNAPSHOT/streams-master/</url> + <id>site.streams.master</id> + <url>scm:svn:https://svn.apache.org/repos/infra/websites/production/streams/content/site/${project.version}/${project.artifactId}/</url> </site> + <!-- <site> + <id>site.streams.sblackmon.home</id> + <url>scp://people.apache.org/home/sblackmon/public_html/${project.version}/${project.artifactId}/</url> + </site> --> </distributionManagement> <properties> @@ -705,6 +710,9 @@ <plugin> <artifactId>maven-site-plugin</artifactId> <version>${site.plugin.version}</version> + <configuration> + <topSiteURL>scm:svn:https://svn.apache.org/repos/infra/websites/production/streams/content/site/${project.version}/${project.artifactId}</topSiteURL> + </configuration> </plugin> <plugin> <groupId>com.github.ferstl</groupId> http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/c5d54a9b/src/site/markdown/dependency-info.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/dependency-info.md b/src/site/markdown/dependency-info.md index 44d4e21..97c6af6 100644 --- a/src/site/markdown/dependency-info.md +++ b/src/site/markdown/dependency-info.md @@ -4,7 +4,7 @@ This project uses [Maven](http://maven.apache.org/ "Maven") for dependency manag Below are some examples of how to import Streams artifacts into your project. -Please note that your project should import multiple artifacts corresponding to the Components in your stream and the Runtime used to execute your stream. +Please note that your project should import multiple artifacts corresponding to the Components in your stream(s) and the Runtime used to execute your stream(s). You should *not* import streams-master (depicted below), because it does not do anything interesting. http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/c5d54a9b/src/site/markdown/downloads.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/downloads.md b/src/site/markdown/downloads.md index 9d055d4..5226618 100644 --- a/src/site/markdown/downloads.md +++ b/src/site/markdown/downloads.md @@ -4,6 +4,6 @@ All downloads can be verified using Apache Streams code signing. ### Streams Project -| Version | Source | asc | md5 | sha1 | +| Version | Source | |---------|--------| -| 0.2-incubating | [zip](https://dist.apache.org/repos/dist/release/incubator/streams/releases/streams-project/streams-project/streams-project-0.2-incubating-source-release.zip) | [asc](https://dist.apache.org/repos/dist/release/incubator/streams/releases/0.2-incubating/streams-project/streams-project-0.2-incubating-source-release.zip.asc) | [md5](https://dist.apache.org/repos/dist/release/incubator/streams/releases/0.2-incubating/streams-project/streams-project-0.2-incubating-source-release.zip.md5) | [sha1](https://dist.apache.org/repos/dist/release/incubator/streams/releases/0.2-incubating/streams-project/streams-project-0.2-incubating-source-release.zip.sha1) | +| 0.2-incubating | [zip](https://dist.apache.org/repos/dist/release/incubator/streams/releases/streams-project/streams-project/streams-project-0.1-incubating-source-release.zip) ([asc](https://dist.apache.org/repos/dist/release/incubator/streams/releases/0.2-incubating/streams-project/streams-project-0.1-incubating-source-release.zip.asc) [md5](https://dist.apache.org/repos/dist/release/incubator/streams/releases/0.2-incubating/streams-project/streams-project-0.1-incubating-source-release.zip.md5) [sha1](https://dist.apache.org/repos/dist/release/incubator/streams/releases/0.2-incubating/streams-project/streams-project-0.1-incubating-source-release.zip.sha1)) | \ No newline at end of file http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/c5d54a9b/src/site/markdown/faq.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/faq.md b/src/site/markdown/faq.md index 0cb87b5..151852a 100644 --- a/src/site/markdown/faq.md +++ b/src/site/markdown/faq.md @@ -29,9 +29,9 @@ Apache Streams is not * one-size-fits-all * only useful for projects fully dedicated to activity streams datasets -The primary Streams git repository incubator-streams (org.apache.streams:streams-project) contains a library of modules inputs, outputs, and reusable components for transforming and enriching data streams. Similar modules can also be hosted externally - so long as they publish maven artifacts compatible with your version of streams, you can import and use them in your streams easily. +The primary Streams git repository incubator-streams (org.apache.streams:streams-project) contains a library of modules inputs, outputs, and reusable components for tranforming and enriching data streams. Similar modules can also be hosted externally - so long as they publish maven artifacts compatible with your version of streams, you can import and use them in your streams easily. -The streams community also supports a separate repository incubator-streams-examples (org.apache.streams:streams-examples) which contains a library of simple streams that are 'ready-to-run'. Look here to see what Streams user code look like. +The streams community also supports a seperate repository incubator-streams-examples (org.apache.streams:streams-examples) which contains a library of simple streams that are 'ready-to-run'. Look here to see what Streams user code look like. ### Why bother with any data framework at all? @@ -49,9 +49,9 @@ Or maybe by joining forces with others who have more than just a passing interes ### How is streams different than "*processing framework*"? -You don't have to look hard to find great data processing frameworks for batch or for real-time. Storm, Spark, Samza, Flink, and Google Cloud Dataflow (soon-to-be Apache Beam) are mature and well-documented. NiFi and Apex are interesting new options. At the core these platforms help you specify inputs, outputs, and a directed graph of computation and then run your code at scale. +You don't have to look hard to find great data processing frameworks for batch or for real-time. Storm, Spark, Samza, Flink, and Dataflow are well-known, well-documented, and solid. At the core these platforms help you specify inputs, outputs, and a directed graph of computation and then run your code at scale. -Streams supports a similar computational model, but is more focused on intelligently modeling the data that will flow through the stream than on stream execution. In this sense Streams is an alternative to avro or protocol buffers which prioritizes flexibility, expressivity, interoperability, and tooling ahead of speed or compute efficiency. +Streams supports a similar computational model, but is more focused on intelligently modeling the data that will flow through the stream. In this sense Streams is an alternative to avro or protocol buffers which prioritizes flexibility, expressivity, interoperability, and tooling ahead of speed or compute efficiency. Streams also seeks to make it easy to design and evolve streams, and to configure complex streams sensibly. Where many processing frameworks leave all business logic and configuration issues to the developer, streams modules are designed to mix-and-match. Streams modules expect to be embedded with other frameworks and are organized to make that process painless. @@ -72,21 +72,19 @@ A better long-term approach is to archive each data series you observe, and labe ### What if I need data from "*specific API*"? No problem - anyone can write a Streams provider. The project contains providers that use a variety of strategies to generate near-real-time data streams, including: - -* sockets -* webhooks -* polling -* scraping + - sockets + - webhooks + - polling + - scraping Providers can run continuously and pass-through new data, or they can work sequentially through a backlog of items. If you need to collect so many items that you can't fit all of their ids in the memory available to your stream, a stream provider can read an arbitrarily long sequence of ids and hand those off to other providers for collection. ### What if I want to keep data in "*unsupported database*"? No problem - anyone can write a Streams persist reader or persist writer. The project contains persist writers that: - -* write documents efficiently with batch-style binary indexing -* write documents one-by-one to services with REST api endpoints -* write data to local or distributed buffers. + - write documents efficiently with batch-style binary indexing + - write documents one-by-one to services with REST api endpoints + - write data to local or distributed buffers. If you just want to use streams providers to collect and feed incoming data into a queueing system to work with outside of streams that's just fine. http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/c5d54a9b/src/site/markdown/release-setup.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/release-setup.md b/src/site/markdown/release-setup.md index cb83c93..a4f5d8a 100644 --- a/src/site/markdown/release-setup.md +++ b/src/site/markdown/release-setup.md @@ -1,75 +1,78 @@ -Release Setup +###Release Setup + These setup steps only need to be performed on a particular machine once. Developers using Linux workstations can skip over the references to Cygwin. If using Windows, install cygwin, including Utils/gnupg and Net/openssh packages. -Create and install a SSH key +####Create and install a SSH key + +1. Open a shell window. If using Windows, open a cygwin window. +2. Use ssh-keygen to create an SSH key. -Open a shell window. If using Windows, open a cygwin window. -Use ssh-keygen to create an SSH key. -Follow the latest steps and guides on the ASF website as you should NOT be using SHA1 and new keys MUST be at least 4096 bits. + Follow the latest steps and guides on the [ASF website](http://www.apache.org/dev/openpgp.html#generate-key) as you should **NOT** be using SHA1 and new keys **MUST** be at least 4096 bits. - $ ssh-keygen -t rsa -b 4096 -Program defaults should be fine. No passphrase is required for the ssh key generation. The keys will be saved in ~/.ssh/id_dsa (private) and ~/.ssh/id_dsa.pub (public). +> $ ssh-keygen -t rsa -b 4096 -See Authenticating By Public Key (OpenSSH) for a good description on why and how to perform this task. + Program defaults should be fine. No passphrase is required for the ssh key generation. The keys will be saved in ~/.ssh/id_dsa (private) and ~/.ssh/id_dsa.pub (public). -SCP your SSH public key ~/.ssh/id_dsa.pub created in last step to ~/id_dsa.pub on people.apache.org. -$ cd ~/.ssh -$ scp id_dsa.pub @people.apache.org:id_dsa.pub -$ You will be prompted for your password. + See [Authenticating By Public Key (OpenSSH)](http://www.networknewz.com/networknewz-10-20030707AuthenticatingbyPublicKeyOpenSSH.html) for a good description on why and how to perform this task. -Use ssh to login to people.apache.org +3. SCP your SSH public key ~/.ssh/id_dsa.pub created in last step to ~/id_dsa.pub on people.apache.org. +> $ cd ~/.ssh +> $ scp id_dsa.pub <your userid>@people.apache.org:id_dsa.pub +> $ You will be prompted for your password. -$ cd ~ -$ ssh @people.apache.org +4. Use ssh to login to people.apache.org -At this point, you will still be prompted for your password. +> $ cd ~ +> $ ssh <your userid>@people.apache.org -Create a ~/.ssh folder in your home directory on people.apache.org and change its file mode to 700. + At this point, you will still be prompted for your password. -$ mkdir ~/.ssh -$ chmod 700 ~/.ssh -Move or append ~/id_dsa.pub to ~/.ssh/authorized_keys and change its file mode to 600. +5. Create a ~/.ssh folder in your home directory on people.apache.org and change its file mode to 700. -$ mv ~/id_dsa.pub ~/.ssh/authorized_keys $ chmod 600 ~/.ssh/authorized_keys +> $ mkdir ~/.ssh +> $ chmod 700 ~/.ssh -*Each public key in the authorized_keys spans only one line. -For example: "ssh-dss AAAAB3NzaC1kc3MAAA ..... agBmmfZ9uAbSqA== dsa-key-20071107" -'#' in the first column is a comment line.* -Exit out of this ssh session. +6. Move or append ~/id_dsa.pub to ~/.ssh/authorized_keys and change its file mode to 600. -Start a new ssh session. No login should be required this time due to the private ssh key on your local box matching up with the public ssh key in your home directory (~/.ssh). +> $ mv ~/id_dsa.pub ~/.ssh/authorized_keys +> $ chmod 600 ~/.ssh/authorized_keys -$ ssh @people.apache.org + * *Each public key in the authorized_keys spans only one line. + * For example: "ssh-dss AAAAB3NzaC1kc3MAAA ..... agBmmfZ9uAbSqA== dsa-key-20071107" + * '#' in the first column is a comment line.* -If you are still prompted for a password, then you have not set up the ssh keys properly. Review the steps above and ensure that all of the steps were followed properly. Or, maybe the instructions are still not quite right and they still need some adjusting. In that case, please update the instructions accordingly. +7. Exit out of this ssh session. +8. Start a new ssh session. No login should be required this time due to the private ssh key on your local box matching up with the public ssh key in your home directory (~/.ssh). -Create a GPG key +> $ ssh <userid>@people.apache.org -Open a shell window. If using Windows, open a cygwin window. -Generate a key-pair with gpg, using default key kind ("RSA and RSA") and keys size (4096). + *If you are still prompted for a password, then you have not set up the ssh keys properly. Review the steps above and ensure that all of the steps were followed properly. Or, maybe the instructions are still not quite right and they still need some adjusting. In that case, please update the instructions accordingly.* -$ gpg --gen-key +####Create a GPG key -The program's default values should be fine. For the "Real Name" enter your full name (ie. Stan Programmer). For the "e-mail address" enter your apache address (ie. [email protected]). You will also be required to enter a "passphrase" for the GPG key generation. Keep track of this as you will need this for the Release processing. +1. Open a shell window. If using Windows, open a cygwin window. +2. Generate a key-pair with gpg, using default key kind ("RSA and RSA") and keys size (4096). -The generated keys are stored in $HOME/.gnupg or %HOME%\Application Data\gnupg subdirectory. +> $ gpg --gen-key -Save the content in this subdirectory to a safe media. This contains your private key used to sign all the Streams release materials. + The program's default values should be fine. For the "Real Name" enter your full name (ie. Stan Programmer). For the "e-mail address" enter your apache address (ie. [email protected]). You will also be required to enter a "passphrase" for the GPG key generation. Keep track of this as you will need this for the Release processing. -Backup your home directory to another media || + * *The generated keys are stored in $HOME/.gnupg or %HOME%\Application Data\gnupg subdirectory.* + * *Save the content in this subdirectory to a safe media. This contains your private key used to sign all the Streams release materials.* -Add your public key to the SVN repository. See the commands describe at the beginning of this KEYS file to perform this task. The gpg key-pair is used to sign the published artifacts for the Streams releases. +3. Backup your home directory to another media || +4. Add your public key to the [SVN repository](https://svn.apache.org/repos/asf/incubator/streams/KEYS). See the commands describe at the beginning of this KEYS file to perform this task. The gpg key-pair is used to sign the published artifacts for the Streams releases. -$ gpg --list-sigs && gpg --armor -- export +> $ gpg --list-sigs <Real Name> && gpg --armor -- export <Real Name> -The KEYS file is updated via normal svn commit procedures. The one under w.a.o/dist/ has to be manually updated from svn. + *The [KEYS](https://svn.apache.org/repos/asf/incubator/streams/KEYS) file is updated via normal svn commit procedures. The one under w.a.o/dist/ has to be manually updated from svn.* -Submit your public key to a key server. E.g. SURFNET or MIT +5. Submit your public key to a key server. E.g. [SURFNET](http://pgp.surfnet.nl:11371/) or [MIT](http://pgp.mit.edu/) -Following the instructions in http://people.apache.org/~henkp/trust/ and ask multiple (at least 3) current Apache committers to sign your public key. +6. Following the instructions in http://people.apache.org/~henkp/trust/ and ask multiple (at least 3) current Apache committers to sign your public key. -Configure Maven +####Configure Maven -Update your ~/.m2/settings.xml with the properties from Publishing Maven Artifacts +1. Update your ~/.m2/settings.xml with the properties from [Publishing Maven Artifacts](http://www.apache.org/dev/publishing-maven-artifacts.html#dev-env) http://git-wip-us.apache.org/repos/asf/incubator-streams-master/blob/c5d54a9b/src/site/markdown/release.md ---------------------------------------------------------------------- diff --git a/src/site/markdown/release.md b/src/site/markdown/release.md index 675ea7c..9e691ee 100644 --- a/src/site/markdown/release.md +++ b/src/site/markdown/release.md @@ -1,197 +1,162 @@ -Release Process - - -Release Steps Overview -There are two distinct sets of artifacts that are released on independent schedules: streams-master & streams-project. The streams-master is the project metadata and only needs to be released when there is a change in the structure of the project itself. The streams-project artifacts are comprised of all streams source code, binaries and a standalone demo. For release setup information, refer to Release Setup Information. - -All of the steps below apply to both the master and project releases, unless otherwise specified. As an alternative to releasing separately, the projects MAY be released together as one combined release. The steps for this can be found below. (Combined Release Steps) - -Common Release Steps - -Environment setup for releasing artifacts (same for SNAPSHOTs and releases) - -Increase the default Java heap available to Maven (required for Java SE 6) - -export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m" - -Use the latest Sun 1.7.0 JDK - -Use Maven 3.2.1 or later - -Make sure the Release Setup steps have been performed. - -Prepare the source for release: - -Cleanup JIRA so the Fix Version in issues resolved since the last release includes this release version correctly. -Update the text files in a working copy of the project root - -Update the CHANGELOG based on the Text release reports from JIRA. -Review and update README.md if needed. -Commit any changes back to git -Stage any Roadmap or Release landing pages on the site. -Create a release candidate branch from master. X should start at 1 and increment if early release candidates fail to complete the release cycle. - -$ git checkout master $ git branch {$project.version}-rcX - -Verify the source has the required license headers before trying to release: - -$ mvn -P apache-release clean rat:check -e -DskipTests - -Do a dry run of the release:prepare step: - -$ mvn -P apache-release release:prepare -DautoVersionSubmodules=true -DdryRun=true -The dry run will not commit any changes back to SCM and gives you the opportunity to verify that the release process will complete as expected. You will be prompted for the following information : - -Release version - take the default (should be ${project.version}-incubating) -SCM release tag - DO NOT TAKE THE DEFAULT - ${project.version}-rcX -New development version - take the default (should be ${project.version}-incubating-SNAPSHOT) -GPG Passphrase -If you cancel a release:prepare before it updates the pom.xml versions, then use the release:clean goal to just remove the extra files that were created. - -The Maven release plugin checks for SNAPSHOT dependencies in pom's. It will not complete the prepare goal until all SNAPSHOT dependencies are resolved. - -Verify that the release process completed as expected - -The release plugin will create pom.xml.tag files which contain the changes that would have been committed to SVN. The only differences between pom.xml.tag and it's corresponding pom.xml file should be the version number. -Check release.properties and make sure that the scm properties have the right version. Sometimes the scm location can be the previous version not the next version. -Verify signatures (Verifying release signatures) -Cleanup the release prepare files again: - -$ mvn -P apache-release release:clean -Prepare the release - -Run the "release:prepare" step for real this time. You'll be prompted for the same version information. - -$ mvn -P apache-release -U clean release:prepare -DautoVersionSubmodules=true - -Backup (zip or tar) your local release candidate directory in case you need to rollback the release after the next step is performed. - -Perform the release - -This step will create a maven staging repository and site for use in testing and voting. - -$ mvn -Papache-release -Darguments='-Dmaven.test.skip.exec=true' release:perform -Dgoals=deploy -DlocalRepoDirectory=. -DlocalCheckout=true - -If your local OS userid doesn't match your Apache userid, then you'll have to also override the value provided by the OS to Maven for the site-deploy step to work. This is known to work for Linux, but not for Mac and unknown for Windows. - --Duser.name=[your_apache_uid] - -Verify the Nexus release artifacts - -Verify the staged artifacts in the nexus repo - -https://repository.apache.org/index.html -Staging repositories (under Build Promotion) --> Name column --> org.apache.streams -Navigate through the artifact tree and make sure that all javadoc, sources, tests, jars, ... have .asc (GPG signature) and .md5 files. See http://people.apache.org/~henkp/repo/faq.html and http://www.apache.org/dev/release-signing.html#openpgp-ascii-detach-sig -Close the nexus staging repo - -https://repository.apache.org/index.html -Staging repositories (under Build Promotion) --> Name column --> org.apache.streams -Click checkbox for the open staging repo (org.apache.streams-XXX) and press Close in the menu bar. -Put the release candidate up for a vote - -Create a VOTE email thread on dev@ to record votes as replies -Create a DISCUSS email thread on dev@ for any vote questions -Perform a review of the release and cast your vote. See the following for more details on Apache releases - -http://www.apache.org/dev/release.html - -A -1 vote does not necessarily mean that the vote must be redone, however it is usually a good idea to rollback the release if a -1 vote is received. See - Recovering from a vetoed release - -After the vote has been open for at least 72 hours, has at least three +1 PMC votes and no -1 votes, then post the results to the vote thread by - -reply to the initial email and prepend to the original subject "[RESULT]" -Include a list of everyone who voted +1, 0 or -1. -If there are fewer than 3 +1 (binding) votes from IPMC members, submit a vote to [email protected] requesting additional IPMC member votes. -Finalizing a release - -Promote the staged nexus artifacts - -https://repository.apache.org/index.html -Staging repositories (under Build Promotion) --> Name column --> org.apache.streams -Click checkbox of the closed staging repo (org.apache.streams-XXX) and select Release. -Copy the source artifacts over to the distribution area - -$ svn co https://dist.apache.org/repos/dist/release/incubator/streams/releases ./streams-releases (KEEP this directory until after the release process has been completed) -$ cd ./streams-releases -$ mkdir ${project.version} -$ cd ./${project.version} -$ wget https://repository.apache.org/content/repositories/releases/org/apache/streams/${project.name}/${project.version}/${project.name}-${project.version}-source-release.zip -$ wget https://repository.apache.org/content/repositories/releases/org/apache/streams/${project.name}/${project.version}/${project.name}-${project.version}-source-release.zip.asc -$ wget https://repository.apache.org/content/repositories/releases/org/apache/streams/${project.name}/${project.version}/${project.name}-${project.version}-source-release.zip.md5 -$ svn add ${project.name}-* -$ svn commit -m "Committing Source Release for ${project.name}-${project.version} -Create an official release tag from the successful release candidate tag. - -$ git checkout streams-project-${project.version}-rcX -$ git tag -a streams-project-${project.version} -m 'release tag streams-project-${project.version}' -$ git push origin :refs/tags/streams-project-${project.version} -Update the staged website - -Update the downloads page to add new version using the mirrored URLs -Modify the URL for the prior release to the archived URL for the release -Publish the website - -WAIT 24hrs after committing releases for mirrors to replicate -Publish updates to the download page -Delete the prior versions - -Navigate to the release directories checked out in the prior steps -Delete the prior release artifacts using the svn delete command -Commit the deletion -Update the JIRA versions page to close all issues, mark the version as "released", and set the date to the date that the release was approved. You may also need to make a new release entry for the next release. - -Announcing the release - -Make a news announcement on the streams homepage. -Make an announcement about the release on the [email protected], [email protected], and [email protected] list as per the Apache Announcement Mailing Lists page) -Recovering from a vetoed release - -Reply to the initial vote email and prepend to the original subject - - -[CANCELED] - -Clean the release prepare files and hard reset the release candidate branch. - -mvn -P apache-release release:clean - -Delete the git tag created by the release:perform step - - -$ git tag -d streams-project-${project.version}-rcX $ git push origin :refs/tags/streams-project-${project.version}-rcX - -Delete the build artifacts on people & www - -$ rm -rfv /www/people.apache.org/builds/streams/${project.version} -$ rm -rfv /www/www.apache.org/dist/streams/${project.version} -Drop the nexus staging repo - -https://repository.apache.org/index.html -Enterprise --> Staging -Staging tab --> Name column --> org.apache.streams -Right click on the closed staging repo (org.apache.streams-XXX) and select Drop. -Remove the staged site - -Make the required updates that caused the vote to be canceled during the next release cycle - - -Verifying release signatures - +###Release Process + +There are two distinct sets of artifacts that are released on independent schedules: streams-master & streams-project. The streams-master is the project metadata and only needs to be released when there is a change in the structure of the project itself. The streams-project artifacts are comprised of all streams source code, binaries and a standalone demo. For release setup information, refer to [Release Setup Information](/release-setup.html). + +All of the steps below apply to both the master and project releases, unless otherwise specified. As an alternative to releasing separately, the projects MAY be released together as one combined release. The steps for this can be found below. ([Combined Release Steps](#combined)) + +####Common Release Steps + +1. Environment setup for releasing artifacts (same for SNAPSHOTs and releases) + + 1. Increase the default Java heap available to Maven (required for Java SE 6) + > export MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=256m" + 2. Use the latest Sun 1.7.0 JDK + 3. Use Maven 3.2.1 or later + 4. Make sure the [Release Setup](release-setup.html) steps have been performed. + +2. Prepare the source for release: + + 1. Cleanup JIRA so the Fix Version in issues resolved since the last release includes this release version correctly. + 2. Update the text files in a working copy of the project root - + 1. Update the CHANGELOG based on the Text release reports from JIRA. + 2. Review and update README.md if needed. + 3. Commit any changes back to git + 3. Stage any Roadmap or Release landing pages on the site. + +3. Create a release candidate branch from master. + X should start at 1 and increment if early release candidates fail to complete the release cycle. +> $ git checkout master +> $ git branch {$project.version}-rcX +4. Verify the source has the required license headers before trying to release: +> $ mvn -P apache-release clean rat:check -e -DskipTests +5. Do a dry run of the release:prepare step: +> $ mvn -P apache-release release:prepare -DautoVersionSubmodules=true -DdryRun=true + + The dry run will not commit any changes back to SCM and gives you the opportunity to verify that the release process will complete as expected. You will be prompted for the following information : + + * Release version - take the default (should be ${project.version}-incubating) + * SCM release tag - *DO NOT TAKE THE DEFAULT* - ${project.version}-rcX + * New development version - take the default (should be ${project.version}-incubating-SNAPSHOT) + * GPG Passphrase + + *If you cancel a release:prepare before it updates the pom.xml versions, then use the release:clean goal to just remove the extra files that were created.* + + The Maven release plugin checks for SNAPSHOT dependencies in pom's. It will not complete the prepare goal until all SNAPSHOT dependencies are resolved. + +6. Verify that the release process completed as expected + 1. The release plugin will create pom.xml.tag files which contain the changes that would have been committed to SVN. The only differences between pom.xml.tag and it's corresponding pom.xml file should be the version number. + 2. Check release.properties and make sure that the scm properties have the right version. Sometimes the scm location can be the previous version not the next version. + 3. Verify signatures ([Verifying release signatures](#verify_signatures)) + +7. Cleanup the release prepare files again: +> $ mvn -P apache-release release:clean +8. Prepare the release + 1. Run the "release:prepare" step for real this time. You'll be prompted for the same version information. + > $ mvn -P apache-release -U clean release:prepare -DautoVersionSubmodules=true + 2. Backup (zip or tar) your local release candidate directory in case you need to rollback the release after the next step is performed. +9. Perform the release + * This step will create a maven staging repository and site for use in testing and voting. + > $ mvn -Papache-release -Darguments='-Dmaven.test.skip.exec=true' release:perform -Dgoals=deploy -DlocalRepoDirectory=. -DlocalCheckout=true + * If your local OS userid doesn't match your Apache userid, then you'll have to also override the value provided by the OS to Maven for the site-deploy step to work. This is known to work for Linux, but not for Mac and unknown for Windows.* + > -Duser.name=[your_apache_uid] +10. Verify the Nexus release artifacts + 1. Verify the staged artifacts in the nexus repo + * https://repository.apache.org/index.html + * Staging repositories (under Build Promotion) --> Name column --> org.apache.streams + * Navigate through the artifact tree and make sure that all javadoc, sources, tests, jars, ... have .asc (GPG signature) and .md5 files. See http://people.apache.org/~henkp/repo/faq.html and http://www.apache.org/dev/release-signing.html#openpgp-ascii-detach-sig + 2. Close the nexus staging repo + * https://repository.apache.org/index.html + * Staging repositories (under Build Promotion) --> Name column --> org.apache.streams + * Click checkbox for the open staging repo (org.apache.streams-XXX) and press Close in the menu bar. +11. Put the release candidate up for a vote + 1. Create a VOTE email thread on dev@ to record votes as replies + 2. Create a DISCUSS email thread on dev@ for any vote questions + 3. Perform a review of the release and cast your vote. See the following for more details on Apache releases + [http://www.apache.org/dev/release.html](http://www.apache.org/dev/release.html) + 4. A -1 vote does not necessarily mean that the vote must be redone, however it is usually a good idea to rollback the release if a -1 vote is received. See - Recovering from a vetoed release + 5. After the vote has been open for at least 72 hours, has at least three +1 PMC votes and no -1 votes, then post the results to the vote thread by - + * reply to the initial email and prepend to the original subject "[RESULT]" + * Include a list of everyone who voted +1, 0 or -1. + 6. If there are fewer than 3 +1 (binding) votes from IPMC members, submit a vote to [email protected] requesting additional IPMC member votes. +12. Finalizing a release + 1. Promote the staged nexus artifacts + * https://repository.apache.org/index.html + * Staging repositories (under Build Promotion) --> Name column --> org.apache.streams + * Click checkbox of the closed staging repo (org.apache.streams-XXX) and select Release. + 2. Copy the source artifacts over to the distribution area +> $ svn co https://dist.apache.org/repos/dist/release/incubator/streams/releases ./streams-releases (KEEP this directory until after the release process has been completed) +> $ cd ./streams-releases +> $ mkdir ${project.version} +> $ cd ./${project.version} +> $ wget https://repository.apache.org/content/repositories/releases/org/apache/streams/${project.name}/${project.version}/${project.name}-${project.version}-source-release.zip +> $ wget https://repository.apache.org/content/repositories/releases/org/apache/streams/${project.name}/${project.version}/${project.name}-${project.version}-source-release.zip.asc +> $ wget https://repository.apache.org/content/repositories/releases/org/apache/streams/${project.name}/${project.version}/${project.name}-${project.version}-source-release.zip.md5 +> $ svn add ${project.name}-* +> $ svn commit -m "Committing Source Release for ${project.name}-${project.version} + 3. Create an official release tag from the successful release candidate tag. +> $ git checkout streams-project-${project.version}-rcX +> $ git tag -a streams-project-${project.version} -m 'release tag streams-project-${project.version}' +> $ git push origin :refs/tags/streams-project-${project.version} + 4. Update the staged website + * Update the downloads page to add new version using the mirrored URLs + * Modify the URL for the prior release to the archived URL for the release + 5. Publish the website + * WAIT 24hrs after committing releases for mirrors to replicate + * Publish updates to the download page + 6. Delete the prior versions + * Navigate to the release directories checked out in the prior steps + * Delete the prior release artifacts using the svn delete command + * Commit the deletion +14. Update the JIRA versions page to close all issues, mark the version as "released", and set the date to the date that the release was approved. You may also need to make a new release entry for the next release. +15. Announcing the release + * Make a news announcement on the streams homepage. + * Make an announcement about the release on the [email protected], [email protected], and [email protected] list as per the Apache Announcement Mailing Lists page) + +####Recovering from a vetoed release + +1. Reply to the initial vote email and prepend to the original subject - + [CANCELED] +2. Clean the release prepare files and hard reset the release candidate branch. +> mvn -P apache-release release:clean + +3. Delete the git tag created by the release:perform step - +> $ git tag -d streams-project-${project.version}-rcX +> $ git push origin :refs/tags/streams-project-${project.version}-rcX + +4. Delete the build artifacts on people & www +> $ rm -rfv /www/people.apache.org/builds/streams/${project.version} +> $ rm -rfv /www/www.apache.org/dist/streams/${project.version} + +5. Drop the nexus staging repo + 1. https://repository.apache.org/index.html + 2. Enterprise --> Staging + 3. Staging tab --> Name column --> org.apache.streams + 4. Right click on the closed staging repo (org.apache.streams-XXX) and select Drop. +5. Remove the staged site +6. Make the required updates that caused the vote to be canceled during the next release cycle + +<a name="verify_signatures" ></a> +####Verifying release signatures On unix platforms the following command can be executed - - for file in `find . -type f -iname '*.asc'` - do - gpg --verify ${file} - done -You'll need to look at the output to ensure it contains only good signatures - +> for file in `find . -type f -iname '*.asc'` +> do +> gpg --verify ${file} +> done -gpg: Good signature from ... gpg: Signature made ... +You'll need to look at the output to ensure it contains only good signatures - +gpg: Good signature from ... +gpg: Signature made ... -Combined Release -In order to perform a combined release of the streams-master and streams-project trunks, do the following: +<a name="combined" ></a> +####Combined Release +In order to perform a combined release of the streams-master and streams-project trunks, do the following: -Perform Steps 1-9 of the release for streams Master & streams Project -Do NOT perform step 10 until steps 1-9 have been completed for BOTH projects -Build the streams-master FIRST -When prompted to change the streams-project dependency on streams-master SNAPSHOT, do so to the release that you just built -Execute the remaining steps using the following e-mail templates -PMC Release Vote + * Perform Steps 1-9 of the [release](#release-steps) for streams Master & streams Project + * Do NOT perform step 10 until steps 1-9 have been completed for BOTH projects + * Build the streams-master FIRST + * When prompted to change the streams-project dependency on streams-master SNAPSHOT, do so to the release that you just built + * Execute the remaining steps using the following e-mail templates + * [PMC Release Vote](PPMC_Combined.txt)
