SAMZA-831: Update Samza 0.10 online documentation
Project: http://git-wip-us.apache.org/repos/asf/samza/repo Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/0fb05861 Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/0fb05861 Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/0fb05861 Branch: refs/heads/samza-sql Commit: 0fb05861017900a06acd38ad55ceef9dce6864d7 Parents: adf4f39 Author: Yi Pan (Data Infrastructure) <[email protected]> Authored: Fri Dec 18 12:26:59 2015 -0800 Committer: Yi Pan (Data Infrastructure) <[email protected]> Committed: Fri Dec 18 16:52:25 2015 -0800 ---------------------------------------------------------------------- docs/_config.yml | 2 +- docs/_docs/replace-versioned.sh | 4 +- docs/_layouts/default.html | 1 + docs/archive/index.html | 8 ++++ docs/community/committers.md | 2 +- docs/contribute/tests.md | 2 +- .../versioned/deploy-samza-job-from-hdfs.md | 2 +- .../tutorials/versioned/deploy-samza-to-CDH.md | 4 +- docs/learn/tutorials/versioned/index.md | 2 - .../versioned/remote-debugging-samza.md | 2 +- .../versioned/run-in-multi-node-yarn.md | 8 ++-- .../versioned/upgrading-from-0.7.0-to-0.8.0.md | 45 -------------------- docs/startup/download/index.md | 19 +++++---- docs/startup/hello-samza/versioned/index.md | 2 +- gradle.properties | 2 +- 15 files changed, 34 insertions(+), 71 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/_config.yml ---------------------------------------------------------------------- diff --git a/docs/_config.yml b/docs/_config.yml index a3bb8d4..dc1a66f 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -25,4 +25,4 @@ exclude: [_docs] baseurl: http://samza.apache.org version: latest # this is the version you will go if you click 'switch version' in "latest" pages. -latest-release: 0.9 +latest-release: '0.10' http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/_docs/replace-versioned.sh ---------------------------------------------------------------------- diff --git a/docs/_docs/replace-versioned.sh b/docs/_docs/replace-versioned.sh index d9fb383..f148781 100755 --- a/docs/_docs/replace-versioned.sh +++ b/docs/_docs/replace-versioned.sh @@ -17,8 +17,8 @@ # under the License. # get the version number -version=`cat _config.yml | grep 'version:' | cut -d' ' -f 2` -latestRelease=`cat _config.yml | grep 'latest-release:' | cut -d' ' -f 2` +version=`cat _config.yml | grep 'version:' | cut -d' ' -f 2 | sed -e "s/'//g"` +latestRelease=`cat _config.yml | grep 'latest-release:' | cut -d' ' -f 2 | sed -e "s/'//g"` DIR=$(dirname $0)/.. if test -z "$version" || test -z "$latestRelease"; then http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/_layouts/default.html ---------------------------------------------------------------------- diff --git a/docs/_layouts/default.html b/docs/_layouts/default.html index 839aadf..7beb734 100644 --- a/docs/_layouts/default.html +++ b/docs/_layouts/default.html @@ -103,6 +103,7 @@ <h1><i class="fa fa-history"></i> Archive</h1> <ul> <li><a href="/archive/index.html#latest">latest</a></li> + <li><a href="/archive/index.html#10">0.10</a></li> <li><a href="/archive/index.html#09">0.9</a></li> <li><a href="/archive/index.html#08">0.8</a></li> <li><a href="/archive/index.html#07">0.7</a></li> http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/archive/index.html ---------------------------------------------------------------------- diff --git a/docs/archive/index.html b/docs/archive/index.html index 1bfeb9b..b0a44c6 100644 --- a/docs/archive/index.html +++ b/docs/archive/index.html @@ -27,6 +27,14 @@ title: Documentation <li><a href="../startup/hello-samza/latest">Hello Samza</a></li> </ul> +<h4 id="10">0.10 Release</h4> + +<ul class="documentation-list"> + <li><a href="../learn/documentation/0.10">Documentation</a></li> + <li><a href="../learn/tutorials/0.10">Tutorials</a></li> + <li><a href="../startup/hello-samza/0.10">Hello Samza</a></li> +</ul> + <h4 id="09">0.9 Release</h4> <ul class="documentation-list"> http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/community/committers.md ---------------------------------------------------------------------- diff --git a/docs/community/committers.md b/docs/community/committers.md index 4c84263..ec2f10c 100644 --- a/docs/community/committers.md +++ b/docs/community/committers.md @@ -70,6 +70,6 @@ Committer, and PMC member<br/> <a href="https://twitter.com/nickpan47"; target="_blank"><i class="fa fa-twitter committer-icon"></i></a> **Navina Ramesh**<br/> -Committer<br/> +Committer, and PMC member<br/> <a href="https://www.linkedin.com/in/rnavina"; target="_blank"><i class="fa fa-linkedin committer-icon"></i></a> <a href="https://twitter.com/navina_r"; target="_blank"><i class="fa fa-twitter committer-icon"></i></a> http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/contribute/tests.md ---------------------------------------------------------------------- diff --git a/docs/contribute/tests.md b/docs/contribute/tests.md index c4f9924..564dc0d 100644 --- a/docs/contribute/tests.md +++ b/docs/contribute/tests.md @@ -19,7 +19,7 @@ title: Tests limitations under the License. --> -Samza's unit tests are written on top of [JUnit](http://junit.org/), and license checking is done with [Apache Rat](http://creadur.apache.org/rat/). Samza's integration tests are written on top of [Zopkio](https://github.com/linkedin/Zopkio). +Samza's unit tests are written on top of [JUnit](http://junit.org/), and license checking is done with [Apache Rat](http://creadur.apache.org/rat/). Samza's integration tests are written on top of [Zopkio](https://github.com/linkedin/Zopkio). As of Samza 0.10, the integration tests are based on Zopkio 0.1.17. ### Running Unit Tests Locally http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md b/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md index c2d934d..ca7b5f1 100644 --- a/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md +++ b/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md @@ -24,7 +24,7 @@ This tutorial uses [hello-samza](../../../startup/hello-samza/{{site.version}}/) ### Upload the package {% highlight bash %} -hadoop fs -put ./target/hello-samza-0.9.1-dist.tar.gz /path/for/tgz +hadoop fs -put ./target/hello-samza-0.10.0-dist.tar.gz /path/for/tgz {% endhighlight %} ### Add HDFS configuration http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md b/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md index b5e0397..36df300 100644 --- a/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md +++ b/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md @@ -26,7 +26,7 @@ The tutorial assumes you have successfully run [hello-samza](../../../startup/he There are a few ways of uploading the package to the cluster's HDFS. If you do not have the job package in your cluster, **scp** from you local machine to the cluster. Then run {% highlight bash %} -hadoop fs -put path/to/hello-samza-0.9.1-dist.tar.gz /path/for/tgz +hadoop fs -put path/to/hello-samza-0.10.0-dist.tar.gz /path/for/tgz {% endhighlight %} ### Get Deloying Scripts @@ -34,7 +34,7 @@ hadoop fs -put path/to/hello-samza-0.9.1-dist.tar.gz /path/for/tgz Untar the job package (assume you will run from the current directory) {% highlight bash %} -tar -xvf path/to/samza-job-package-0.9.1-dist.tar.gz -C ./ +tar -xvf path/to/samza-job-package-0.10.0-dist.tar.gz -C ./ {% endhighlight %} ### Add Package Path to Properties File http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/learn/tutorials/versioned/index.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/index.md b/docs/learn/tutorials/versioned/index.md index 8b2b692..b4d687a 100644 --- a/docs/learn/tutorials/versioned/index.md +++ b/docs/learn/tutorials/versioned/index.md @@ -29,8 +29,6 @@ title: Tutorials [Run Hello-samza without Internet](run-hello-samza-without-internet.html) -[Upgrading from 0.7.0 to 0.8.0](upgrading-from-0.7.0-to-0.8.0.html) - <!-- TODO a bunch of tutorials [Log Walkthrough](log-walkthrough.html) <a href="configuring-kafka-system.html">Configuring a Kafka System</a><br/> http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/learn/tutorials/versioned/remote-debugging-samza.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/remote-debugging-samza.md b/docs/learn/tutorials/versioned/remote-debugging-samza.md index efef045..ad1d04d 100644 --- a/docs/learn/tutorials/versioned/remote-debugging-samza.md +++ b/docs/learn/tutorials/versioned/remote-debugging-samza.md @@ -83,7 +83,7 @@ Once the grid starts, you can start the wikipedia-feed Samza job. {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.9.1-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-0.10.0-dist.tar.gz -C deploy/samza deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties {% endhighlight %} http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md b/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md index 50ac211..8657603 100644 --- a/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md +++ b/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md @@ -84,8 +84,8 @@ tar -xvf scala-2.10.4.tgz cp /tmp/scala-2.10.4/lib/scala-compiler.jar $HADOOP_YARN_HOME/share/hadoop/hdfs/lib cp /tmp/scala-2.10.4/lib/scala-library.jar $HADOOP_YARN_HOME/share/hadoop/hdfs/lib curl -L http://search.maven.org/remotecontent?filepath=org/clapper/grizzled-slf4j_2.10/1.0.1/grizzled-slf4j_2.10-1.0.1.jar > $HADOOP_YARN_HOME/share/hadoop/hdfs/lib/grizzled-slf4j_2.10-1.0.1.jar -curl -L http://search.maven.org/remotecontent?filepath=org/apache/samza/samza-yarn_2.10/0.9.1/samza-yarn_2.10-0.9.1.jar > $HADOOP_YARN_HOME/share/hadoop/hdfs/lib/samza-yarn_2.10-0.9.1.jar -curl -L http://search.maven.org/remotecontent?filepath=org/apache/samza/samza-core_2.10/0.9.1/samza-core_2.10-0.9.1.jar > $HADOOP_YARN_HOME/share/hadoop/hdfs/lib/samza-core_2.10-0.9.1.jar +curl -L http://search.maven.org/remotecontent?filepath=org/apache/samza/samza-yarn_2.10/0.10.0/samza-yarn_2.10-0.10.0.jar > $HADOOP_YARN_HOME/share/hadoop/hdfs/lib/samza-yarn_2.10-0.10.0.jar +curl -L http://search.maven.org/remotecontent?filepath=org/apache/samza/samza-core_2.10/0.10.0/samza-core_2.10-0.10.0.jar > $HADOOP_YARN_HOME/share/hadoop/hdfs/lib/samza-core_2.10-0.10.0.jar {% endhighlight %} 6\. Add http configuration in core-site.xml (create the core-site.xml file and add content). @@ -145,7 +145,7 @@ vi src/main/config/wikipedia-feed.properties Change the yarn.package.path property to be: {% highlight jproperties %} -yarn.package.path=http://yourHostname:8000/target/hello-samza-0.9.1-dist.tar.gz +yarn.package.path=http://yourHostname:8000/target/hello-samza-0.10.0-dist.tar.gz {% endhighlight %} 3\. Complie hello-samza. @@ -153,7 +153,7 @@ yarn.package.path=http://yourHostname:8000/target/hello-samza-0.9.1-dist.tar.gz {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.9.1-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-0.10.0-dist.tar.gz -C deploy/samza {% endhighlight %} 4\. Deploy Samza job package to Http server.. http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/learn/tutorials/versioned/upgrading-from-0.7.0-to-0.8.0.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/upgrading-from-0.7.0-to-0.8.0.md b/docs/learn/tutorials/versioned/upgrading-from-0.7.0-to-0.8.0.md deleted file mode 100644 index 340762e..0000000 --- a/docs/learn/tutorials/versioned/upgrading-from-0.7.0-to-0.8.0.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -layout: page -title: Upgrading from 0.7.0 to 0.8.0 ---- -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> - -Samza's [checkpointing](../../documentation/{{site.version}}/container/checkpointing.html) implementation changed between Samza 0.7.0 and 0.8.0. If you are running a Samza job with 0.7.0, and upgrade to 0.8.0, your job's checkpoint offsets will be lost, and the job will start (by default) with the most recent message in its input streams. If this is undesirable, and a job needs to pick up where it left off, the following steps must be taken: - -1. Shutdown your job. -2. Run the [CheckpointMigrationTool](https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob;f=samza-kafka/src/main/scala/org/apache/samza/util/CheckpointMigrationTool.scala;h=5c4b3c4c544ae4367377b1a84d9a85a3de671018;hb=0.8.0). -3. Start your job. - -The CheckpointMigrationTool is responsible for migrating your checkpoint topic from the 0.7.0 style format to the 0.8.0 style format. This tool works only against Kafka, so you must be storing your checkpoints in Kafka with the [KafkaCheckpointManager](https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob;f=samza-kafka/src/main/scala/org/apache/samza/checkpoint/kafka/KafkaCheckpointManager.scala;h=1d5627d0c561a0be6b48ee307b755958e62b783e;hb=0.8.0). - -### Running CheckpointMigrationTool - -Checkout Samza 0.8.0: - - git clone http://git-wip-us.apache.org/repos/asf/samza.git - cd samza - git fetch origin 0.8.0 - git checkout 0.8.0 - -Run the checkpoint migration task: - - ./gradlew samza-shell:checkpointMigrationTool -PconfigPath=file:///path/to/job/config.properties - -The configPath property should be pointed at the .properties file for the job you wish to migrate. The tool will use the job's properties file to connect to the Kafka cluster, and migrate the checkpointed offsets to the 0.8.0 format. Once the tool is complete, the job should be restarted so that it can pick up the migrated offsets. - -_NOTE: The checkpointMigrationTool task must be run from a machine that can connect to the Kafka cluster._ http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/startup/download/index.md ---------------------------------------------------------------------- diff --git a/docs/startup/download/index.md b/docs/startup/download/index.md index ee05714..138569e 100644 --- a/docs/startup/download/index.md +++ b/docs/startup/download/index.md @@ -29,6 +29,7 @@ Starting in June of 2015, Samza will begin requiring JDK7 or higher. Please see ### Source Releases + * [samza-sources-0.10.0.tgz](http://www.apache.org/dyn/closer.lua/samza/0.10.0) * [samza-sources-0.9.1.tgz](http://www.apache.org/dyn/closer.lua/samza/0.9.1) * [samza-sources-0.9.0.tgz](http://www.apache.org/dyn/closer.lua/samza/0.9.0) * [samza-sources-0.8.0-incubating.tgz](https://archive.apache.org/dist/incubator/samza/0.8.0-incubating) @@ -46,12 +47,12 @@ A Maven-based Samza project can pull in all required dependencies Samza dependen <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-api</artifactId> - <version>0.9.1</version> + <version>0.10.0</version> </dependency> <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-core_2.10</artifactId> - <version>0.9.1</version> + <version>0.10.0</version> <scope>runtime</scope> </dependency> <dependency> @@ -59,37 +60,37 @@ A Maven-based Samza project can pull in all required dependencies Samza dependen <artifactId>samza-shell</artifactId> <classifier>dist</classifier> <type>tgz</type> - <version>0.9.1</version> + <version>0.10.0</version> <scope>runtime</scope> </dependency> <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-yarn_2.10</artifactId> - <version>0.9.1</version> + <version>0.10.0</version> <scope>runtime</scope> </dependency> <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-kv_2.10</artifactId> - <version>0.9.1</version> + <version>0.10.0</version> <scope>runtime</scope> </dependency> <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-kv-rocksdb_2.10</artifactId> - <version>0.9.1</version> + <version>0.10.0</version> <scope>runtime</scope> </dependency> <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-kv-inmemory_2.10</artifactId> - <version>0.9.1</version> + <version>0.10.0</version> <scope>runtime</scope> </dependency> <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-kafka_2.10</artifactId> - <version>0.9.1</version> + <version>0.10.0</version> <scope>runtime</scope> </dependency> {% endhighlight %} @@ -100,7 +101,7 @@ Samza versions less than 0.9 should include this additional dependency. <dependency> <groupId>org.apache.samza</groupId> <artifactId>samza-serializers_2.10</artifactId> - <version>0.9.1</version> + <version>0.8.1</version> </dependency> {% endhighlight %} http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/docs/startup/hello-samza/versioned/index.md ---------------------------------------------------------------------- diff --git a/docs/startup/hello-samza/versioned/index.md b/docs/startup/hello-samza/versioned/index.md index 8fb4a94..e1ef4da 100644 --- a/docs/startup/hello-samza/versioned/index.md +++ b/docs/startup/hello-samza/versioned/index.md @@ -61,7 +61,7 @@ Then, you can continue w/ the following command in hello-samza project: {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.10.0-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-0.10.1-SNAPSHOT-dist.tar.gz -C deploy/samza {% endhighlight %} ### Run a Samza Job http://git-wip-us.apache.org/repos/asf/samza/blob/0fb05861/gradle.properties ---------------------------------------------------------------------- diff --git a/gradle.properties b/gradle.properties index b18c0cb..16e1f5d 100644 --- a/gradle.properties +++ b/gradle.properties @@ -15,7 +15,7 @@ # specific language governing permissions and limitations # under the License. group=org.apache.samza -version=0.10.0-SNAPSHOT +version=0.10.1-SNAPSHOT scalaVersion=2.10 gradleVersion=2.0
