Repository: samza Updated Branches: refs/heads/master 0176138a1 -> 00cd44ac9
Update hardcoded versions for the 1.0.0 release. Author: Prateek Maheshwari <[email protected]> Closes #754 from prateekm/version-updates Project: http://git-wip-us.apache.org/repos/asf/samza/repo Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/00cd44ac Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/00cd44ac Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/00cd44ac Branch: refs/heads/master Commit: 00cd44ac9101bd89933b05964ef5168bce4ff334 Parents: 0176138 Author: Prateek Maheshwari <[email protected]> Authored: Mon Oct 22 17:55:06 2018 -0700 Committer: Prateek Maheshwari <[email protected]> Committed: Mon Oct 22 17:55:06 2018 -0700 ---------------------------------------------------------------------- RELEASE.md | 16 +- .../versioned/deployment/standalone.md | 2 +- .../versioned/standalone/standalone.md | 217 ------------------- .../versioned/deploy-samza-job-from-hdfs.md | 2 +- .../tutorials/versioned/deploy-samza-to-CDH.md | 4 +- .../versioned/hello-samza-high-level-yarn.md | 2 +- .../versioned/hello-samza-high-level-zk.md | 2 +- .../versioned/remote-debugging-samza.md | 2 +- .../versioned/run-in-multi-node-yarn.md | 4 +- .../versioned/samza-rest-getting-started.md | 2 +- docs/startup/hello-samza/versioned/index.md | 2 +- gradle.properties | 2 +- samza-test/src/main/config/join/README | 8 +- samza-test/src/main/python/configs/tests.json | 2 +- samza-test/src/main/python/stream_processor.py | 2 +- 15 files changed, 32 insertions(+), 237 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/RELEASE.md ---------------------------------------------------------------------- diff --git a/RELEASE.md b/RELEASE.md index 31b1e29..f73a186 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -28,6 +28,7 @@ And before you proceed, do the following steps: version=$VERSION * change the samza_executable variable in samza-test/src/main/python/configs/tests.json to $VERSION w/o the suffix '-SNAPSHOT'. * change the samza-test versions in samza-test/src/main/config/join/README to $VERSION w/o the suffix '-SNAPSHOT'. + * change the executable version in samza-test/src/main/python/stream_processor.py to $VERSION w/o the suffix '-SNAPSHOT'. * push the changes to the $VERSION branch Validate Samza using all our supported build matrix. @@ -55,7 +56,10 @@ Then build the tarball: Then sign it: - gpg --sign --armor --detach-sig build/distribution/source/apache-samza-*.tgz + gpg --sign --armor --detach-sig ./build/distribution/source/apache-samza-*.tgz + +Create MD5 signature: + gpg --print-md MD5 ./build/distribution/source/apache-samza-*.tgz > ./build/distribution/source/apache-samza-*.tgz.md5 Create SHA1 signature: @@ -85,7 +89,7 @@ Create SHA1 signature: gpg --print-md SHA1 ./build/distribution/source/apache-samza-*.tgz > ./build/distribution/source/apache-samza-*.tgz.sha1 -Edit `$HOME/.gradle/gradle.properties` and add your GPG key information: +Edit `$HOME/.gradle/gradle.properties` and add your GPG key information (without the comments): signing.keyId=01234567 # Your GPG key ID, as 8 hex digits signing.secretKeyRingFile=/path/to/secring.gpg # Normally in $HOME/.gnupg/secring.gpg @@ -112,6 +116,14 @@ repository just created, and close it. This may take a minute or so. When it finishes, the UI shows a staging repository URL. This can be used in a project that depends on Samza, to test the release candidate. +Upload the build artifacts to your Apache home directory: + sftp <username>@home.apache.org + mkdir samza-$VERSION-rc0 + cd samza-$VERSION-rc0 + put ./build/distribution/source/apache-samza-* . + put ./samza-tools/build/distributions/samza-tools-* . + bye + If the VOTE has successfully passed on the release candidate, you can log in to the [repository web interface](https://repository.apache.org) (same as above) and "release" the org.apache.samza repository listed under "Staging Repositories". http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/documentation/versioned/deployment/standalone.md ---------------------------------------------------------------------- diff --git a/docs/learn/documentation/versioned/deployment/standalone.md b/docs/learn/documentation/versioned/deployment/standalone.md index c7425f6..146f9cb 100644 --- a/docs/learn/documentation/versioned/deployment/standalone.md +++ b/docs/learn/documentation/versioned/deployment/standalone.md @@ -163,7 +163,7 @@ Before you can run the standalone job, you need to build a package for it using ```bash mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.15.0-SNAPSHOT-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-1.0.0-dist.tar.gz -C deploy/samza ``` ### Deploy binaries http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/documentation/versioned/standalone/standalone.md ---------------------------------------------------------------------- diff --git a/docs/learn/documentation/versioned/standalone/standalone.md b/docs/learn/documentation/versioned/standalone/standalone.md deleted file mode 100644 index c7425f6..0000000 --- a/docs/learn/documentation/versioned/standalone/standalone.md +++ /dev/null @@ -1,217 +0,0 @@ ---- -layout: page -title: Run as embedded library. ---- -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> - -- [Introduction](#introduction) -- [User guide](#user-guide) - - [Setup dependencies](#setup-dependencies) - - [Configuration](#configuration) - - [Code sample](#code-sample) -- [Quick start guide](#quick-start-guide) - - [Setup zookeeper](#setup-zookeeper) - - [Setup kafka](#setup-kafka) - - [Build binaries](#build-binaries) - - [Deploy binaries](#deploy-binaries) - - [Inspect results](#inspect-results) -- [Coordinator internals](#coordinator-internals) - -# - -# Introduction - -With Samza 0.13.0, the deployment model of samza jobs has been simplified and decoupled from YARN. _Standalone_ model provides the stream processing capabilities of samza packaged in the form of a library with pluggable coordination. This library model offers an easier integration path and promotes a flexible deployment model for an application. Using the standalone mode, you can leverage Samza processors directly in your application and deploy Samza applications to self-managed clusters. - -A standalone application typically is comprised of multiple _stream processors_. A _stream processor_ encapsulates a user defined processing function and is responsible for processing a subset of input topic partitions. A stream processor of a standalone application is uniquely identified by a _processorId_. - -Samza provides pluggable job _coordinator_ layer to perform leader election and assign work to the stream processors. Standalone supports Zookeeper coordination out of the box and uses it for distributed coordination between the stream processors of standalone application. A processor can become part of a standalone application by setting its app.name(Ex: app.name=group\_1) and joining the group. - -In samza standalone, the input topic partitions are distributed between the available processors dynamically at runtime. In each standalone application, one stream processor will be chosen as a leader initially to mediate the assignment of input topic partitions to the stream processors. If the number of available processors changes(for example, if a processors is shutdown or added), then the leader processor will regenerate the partition assignments and re-distribute it to all the processors. - -On processor group change, the act of re-assigning input topic partitions to the remaining live processors in the group is known as rebalancing the group. On failure of the leader processor of a standalone application, an another stream processor of the standalone application will be chosen as leader. - -## User guide - -Samza standalone is designed to help you to have more control over the deployment of the application. So it is your responsibility to configure and deploy the processors. In case of ZooKeeper coordination, you have to configure the URL for an instance of ZooKeeper. - -A stream processor is identified by a unique processorID which is generated by the pluggable ProcessorIdGenerator abstraction. ProcessorId of the stream processor is used with the coordination service. Samza supports UUID based ProcessorIdGenerator out of the box. - -The diagram below shows a input topic with three partitions and an standalone application with three processors consuming messages from it. - -<img src="/img/versioned/learn/documentation/standalone/standalone-application.jpg" alt="Standalone application" height="550px" width="700px" align="middle"> - -When a group is first initialized, each stream processor typically starts processing messages from either the earliest or latest offset of the input topic partition. The messages in each partition are sequentially delivered to the user defined processing function. As the stream processor makes progress, it commits the offsets of the messages it has successfully processed. For example, in the figure above, the stream processor position is at offset 7 and its last committed offset is at offset 3. - -When a input partition is reassigned to another processor in the group, the initial position is set to the last committed offset. If the processor-1 in the example above suddenly crashed, then the live processor taking over the partition would begin consumption from offset 3. In that case, it would not have to reprocess the messages up to the crashed processor's position of 3. - -### Setup dependencies - -Add the following samza-standalone maven dependencies to your project. - -```xml -<dependency> - <groupId>org.apache.samza</groupId> - <artifactId>samza-kafka_2.11</artifactId> - <version>1.0</version> -</dependency> -<dependency> - <groupId>org.apache.samza</groupId> - <artifactId>samza-core_2.11</artifactId> - <version>1.0</version> -</dependency> -<dependency> - <groupId>org.apache.samza</groupId> - <artifactId>samza-api</artifactId> - <version>1.0</version> -</dependency> -``` - -### Configuration - -A samza standalone application requires you to define the following mandatory configurations: - -```bash -job.coordinator.factory=org.apache.samza.zk.ZkJobCoordinatorFactory -job.coordinator.zk.connect=your_zk_connection(for local zookeeper, use localhost:2181) -task.name.grouper.factory=org.apache.samza.container.grouper.task.GroupByContainerIdsFactory -``` - -You have to configure the stream processor with the kafka brokers as defined in the following sample(we have assumed that the broker is running on localhost): - -```bash -systems.kafka.samza.factory=org.apache.samza.system.kafka.KafkaSystemFactory -systems.kafka.samza.msg.serde=json -systems.kafka.consumer.zookeeper.connect=localhost:2181 -systems.kafka.producer.bootstrap.servers=localhost:9092 -``` - -### Code sample - -Here's a sample standalone application with app.name set to sample-test. Running this class would launch a stream processor. - -```java -public class PageViewEventExample implements StreamApplication { - - public static void main(String[] args) { - CommandLine cmdLine = new CommandLine(); - OptionSet options = cmdLine.parser().parse(args); - Config config = cmdLine.loadConfig(options); - - ApplicationRunner runner = ApplicationRunners.getApplicationRunner(ApplicationClassUtils.fromConfig(config), config); - runner.run(); - runner.waitForFinish(); - } - - @Override - public void describe(StreamAppDescriptor appDesc) { - MessageStream<PageViewEvent> pageViewEvents = null; - pageViewEvents = appDesc.getInputStream("inputStream", new JsonSerdeV2<>(PageViewEvent.class)); - OutputStream<KV<String, PageViewCount>> pageViewEventPerMemberStream = - appDesc.getOutputStream("outputStream", new JsonSerdeV2<>(PageViewEvent.class)); - pageViewEvents.sendTo(pageViewEventPerMemberStream); - } -} -``` - -## Quick start guide - -The [Hello-samza](https://github.com/apache/samza-hello-samza/) project contains sample Samza standalone applications. Here are step by step instruction guide to install, build and run a standalone application binaries using the local zookeeper cluster for coordination. Check out the hello-samza project by running the following commands: - -```bash -git clone https://git.apache.org/samza-hello-samza.git hello-samza -cd hello-samza -``` - -### Setup Zookeeper - -Run the following command to install and start a local zookeeper cluster. - -```bash -./bin/grid install zookeeper -./bin/grid start zookeeper -``` - -### Setup Kafka - -Run the following command to install and start a local kafka cluster. - -```bash -./bin/grid install kafka -./bin/grid start zookeeper -``` - -### Build binaries - -Before you can run the standalone job, you need to build a package for it using the following command. - -```bash -mvn clean package -mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.15.0-SNAPSHOT-dist.tar.gz -C deploy/samza -``` - -### Deploy binaries - -To run the sample standalone application [WikipediaZkLocalApplication](https://github.com/apache/samza-hello-samza/blob/master/src/main/java/samza/examples/wikipedia/application/WikipediaZkLocalApplication.java) - -```bash -./bin/deploy.sh -./deploy/samza/bin/run-class.sh samza.examples.wikipedia.application.WikipediaZkLocalApplication --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-application-local-runner.properties -``` - -### Inspect results - -The standalone application reads messages from the wikipedia-edits topic, and calculates counts, every ten seconds, for all edits that were made during that window. It outputs these counts to the local wikipedia-stats kafka topic. To inspect events in output topic, run the following command. - -```bash -./deploy/kafka/bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic wikipedia-stats -``` - -Events produced to the output topic from the standalone application launched above will be of the following form: - -``` -{"is-talk":2,"bytes-added":5276,"edits":13,"unique-titles":13} -{"is-bot-edit":1,"is-talk":3,"bytes-added":4211,"edits":30,"unique-titles":30,"is-unpatrolled":1,"is-new":2,"is-minor":7} -``` - -# Coordinator internals - -A samza application is comprised of multiple stream processors. A processor can become part of a standalone application by setting its app.name(Ex: app.name=group\_1) and joining the group. In samza standalone, the input topic partitions are distributed between the available processors dynamically at runtime. If the number of available processors changes(for example, if some processors are shutdown or added), then the partition assignments will be regenerated and re-distributed to all the processors. One processor will be elected as leader and it will generate the partition assignments and distribute it to the other processors in the group. - -To mediate the partition assignments between processors, samza standalone relies upon a coordination service. The main responsibilities of coordination service are the following: - -**Leader Election** - Elects a single processor to generate the partition assignments and distribute it to other processors in the group. - -**Distributed barrier** - Coordination primitive used by the processors to reach consensus(agree) on an partition assignment. - -By default, embedded samza uses Zookeeper for coordinating between processors of an application and store the partition assignment state. Coordination sequence for a standalone application is listed below: - -1. Each processor(participant) will register with the coordination service(e.g: Zookeeper) with its participant ID. - -2. One of the participants will be elected as the leader. - -3. The leader will monitor the list of all the active participants. - -4. Whenever the list of the participants changes in a group, the leader will generate a new partition assignments for the current participants and persist it to a common storage. - -5. Participants are notified that the new partition assignment is available. Notification is done through the coordination service(e.g. ZooKeeper). - -6. The participants will stop processing, pick up the new partition assignment, and then resume processing. - -In order to ensure that no two partitions are processed by different processors, processing is paused and all the processors will synchronize on a distributed barrier. Once all the processors are paused, the new partition assignments are applied, after which processing resumes. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md b/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md index d4ac911..09d93e7 100644 --- a/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md +++ b/docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md @@ -24,7 +24,7 @@ This tutorial uses [hello-samza](../../../startup/hello-samza/{{site.version}}/) ### Upload the package {% highlight bash %} -hadoop fs -put ./target/hello-samza-0.14.1-dist.tar.gz /path/for/tgz +hadoop fs -put ./target/hello-samza-1.0.0-dist.tar.gz /path/for/tgz {% endhighlight %} ### Add HDFS configuration http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md b/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md index 5fdcd28..7657a21 100644 --- a/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md +++ b/docs/learn/tutorials/versioned/deploy-samza-to-CDH.md @@ -34,7 +34,7 @@ mvn clean package -Dhadoop.version=cdh5.4.0 There are a few ways of uploading the package to the cluster's HDFS. If you do not have the job package in your cluster, **scp** from you local machine to the cluster. Then run {% highlight bash %} -hadoop fs -put path/to/hello-samza-0.14.1-dist.tar.gz /path/for/tgz +hadoop fs -put path/to/hello-samza-1.0.0-dist.tar.gz /path/for/tgz {% endhighlight %} ### Get Deploying Scripts @@ -42,7 +42,7 @@ hadoop fs -put path/to/hello-samza-0.14.1-dist.tar.gz /path/for/tgz Untar the job package (assume you will run from the current directory) {% highlight bash %} -tar -xvf path/to/samza-job-package-0.14.1-dist.tar.gz -C ./ +tar -xvf path/to/samza-job-package-1.0.0-dist.tar.gz -C ./ {% endhighlight %} ### Add Package Path to Properties File http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/tutorials/versioned/hello-samza-high-level-yarn.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/hello-samza-high-level-yarn.md b/docs/learn/tutorials/versioned/hello-samza-high-level-yarn.md index dbdd63c..cfe589a 100644 --- a/docs/learn/tutorials/versioned/hello-samza-high-level-yarn.md +++ b/docs/learn/tutorials/versioned/hello-samza-high-level-yarn.md @@ -63,7 +63,7 @@ Then, you can continue w/ the following command in hello-samza project: {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.15.0-SNAPSHOT-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-1.0.1-SNAPSHOT-dist.tar.gz -C deploy/samza {% endhighlight %} ### Run a Samza Application http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/tutorials/versioned/hello-samza-high-level-zk.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/hello-samza-high-level-zk.md b/docs/learn/tutorials/versioned/hello-samza-high-level-zk.md index cc35ad0..e52c6e5 100644 --- a/docs/learn/tutorials/versioned/hello-samza-high-level-zk.md +++ b/docs/learn/tutorials/versioned/hello-samza-high-level-zk.md @@ -59,7 +59,7 @@ With the environment setup complete, let us move on to building the hello-samza {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.15.0-SNAPSHOT-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-1.0.1-SNAPSHOT-dist.tar.gz -C deploy/samza {% endhighlight %} We are now all set to deploy the application locally. http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/tutorials/versioned/remote-debugging-samza.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/remote-debugging-samza.md b/docs/learn/tutorials/versioned/remote-debugging-samza.md index a8ad863..2a59496 100644 --- a/docs/learn/tutorials/versioned/remote-debugging-samza.md +++ b/docs/learn/tutorials/versioned/remote-debugging-samza.md @@ -83,7 +83,7 @@ Once the grid starts, you can start the wikipedia-feed Samza job. {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.14.1-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-1.0.0-dist.tar.gz -C deploy/samza deploy/samza/bin/run-job.sh --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties {% endhighlight %} http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md b/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md index aa283c7..203c046 100644 --- a/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md +++ b/docs/learn/tutorials/versioned/run-in-multi-node-yarn.md @@ -145,7 +145,7 @@ vi src/main/config/wikipedia-feed.properties Change the yarn.package.path property to be: {% highlight jproperties %} -yarn.package.path=http://yourHostname:8000/target/hello-samza-0.14.1-dist.tar.gz +yarn.package.path=http://yourHostname:8000/target/hello-samza-1.0.0-dist.tar.gz {% endhighlight %} 3\. Compile hello-samza. @@ -153,7 +153,7 @@ yarn.package.path=http://yourHostname:8000/target/hello-samza-0.14.1-dist.tar.gz {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.14.1-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-1.0.0-dist.tar.gz -C deploy/samza {% endhighlight %} 4\. Deploy Samza job package to Http server.. http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/learn/tutorials/versioned/samza-rest-getting-started.md ---------------------------------------------------------------------- diff --git a/docs/learn/tutorials/versioned/samza-rest-getting-started.md b/docs/learn/tutorials/versioned/samza-rest-getting-started.md index 5ef7b4a..a4d4b9d 100644 --- a/docs/learn/tutorials/versioned/samza-rest-getting-started.md +++ b/docs/learn/tutorials/versioned/samza-rest-getting-started.md @@ -48,7 +48,7 @@ Run the following commands: {% highlight bash %} cd samza-rest/build/distributions/ mkdir -p deploy/samza-rest -tar -xvf ./samza-rest-0.15.0-SNAPSHOT.tgz -C deploy/samza-rest +tar -xvf ./samza-rest-1.0.1-SNAPSHOT.tgz -C deploy/samza-rest {% endhighlight %} #### Configure the Installations Path http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/docs/startup/hello-samza/versioned/index.md ---------------------------------------------------------------------- diff --git a/docs/startup/hello-samza/versioned/index.md b/docs/startup/hello-samza/versioned/index.md index e5b3139..1d53614 100644 --- a/docs/startup/hello-samza/versioned/index.md +++ b/docs/startup/hello-samza/versioned/index.md @@ -61,7 +61,7 @@ Then, you can continue w/ the following command in hello-samza project: {% highlight bash %} mvn clean package mkdir -p deploy/samza -tar -xvf ./target/hello-samza-0.15.0-SNAPSHOT-dist.tar.gz -C deploy/samza +tar -xvf ./target/hello-samza-1.0.1-SNAPSHOT-dist.tar.gz -C deploy/samza {% endhighlight %} ### Run a Samza Job http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/gradle.properties ---------------------------------------------------------------------- diff --git a/gradle.properties b/gradle.properties index c7e8447..b69f9ba 100644 --- a/gradle.properties +++ b/gradle.properties @@ -15,7 +15,7 @@ # specific language governing permissions and limitations # under the License. group=org.apache.samza -version=1.0.0-SNAPSHOT +version=1.0.1-SNAPSHOT scalaVersion=2.11 gradleVersion=2.8 http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/samza-test/src/main/config/join/README ---------------------------------------------------------------------- diff --git a/samza-test/src/main/config/join/README b/samza-test/src/main/config/join/README index 8ec3603..2268985 100644 --- a/samza-test/src/main/config/join/README +++ b/samza-test/src/main/config/join/README @@ -44,17 +44,17 @@ Deploy Zookeeper, YARN and Kafka: > cd $HELLO_SAMZA_SRC > for i in zookeeper kafka yarn; do ./bin/grid install $i; ./bin/grid start > $i; done -Update the "yarn.package.path" to $DEPLOY_DIR/samza-test_2.11-0.15.0-SNAPSHOT.tgz +Update the "yarn.package.path" to $DEPLOY_DIR/samza-test_2.11-1.0.1-SNAPSHOT.tgz > cd $SAMZA_SRC > vi samza-test/src/main/config/join/common.properties -yarn.package.path=file:///path/to/samza-hello-samza/deploy/samza-test_2.11-0.15.0-SNAPSHOT.tgz +yarn.package.path=file:///path/to/samza-hello-samza/deploy/samza-test_2.11-1.0.1-SNAPSHOT.tgz Then release and extract the test tarball: > cd $SAMZA_SRC > ./gradlew releaseTestJobs -> cp samza-test/build/distributions/samza-test_2.11-0.15.0-SNAPSHOT.tgz $DEPLOY_DIR +> cp samza-test/build/distributions/samza-test_2.11-1.0.1-SNAPSHOT.tgz $DEPLOY_DIR > mkdir $DEPLOY_DIR/samza -> tar -xvf $DEPLOY_DIR/samza-test_2.11-0.15.0-SNAPSHOT.tgz -C $DEPLOY_DIR/samza +> tar -xvf $DEPLOY_DIR/samza-test_2.11-1.0.1-SNAPSHOT.tgz -C $DEPLOY_DIR/samza Finally, create the kafka topics and start the samza jobs: > ./bin/setup-int-test.sh $DEPLOY_DIR http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/samza-test/src/main/python/configs/tests.json ---------------------------------------------------------------------- diff --git a/samza-test/src/main/python/configs/tests.json b/samza-test/src/main/python/configs/tests.json index 59db9db..33ec9e3 100644 --- a/samza-test/src/main/python/configs/tests.json +++ b/samza-test/src/main/python/configs/tests.json @@ -1,5 +1,5 @@ { - "samza_executable": "samza-test_2.11-0.15.0-SNAPSHOT.tgz", + "samza_executable": "samza-test_2.11-1.0.1-SNAPSHOT.tgz", "samza_install_path": "deploy/smoke_tests", "samza_config_factory": "org.apache.samza.config.factories.PropertiesConfigFactory" } http://git-wip-us.apache.org/repos/asf/samza/blob/00cd44ac/samza-test/src/main/python/stream_processor.py ---------------------------------------------------------------------- diff --git a/samza-test/src/main/python/stream_processor.py b/samza-test/src/main/python/stream_processor.py index a7a2bc8..dacd59c 100644 --- a/samza-test/src/main/python/stream_processor.py +++ b/samza-test/src/main/python/stream_processor.py @@ -43,7 +43,7 @@ class StreamProcessor: logger.info('Running processor start command: {0}'.format(self.processor_start_command)) self.deployment_config = { 'install_path': os.path.join(runtime.get_active_config('remote_install_path'), 'deploy/{0}'.format(self.processor_id)), - 'executable': 'samza-test_2.11-0.15.0-SNAPSHOT.tgz', + 'executable': 'samza-test_2.11-1.0.1-SNAPSHOT.tgz', 'post_install_cmds': [], 'start_command': self.processor_start_command, 'stop_command': '',
