This is an automated email from the ASF dual-hosted git repository. mergebot-role pushed a commit to branch mergebot in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit a232c95ddf7fe8b4a521d12291ec5c80290b622d Author: Thomas Weise <[email protected]> AuthorDate: Sun Jul 22 15:43:22 2018 -0700 Add instructions for local Flink cluster. --- src/contribute/portability.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/contribute/portability.md b/src/contribute/portability.md index cb16a66..4e1df83 100644 --- a/src/contribute/portability.md +++ b/src/contribute/portability.md @@ -174,18 +174,25 @@ their respective components. ### Status -MVP complete (for Flink runner). See the +MVP in progress (near completion for Flink runner). See the [Portability support table](https://docs.google.com/spreadsheets/d/1KDa_FGn1ShjomGd-UUDOhuh2q73de2tPz6BqHpzqvNI/edit?usp=sharing) for details. #### Running Python wordcount on Flink The Flink runner is currently the only runner to support portable pipeline execution. -To run a basic Python wordcount (in batch mode): +To run a basic Python wordcount (in batch mode) with embedded Flink: 1. Run once to build the SDK harness container: `./gradlew -p sdks/python/container docker` 2. Start the Flink portable JobService endpoint: `./gradlew :beam-runners-flink_2.11-job-server:runShadow` 3. Submit the wordcount pipeline to above endpoint: `./gradlew :beam-sdks-python:portableWordCount` +To run on a separate [Flink cluster](https://ci.apache.org/projects/flink/flink-docs-release-1.5/quickstart/setup_quickstart.html): + +1. Start local Flink cluster +2. Create shaded JobService jar: `./gradlew :beam-runners-flink_2.11-job-server:installShadowDist` +3. Start JobService with Flink web service endpoint: `java -jar ./runners/flink/job-server/build/install/beam-runners-flink_2.11-job-server-shadow/lib/beam-runners-flink_2.11-job-server-*.jar--job-host=localhost:8099" "--artifacts-dir=/tmp/flink-artifacts" "--flink-master-url=localhost:8081"` +4. Submit the pipeline. + Note: A subset of the functionality is also supported in streaming mode; use `--streaming` in the command line to enable it.
