This is an automated email from the ASF dual-hosted git repository.
altay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new b5f7a5d [BEAM-7632] Update Python quickstart guide for Flink and Spark
new 4a7ba2d Merge pull request #8949 from ibzib/quickstart
b5f7a5d is described below
commit b5f7a5d382390b1b81ca560d2bb8d111fb6512fb
Author: Kyle Weaver <[email protected]>
AuthorDate: Tue Jun 25 17:11:19 2019 -0700
[BEAM-7632] Update Python quickstart guide for Flink and Spark
---
website/src/get-started/quickstart-py.md | 9 ++++++---
website/src/get-started/wordcount-example.md | 9 ++++++---
website/src/roadmap/portability.md | 29 ++++++++++++++++++----------
3 files changed, 31 insertions(+), 16 deletions(-)
diff --git a/website/src/get-started/quickstart-py.md
b/website/src/get-started/quickstart-py.md
index 95740d5..ae07ec3 100644
--- a/website/src/get-started/quickstart-py.md
+++ b/website/src/get-started/quickstart-py.md
@@ -176,17 +176,20 @@ This runner is not yet available for the Python SDK.
{:.runner-flink-local}
```
-This runner is not yet available for the Python SDK.
+Currently, running wordcount.py on Flink requires a full download of the Beam
source code.
+See https://beam.apache.org/roadmap/portability/#python-on-flink for more
information.
```
{:.runner-flink-cluster}
```
-This runner is not yet available for the Python SDK.
+Currently, running wordcount.py on Flink requires a full download of the Beam
source code.
+See https://beam.apache.org/documentation/runners/flink/ for more information.
```
{:.runner-spark}
```
-This runner is not yet available for the Python SDK.
+Currently, running wordcount.py on Spark requires a full download of the Beam
source code.
+See https://beam.apache.org/roadmap/portability/#python-on-spark for more
information.
```
{:.runner-dataflow}
diff --git a/website/src/get-started/wordcount-example.md
b/website/src/get-started/wordcount-example.md
index fba69b3..f0a8655 100644
--- a/website/src/get-started/wordcount-example.md
+++ b/website/src/get-started/wordcount-example.md
@@ -407,17 +407,20 @@ This runner is not yet available for the Python SDK.
{:.runner-flink-local}
```
-This runner is not yet available for the Python SDK.
+Currently, running wordcount.py on Flink requires a full download of the Beam
source code.
+See https://beam.apache.org/roadmap/portability/#python-on-flink for more
information.
```
{:.runner-flink-cluster}
```
-This runner is not yet available for the Python SDK.
+Currently, running wordcount.py on Flink requires a full download of the Beam
source code.
+See https://beam.apache.org/documentation/runners/flink/ for more information.
```
{:.runner-spark}
```
-This runner is not yet available for the Python SDK.
+Currently, running wordcount.py on Spark requires a full download of the Beam
source code.
+See https://beam.apache.org/roadmap/portability/#python-on-spark for more
information.
```
{:.runner-dataflow}
diff --git a/website/src/roadmap/portability.md
b/website/src/roadmap/portability.md
index 7c0dd2a..882a6f3 100644
--- a/website/src/roadmap/portability.md
+++ b/website/src/roadmap/portability.md
@@ -144,25 +144,34 @@ their respective components.
MVP, and FeatureCompletness nearly done (missing SDF, timers) for
SDKs, Python ULR, and shared java runners library.
-Flink is the first runner to fully leverage this, with focus moving to
-Performance.
+Currently, the Flink and Spark runners support portable pipeline execution.
See the
[Portability support
table](https://s.apache.org/apache-beam-portability-support-table)
for details.
-### Running Python wordcount on Flink or Spark {#python-on-flink}
+### Running Python wordcount on Flink {#python-on-flink}
-Currently, the Flink and Spark runners support portable pipeline execution.
-To run a basic Python wordcount (in batch mode) with embedded Flink or Spark:
+To run a basic Python wordcount (in batch mode) with embedded Flink:
1. Run once to build the SDK harness container: `./gradlew
:sdks:python:container:docker`
-2. Choose one:
- * Start the Flink portable JobService endpoint: `./gradlew
:runners:flink:1.5:job-server:runShadow`
- * Or start the Spark portable JobService endpoint: `./gradlew
:runners:spark:job-server:runShadow`
-3. Submit the wordcount pipeline to above endpoint: `./gradlew
:sdks:python:portableWordCount -PjobEndpoint=localhost:8099
-PenvironmentType=LOOPBACK`
+2. Start the Flink portable JobService endpoint: `./gradlew
:runners:flink:1.5:job-server:runShadow`
+3. In a new terminal, submit the wordcount pipeline to above endpoint:
`./gradlew :sdks:python:portableWordCount -PjobEndpoint=localhost:8099
-PenvironmentType=LOOPBACK`
-To run the pipeline in streaming mode (currently only supported on Flink):
`./gradlew :sdks:python:portableWordCount -PjobEndpoint=localhost:8099
-Pstreaming`
+To run the pipeline in streaming mode: `./gradlew
:sdks:python:portableWordCount -PjobEndpoint=localhost:8099 -Pstreaming`
Please see the [Flink Runner page]({{ site.baseurl
}}/documentation/runners/flink/) for more information on
how to run portable pipelines on top of Flink.
+### Running Python wordcount on Spark {#python-on-spark}
+
+To run a basic Python wordcount (in batch mode) with embedded Spark:
+
+1. Run once to build the SDK harness container: `./gradlew
:sdks:python:container:docker`
+2. Start the Spark portable JobService endpoint: `./gradlew
:runners:spark:job-server:runShadow`
+3. In a new terminal, submit the wordcount pipeline to above endpoint:
`./gradlew :sdks:python:portableWordCount -PjobEndpoint=localhost:8099
-PenvironmentType=LOOPBACK`
+
+Python streaming mode is not yet supported on Spark.
+
+Please see the [Spark Runner page]({{ site.baseurl
}}/documentation/runners/spark/) for more information on
+how to run portable pipelines on top of Spark.
+
