Repository: spark Updated Branches: refs/heads/master e86fbdb1e -> ddc5baf17
[SPARK-8320] [STREAMING] Add example in streaming programming guide that shows union of multiple input streams Added python code to https://spark.apache.org/docs/latest/streaming-programming-guide.html to the Level of Parallelism in Data Receiving section. Please review and let me know if there are any additional changes that are needed. Thank you. Author: Neelesh Srinivas Salian <[email protected]> Closes #6862 from nssalian/SPARK-8320 and squashes the following commits: 4bfd126 [Neelesh Srinivas Salian] Changed loop structure to be more in line with Python style e5345de [Neelesh Srinivas Salian] Changes to kafak append, for loop and show to print() 3fc5c6d [Neelesh Srinivas Salian] SPARK-8320 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ddc5baf1 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ddc5baf1 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ddc5baf1 Branch: refs/heads/master Commit: ddc5baf17d7b09623b91190ee7754a6c8f7b5d10 Parents: e86fbdb Author: Neelesh Srinivas Salian <[email protected]> Authored: Thu Jun 18 09:44:36 2015 -0700 Committer: Davies Liu <[email protected]> Committed: Thu Jun 18 09:44:36 2015 -0700 ---------------------------------------------------------------------- docs/streaming-programming-guide.md | 8 ++++++++ 1 file changed, 8 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/ddc5baf1/docs/streaming-programming-guide.md ---------------------------------------------------------------------- diff --git a/docs/streaming-programming-guide.md b/docs/streaming-programming-guide.md index 1eb3b30..b784d59 100644 --- a/docs/streaming-programming-guide.md +++ b/docs/streaming-programming-guide.md @@ -1937,6 +1937,14 @@ JavaPairDStream<String, String> unifiedStream = streamingContext.union(kafkaStre unifiedStream.print(); {% endhighlight %} </div> +<div data-lang="python" markdown="1"> +{% highlight python %} +numStreams = 5 +kafkaStreams = [KafkaUtils.createStream(...) for _ in range (numStreams)] +unifiedStream = streamingContext.union(kafkaStreams) +unifiedStream.print() +{% endhighlight %} +</div> </div> Another parameter that should be considered is the receiver's blocking interval, --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
