This is an automated email from the ASF dual-hosted git repository. kabhwan pushed a commit to branch branch-3.1 in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.1 by this push: new 3362ea0bb87 [SPARK-39219][DOC] Promote Structured Streaming over DStream 3362ea0bb87 is described below commit 3362ea0bb87ff73b12905fc78b8ff412192e6024 Author: Jungtaek Lim <kabhwan.opensou...@gmail.com> AuthorDate: Thu May 19 11:50:05 2022 +0900 [SPARK-39219][DOC] Promote Structured Streaming over DStream ### What changes were proposed in this pull request? This PR proposes to add NOTE section for DStream guide doc to promote Structured Streaming. Screenshot: <img width="992" alt="screenshot-spark-streaming-programming-guide-change" src="https://user-images.githubusercontent.com/1317309/168977732-4c32db9a-0fb1-4a82-a542-bf385e5f3683.png"> ### Why are the changes needed? We see efforts of community are more focused on Structured Streaming (based on Spark SQL) than Spark Streaming (DStream). We would like to encourage end users to use Structured Streaming than Spark Streaming whenever possible for their workloads. ### Does this PR introduce _any_ user-facing change? Yes, doc change. ### How was this patch tested? N/A Closes #36590 from HeartSaVioR/SPARK-39219. Authored-by: Jungtaek Lim <kabhwan.opensou...@gmail.com> Signed-off-by: Jungtaek Lim <kabhwan.opensou...@gmail.com> (cherry picked from commit 7d153392db2f61104da0af1cb175f4ee7c7fbc38) Signed-off-by: Jungtaek Lim <kabhwan.opensou...@gmail.com> --- docs/streaming-programming-guide.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/streaming-programming-guide.md b/docs/streaming-programming-guide.md index b13bf8dd2e7..4a104238a6d 100644 --- a/docs/streaming-programming-guide.md +++ b/docs/streaming-programming-guide.md @@ -23,6 +23,14 @@ license: | * This will become a table of contents (this text will be scraped). {:toc} +# Note + +Spark Streaming is the previous generation of Spark’s streaming engine. There are no longer +updates to Spark Streaming and it’s a legacy project. There is a newer and easier to use +streaming engine in Spark called Structured Streaming. You should use Spark Structured Streaming +for your streaming applications and pipelines. See +[Structured Streaming Programming Guide](structured-streaming-programming-guide.html). + # Overview Spark Streaming is an extension of the core Spark API that enables scalable, high-throughput, fault-tolerant stream processing of live data streams. Data can be ingested from many sources --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org