+1
On Thu, Jan 12, 2023 at 5:08 PM Tathagata Das <tathagata.das1...@gmail.com> wrote: > +1 > > On Thu, Jan 12, 2023 at 7:46 PM Hyukjin Kwon <gurwls...@gmail.com> wrote: > >> +1 >> >> On Fri, 13 Jan 2023 at 08:51, Jungtaek Lim <kabhwan.opensou...@gmail.com> >> wrote: >> >>> bump for more visibility. >>> >>> On Wed, Jan 11, 2023 at 12:20 PM Jungtaek Lim < >>> kabhwan.opensou...@gmail.com> wrote: >>> >>>> Hi dev, >>>> >>>> I'd like to propose the deprecation of DStream in Spark 3.4, in favor >>>> of promoting Structured Streaming. >>>> (Sorry for the late proposal, if we don't make the change in 3.4, we >>>> will have to wait for another 6 months.) >>>> >>>> We have been focusing on Structured Streaming for years (across >>>> multiple major and minor versions), and during the time we haven't made any >>>> improvements for DStream. Furthermore, recently we updated the DStream doc >>>> to explicitly say DStream is a legacy project. >>>> >>>> https://spark.apache.org/docs/latest/streaming-programming-guide.html#note >>>> >>>> The baseline of deprecation is that we don't see a particular use case >>>> which only DStream solves. This is a different story with GraphX and MLLIB, >>>> as we don't have replacements for that. >>>> >>>> The proposal does not mean we will remove the API soon, as the Spark >>>> project has been making deprecation against public API. I don't intend to >>>> propose the target version for removal. The goal is to guide users to >>>> refrain from constructing a new workload with DStream. We might want to go >>>> with this in future, but it would require a new discussion thread at that >>>> time. >>>> >>>> What do you think? >>>> >>>> Thanks, >>>> Jungtaek Lim (HeartSaVioR) >>>> >>>
