Yes, exactly. I'm sorry to bring confusion - should have clarified action items on the proposal.
On Fri, Jan 13, 2023 at 3:31 PM Dongjoon Hyun <dongjoon.h...@gmail.com> wrote: > Then, could you elaborate `the proposed code change` specifically? > Maybe, usual deprecation warning logs and annotation on the API? > > > On Thu, Jan 12, 2023 at 10:05 PM Jungtaek Lim < > kabhwan.opensou...@gmail.com> wrote: > >> Maybe I need to clarify - my proposal is "explicitly" deprecating it, >> which incurs code change for sure. Guidance on the Spark website is done >> already as I mentioned - we updated the DStream doc page to mention that >> DStream is a "legacy" project and users should move to SS. I don't feel >> this is sufficient to refrain users from using it, hence initiating >> this proposal. >> >> Sorry to make confusion. I just wanted to make sure the goal of the >> proposal is not "removing" the API. The discussion on the removal of API >> doesn't tend to go well, so I wanted to make sure I don't mean that. >> >> On Fri, Jan 13, 2023 at 2:46 PM Dongjoon Hyun <dongjoon.h...@gmail.com> >> wrote: >> >>> +1 for the proposal (guiding only without any code change). >>> >>> Thanks, >>> Dongjoon. >>> >>> On Thu, Jan 12, 2023 at 9:33 PM Shixiong Zhu <zsxw...@gmail.com> wrote: >>> >>>> +1 >>>> >>>> >>>> On Thu, Jan 12, 2023 at 5:08 PM Tathagata Das < >>>> tathagata.das1...@gmail.com> wrote: >>>> >>>>> +1 >>>>> >>>>> On Thu, Jan 12, 2023 at 7:46 PM Hyukjin Kwon <gurwls...@gmail.com> >>>>> wrote: >>>>> >>>>>> +1 >>>>>> >>>>>> On Fri, 13 Jan 2023 at 08:51, Jungtaek Lim < >>>>>> kabhwan.opensou...@gmail.com> wrote: >>>>>> >>>>>>> bump for more visibility. >>>>>>> >>>>>>> On Wed, Jan 11, 2023 at 12:20 PM Jungtaek Lim < >>>>>>> kabhwan.opensou...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi dev, >>>>>>>> >>>>>>>> I'd like to propose the deprecation of DStream in Spark 3.4, in >>>>>>>> favor of promoting Structured Streaming. >>>>>>>> (Sorry for the late proposal, if we don't make the change in 3.4, >>>>>>>> we will have to wait for another 6 months.) >>>>>>>> >>>>>>>> We have been focusing on Structured Streaming for years (across >>>>>>>> multiple major and minor versions), and during the time we haven't >>>>>>>> made any >>>>>>>> improvements for DStream. Furthermore, recently we updated the DStream >>>>>>>> doc >>>>>>>> to explicitly say DStream is a legacy project. >>>>>>>> >>>>>>>> https://spark.apache.org/docs/latest/streaming-programming-guide.html#note >>>>>>>> >>>>>>>> The baseline of deprecation is that we don't see a particular use >>>>>>>> case which only DStream solves. This is a different story with GraphX >>>>>>>> and >>>>>>>> MLLIB, as we don't have replacements for that. >>>>>>>> >>>>>>>> The proposal does not mean we will remove the API soon, as the >>>>>>>> Spark project has been making deprecation against public API. I don't >>>>>>>> intend to propose the target version for removal. The goal is to guide >>>>>>>> users to refrain from constructing a new workload with DStream. We >>>>>>>> might >>>>>>>> want to go with this in future, but it would require a new discussion >>>>>>>> thread at that time. >>>>>>>> >>>>>>>> What do you think? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Jungtaek Lim (HeartSaVioR) >>>>>>>> >>>>>>>
