Hi, We are working on the new arbitrary state API support for streaming called transformWithState - (epic here - https://issues.apache.org/jira/browse/SPARK-46815). We have few PRs left and we would like to get these into Spark 4.0. However, if we can't get them in within the deadline, we will target the next 4.1 release.
cc - @Jungtaek Lim <kabhwan.opensou...@gmail.com> Thanks, Anish On Thu, Jan 16, 2025 at 5:30 AM Paddy Xu <xupa...@gmail.com> wrote: > I am working on a project > https://issues.apache.org/jira/browse/SPARK-48918 which targets Spark > 4.0. Some items on the list would need to go into this new branch. > > Now three PRs are ongoing: > https://github.com/apache/spark/pull/49373 > https://github.com/apache/spark/pull/49339 > https://github.com/apache/spark/pull/49111 > > On 2025/01/15 13:41:07 Wenchen Fan wrote: > > Hi all, > > > > We have cut the "branch-4.0" and I'm sending this email to collect the > > information for ongoing projects targeting Spark 4.0. Please reply to > this > > email to share the project progress with the community. > > > > Note that, the scheduled code freeze date is Feb 1, and RC1 cut date is > Feb > > 15. > > > > Thanks, > > Wenchen > > > > Cheers, > Paddy > > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
