I'm also a +1 on the newer APIs. We had a lot of learnings from using
flatMapGroupsWithState and I believe that we can make the APIs a lot easier
to use.
On Wed, Nov 29, 2023 at 6:43 PM Anish Shrigondekar
wrote:
> Hi dev,
>
> Addressed the comments that Jungtaek had on the doc. Bumping the
Hi Pavan,
Thanks for your answers.
Given these responses , it seems like you have already taken a
comprehensive approach to address the challenges associated with dynamic
scaling in Spark Structured Streaming. IMO, It would also be beneficial to
engage with other members as well, or gather
+1. Looking forward to seeing how the new API brings in new streaming use
cases!
Best Regards,
Shixiong Zhu
On Wed, Nov 29, 2023 at 6:42 PM Anish Shrigondekar
wrote:
> Hi dev,
>
> Addressed the comments that Jungtaek had on the doc. Bumping the thread
> once again to see if other folks have
Hi Mich,
As always thanks for looking keenly on the design, really appreciate your
inputs on this Ticket. Would love to improve this further and cover more
edge-cases if any.
I can answer the concerns you have below. I believe I have covered some of
them in the proposal, If at all I missed out
