Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[Stefan Richter]

+1

Thanks,
Stefan


> On 5. Sep 2023, at 10:45, David Morávek  wrote:
> 
> +1 since there is an alternative, more complete implementation available
> 
> Best,
> D.
> 
> On Sat, Sep 2, 2023 at 12:07 AM David Anderson  > wrote:
> 
>> +1
>> 
>> Keeping the legacy implementation in place is confusing and encourages
>> adoption of something that really shouldn't be used.
>> 
>> Thanks for driving this,
>> David
>> 
>> On Fri, Sep 1, 2023 at 8:45 AM Jing Ge  wrote:
>>> 
>>> Hi Wencong,
>>> 
>>> Thanks for your clarification! +1
>>> 
>>> Best regards,
>>> Jing
>>> 
>>> On Fri, Sep 1, 2023 at 12:36 PM Wencong Liu 
>> wrote:
>>> 
 Hi Jing,
 
 
 Thanks for your reply！
 
 
> Or the "independent module extraction" mentioned in the FLIP does
>> mean an
 independent module in Flink?
 
 
 Yes. If there are submodules in Flink repository needs the iteration
 (currently not),
 we could consider extracting them to a new submodule of Flink.
 
 
> users will have to add one more dependency of Flink ML. If iteration
>> is
 the
 only feature they need, it will look a little bit weird.
 
 
 If users only need to execute iteration jobs, they can simply remove
>> the
 Flink
 dependency and add the necessary dependencies related to Flink ML.
 However,
 they can still utilize the DataStream API as it is also a dependency of
 Flink ML.
 
 
 Keeping an iteration submodule in Flink repository and make Flink ML
 depends it
 is also another solution. But the current implementation of Iteration
>> in
 DataStream
 should be removed definitely due to its Incompleteness.
 
 
 The placement of the Iteration API in the repository is a topic that
>> has
 multiple
 potential solutions. WDYT?
 
 
 Best,
 Wencong
 
 
 
 
 
 
 
 
 
 
 
 At 2023-09-01 17:59:34, "Jing Ge"  wrote:
> Hi Wencong,
> 
> Thanks for the proposal!
> 
> "The Iteration API in DataStream is planned be deprecated in Flink
>> 1.19
 and
> then finally removed in Flink 2.0. For the users that rely on the
 Iteration
> API in DataStream, they will have to migrate to Flink ML."
> - Does it make sense to migrate the iteration module into Flink
>> directly?
> Or the "independent module extraction" mentioned in the FLIP does
>> mean an
> independent module in Flink? Since the iteration will be removed in
>> Flink,
> users will have to add one more dependency of Flink ML. If iteration
>> is
 the
> only feature they need, it will look a little bit weird.
> 
> 
> Best regards,
> Jing
> 
> On Fri, Sep 1, 2023 at 11:05 AM weijie guo >> 
> wrote:
> 
>> Thanks, +1 for this.
>> 
>> Best regards,
>> 
>> Weijie
>> 
>> 
>> Yangze Guo  于2023年9月1日周五 14:29写道：
>> 
>>> +1
>>> 
>>> Thanks for driving this.
>>> 
>>> Best,
>>> Yangze Guo
>>> 
>>> On Fri, Sep 1, 2023 at 2:00 PM Xintong Song <
>> tonysong...@gmail.com>
>> wrote:
 
 +1
 
 Best,
 
 Xintong
 
 
 
 On Fri, Sep 1, 2023 at 1:11 PM Dong Lin 
 wrote:
 
> Thanks Wencong for initiating the discussion.
> 
> +1 for the proposal.
> 
> On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu <
>> liuwencle...@163.com
> 
>>> wrote:
> 
>> Hi devs,
>> 
>> I would like to start a discussion on FLIP-357: Deprecate
 Iteration
>>> API
> of
>> DataStream [1].
>> 
>> Currently, the Iteration API of DataStream is incomplete.
>> For
>>> instance,
> it
>> lacks support
>> for iteration in sync mode and exactly once semantics.
>> Additionally,
>>> it
>> does not offer the
>> ability to set iteration termination conditions. As a
>> result,
 it's
>>> hard
>> for developers to
>> build an iteration pipeline by DataStream in the practical
>>> applications
>> such as machine learning.
>> 
>> FLIP-176: Unified Iteration to Support Algorithms [2] has
>> introduced
>>> a
>> unified iteration library
>> in the Flink ML repository. This library addresses all the
 issues
>>> present
>> in the Iteration API of
>> DataStream and could provide solution for all the iteration
>>> use-cases.
>> However, maintaining two
>> separate implementations of iteration in both the Flink
 repository
>>> and
> the
>> Flink ML repository
>> would introduce unnecessary complexity and make it
>> difficult to
>>> maintain
>> the Iteration API.
>> 
>> As such I propose 

Re: Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[David Morávek]

+1 since there is an alternative, more complete implementation available

Best,
D.

On Sat, Sep 2, 2023 at 12:07 AM David Anderson  wrote:

> +1
>
> Keeping the legacy implementation in place is confusing and encourages
> adoption of something that really shouldn't be used.
>
> Thanks for driving this,
> David
>
> On Fri, Sep 1, 2023 at 8:45 AM Jing Ge  wrote:
> >
> > Hi Wencong,
> >
> > Thanks for your clarification! +1
> >
> > Best regards,
> > Jing
> >
> > On Fri, Sep 1, 2023 at 12:36 PM Wencong Liu 
> wrote:
> >
> > > Hi Jing,
> > >
> > >
> > > Thanks for your reply！
> > >
> > >
> > > > Or the "independent module extraction" mentioned in the FLIP does
> mean an
> > > independent module in Flink?
> > >
> > >
> > > Yes. If there are submodules in Flink repository needs the iteration
> > > (currently not),
> > > we could consider extracting them to a new submodule of Flink.
> > >
> > >
> > > > users will have to add one more dependency of Flink ML. If iteration
> is
> > > the
> > > only feature they need, it will look a little bit weird.
> > >
> > >
> > > If users only need to execute iteration jobs, they can simply remove
> the
> > > Flink
> > > dependency and add the necessary dependencies related to Flink ML.
> > > However,
> > > they can still utilize the DataStream API as it is also a dependency of
> > > Flink ML.
> > >
> > >
> > > Keeping an iteration submodule in Flink repository and make Flink ML
> > > depends it
> > > is also another solution. But the current implementation of Iteration
> in
> > > DataStream
> > > should be removed definitely due to its Incompleteness.
> > >
> > >
> > > The placement of the Iteration API in the repository is a topic that
> has
> > > multiple
> > > potential solutions. WDYT?
> > >
> > >
> > > Best,
> > > Wencong
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > At 2023-09-01 17:59:34, "Jing Ge"  wrote:
> > > >Hi Wencong,
> > > >
> > > >Thanks for the proposal!
> > > >
> > > >"The Iteration API in DataStream is planned be deprecated in Flink
> 1.19
> > > and
> > > >then finally removed in Flink 2.0. For the users that rely on the
> > > Iteration
> > > >API in DataStream, they will have to migrate to Flink ML."
> > > >- Does it make sense to migrate the iteration module into Flink
> directly?
> > > >Or the "independent module extraction" mentioned in the FLIP does
> mean an
> > > >independent module in Flink? Since the iteration will be removed in
> Flink,
> > > >users will have to add one more dependency of Flink ML. If iteration
> is
> > > the
> > > >only feature they need, it will look a little bit weird.
> > > >
> > > >
> > > >Best regards,
> > > >Jing
> > > >
> > > >On Fri, Sep 1, 2023 at 11:05 AM weijie guo  >
> > > >wrote:
> > > >
> > > >> Thanks, +1 for this.
> > > >>
> > > >> Best regards,
> > > >>
> > > >> Weijie
> > > >>
> > > >>
> > > >> Yangze Guo  于2023年9月1日周五 14:29写道：
> > > >>
> > > >> > +1
> > > >> >
> > > >> > Thanks for driving this.
> > > >> >
> > > >> > Best,
> > > >> > Yangze Guo
> > > >> >
> > > >> > On Fri, Sep 1, 2023 at 2:00 PM Xintong Song <
> tonysong...@gmail.com>
> > > >> wrote:
> > > >> > >
> > > >> > > +1
> > > >> > >
> > > >> > > Best,
> > > >> > >
> > > >> > > Xintong
> > > >> > >
> > > >> > >
> > > >> > >
> > > >> > > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin 
> > > wrote:
> > > >> > >
> > > >> > > > Thanks Wencong for initiating the discussion.
> > > >> > > >
> > > >> > > > +1 for the proposal.
> > > >> > > >
> > > >> > > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu <
> liuwencle...@163.com
> > > >
> > > >> > wrote:
> > > >> > > >
> > > >> > > > > Hi devs,
> > > >> > > > >
> > > >> > > > > I would like to start a discussion on FLIP-357: Deprecate
> > > Iteration
> > > >> > API
> > > >> > > > of
> > > >> > > > > DataStream [1].
> > > >> > > > >
> > > >> > > > > Currently, the Iteration API of DataStream is incomplete.
> For
> > > >> > instance,
> > > >> > > > it
> > > >> > > > > lacks support
> > > >> > > > > for iteration in sync mode and exactly once semantics.
> > > >> Additionally,
> > > >> > it
> > > >> > > > > does not offer the
> > > >> > > > > ability to set iteration termination conditions. As a
> result,
> > > it's
> > > >> > hard
> > > >> > > > > for developers to
> > > >> > > > > build an iteration pipeline by DataStream in the practical
> > > >> > applications
> > > >> > > > > such as machine learning.
> > > >> > > > >
> > > >> > > > > FLIP-176: Unified Iteration to Support Algorithms [2] has
> > > >> introduced
> > > >> > a
> > > >> > > > > unified iteration library
> > > >> > > > > in the Flink ML repository. This library addresses all the
> > > issues
> > > >> > present
> > > >> > > > > in the Iteration API of
> > > >> > > > > DataStream and could provide solution for all the iteration
> > > >> > use-cases.
> > > >> > > > > However, maintaining two
> > > >> > > > > separate implementations of iteration in both the Flink
> > > repository
> > > >> > and
> > > >> > > > the
> > > >> > > > > Flink ML 

Re: Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[David Anderson]

+1

Keeping the legacy implementation in place is confusing and encourages
adoption of something that really shouldn't be used.

Thanks for driving this,
David

On Fri, Sep 1, 2023 at 8:45 AM Jing Ge  wrote:
>
> Hi Wencong,
>
> Thanks for your clarification! +1
>
> Best regards,
> Jing
>
> On Fri, Sep 1, 2023 at 12:36 PM Wencong Liu  wrote:
>
> > Hi Jing,
> >
> >
> > Thanks for your reply！
> >
> >
> > > Or the "independent module extraction" mentioned in the FLIP does mean an
> > independent module in Flink?
> >
> >
> > Yes. If there are submodules in Flink repository needs the iteration
> > (currently not),
> > we could consider extracting them to a new submodule of Flink.
> >
> >
> > > users will have to add one more dependency of Flink ML. If iteration is
> > the
> > only feature they need, it will look a little bit weird.
> >
> >
> > If users only need to execute iteration jobs, they can simply remove the
> > Flink
> > dependency and add the necessary dependencies related to Flink ML.
> > However,
> > they can still utilize the DataStream API as it is also a dependency of
> > Flink ML.
> >
> >
> > Keeping an iteration submodule in Flink repository and make Flink ML
> > depends it
> > is also another solution. But the current implementation of Iteration in
> > DataStream
> > should be removed definitely due to its Incompleteness.
> >
> >
> > The placement of the Iteration API in the repository is a topic that has
> > multiple
> > potential solutions. WDYT?
> >
> >
> > Best,
> > Wencong
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > At 2023-09-01 17:59:34, "Jing Ge"  wrote:
> > >Hi Wencong,
> > >
> > >Thanks for the proposal!
> > >
> > >"The Iteration API in DataStream is planned be deprecated in Flink 1.19
> > and
> > >then finally removed in Flink 2.0. For the users that rely on the
> > Iteration
> > >API in DataStream, they will have to migrate to Flink ML."
> > >- Does it make sense to migrate the iteration module into Flink directly?
> > >Or the "independent module extraction" mentioned in the FLIP does mean an
> > >independent module in Flink? Since the iteration will be removed in Flink,
> > >users will have to add one more dependency of Flink ML. If iteration is
> > the
> > >only feature they need, it will look a little bit weird.
> > >
> > >
> > >Best regards,
> > >Jing
> > >
> > >On Fri, Sep 1, 2023 at 11:05 AM weijie guo 
> > >wrote:
> > >
> > >> Thanks, +1 for this.
> > >>
> > >> Best regards,
> > >>
> > >> Weijie
> > >>
> > >>
> > >> Yangze Guo  于2023年9月1日周五 14:29写道：
> > >>
> > >> > +1
> > >> >
> > >> > Thanks for driving this.
> > >> >
> > >> > Best,
> > >> > Yangze Guo
> > >> >
> > >> > On Fri, Sep 1, 2023 at 2:00 PM Xintong Song 
> > >> wrote:
> > >> > >
> > >> > > +1
> > >> > >
> > >> > > Best,
> > >> > >
> > >> > > Xintong
> > >> > >
> > >> > >
> > >> > >
> > >> > > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin 
> > wrote:
> > >> > >
> > >> > > > Thanks Wencong for initiating the discussion.
> > >> > > >
> > >> > > > +1 for the proposal.
> > >> > > >
> > >> > > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu  > >
> > >> > wrote:
> > >> > > >
> > >> > > > > Hi devs,
> > >> > > > >
> > >> > > > > I would like to start a discussion on FLIP-357: Deprecate
> > Iteration
> > >> > API
> > >> > > > of
> > >> > > > > DataStream [1].
> > >> > > > >
> > >> > > > > Currently, the Iteration API of DataStream is incomplete. For
> > >> > instance,
> > >> > > > it
> > >> > > > > lacks support
> > >> > > > > for iteration in sync mode and exactly once semantics.
> > >> Additionally,
> > >> > it
> > >> > > > > does not offer the
> > >> > > > > ability to set iteration termination conditions. As a result,
> > it's
> > >> > hard
> > >> > > > > for developers to
> > >> > > > > build an iteration pipeline by DataStream in the practical
> > >> > applications
> > >> > > > > such as machine learning.
> > >> > > > >
> > >> > > > > FLIP-176: Unified Iteration to Support Algorithms [2] has
> > >> introduced
> > >> > a
> > >> > > > > unified iteration library
> > >> > > > > in the Flink ML repository. This library addresses all the
> > issues
> > >> > present
> > >> > > > > in the Iteration API of
> > >> > > > > DataStream and could provide solution for all the iteration
> > >> > use-cases.
> > >> > > > > However, maintaining two
> > >> > > > > separate implementations of iteration in both the Flink
> > repository
> > >> > and
> > >> > > > the
> > >> > > > > Flink ML repository
> > >> > > > > would introduce unnecessary complexity and make it difficult to
> > >> > maintain
> > >> > > > > the Iteration API.
> > >> > > > >
> > >> > > > > As such I propose deprecating the Iteration API of DataStream
> > and
> > >> > > > removing
> > >> > > > > it completely in the next
> > >> > > > > major version. In the future, if other modules in the Flink
> > >> > repository
> > >> > > > > require the use of the
> > >> > > > > Iteration API, we can consider extracting all Iteration
> > >> > implementations
> > >> > > > > from the Flink 

Re: Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[Jing Ge]

Hi Wencong,

Thanks for your clarification! +1

Best regards,
Jing

On Fri, Sep 1, 2023 at 12:36 PM Wencong Liu  wrote:

> Hi Jing,
>
>
> Thanks for your reply！
>
>
> > Or the "independent module extraction" mentioned in the FLIP does mean an
> independent module in Flink?
>
>
> Yes. If there are submodules in Flink repository needs the iteration
> (currently not),
> we could consider extracting them to a new submodule of Flink.
>
>
> > users will have to add one more dependency of Flink ML. If iteration is
> the
> only feature they need, it will look a little bit weird.
>
>
> If users only need to execute iteration jobs, they can simply remove the
> Flink
> dependency and add the necessary dependencies related to Flink ML.
> However,
> they can still utilize the DataStream API as it is also a dependency of
> Flink ML.
>
>
> Keeping an iteration submodule in Flink repository and make Flink ML
> depends it
> is also another solution. But the current implementation of Iteration in
> DataStream
> should be removed definitely due to its Incompleteness.
>
>
> The placement of the Iteration API in the repository is a topic that has
> multiple
> potential solutions. WDYT?
>
>
> Best,
> Wencong
>
>
>
>
>
>
>
>
>
>
>
> At 2023-09-01 17:59:34, "Jing Ge"  wrote:
> >Hi Wencong,
> >
> >Thanks for the proposal!
> >
> >"The Iteration API in DataStream is planned be deprecated in Flink 1.19
> and
> >then finally removed in Flink 2.0. For the users that rely on the
> Iteration
> >API in DataStream, they will have to migrate to Flink ML."
> >- Does it make sense to migrate the iteration module into Flink directly?
> >Or the "independent module extraction" mentioned in the FLIP does mean an
> >independent module in Flink? Since the iteration will be removed in Flink,
> >users will have to add one more dependency of Flink ML. If iteration is
> the
> >only feature they need, it will look a little bit weird.
> >
> >
> >Best regards,
> >Jing
> >
> >On Fri, Sep 1, 2023 at 11:05 AM weijie guo 
> >wrote:
> >
> >> Thanks, +1 for this.
> >>
> >> Best regards,
> >>
> >> Weijie
> >>
> >>
> >> Yangze Guo  于2023年9月1日周五 14:29写道：
> >>
> >> > +1
> >> >
> >> > Thanks for driving this.
> >> >
> >> > Best,
> >> > Yangze Guo
> >> >
> >> > On Fri, Sep 1, 2023 at 2:00 PM Xintong Song 
> >> wrote:
> >> > >
> >> > > +1
> >> > >
> >> > > Best,
> >> > >
> >> > > Xintong
> >> > >
> >> > >
> >> > >
> >> > > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin 
> wrote:
> >> > >
> >> > > > Thanks Wencong for initiating the discussion.
> >> > > >
> >> > > > +1 for the proposal.
> >> > > >
> >> > > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu  >
> >> > wrote:
> >> > > >
> >> > > > > Hi devs,
> >> > > > >
> >> > > > > I would like to start a discussion on FLIP-357: Deprecate
> Iteration
> >> > API
> >> > > > of
> >> > > > > DataStream [1].
> >> > > > >
> >> > > > > Currently, the Iteration API of DataStream is incomplete. For
> >> > instance,
> >> > > > it
> >> > > > > lacks support
> >> > > > > for iteration in sync mode and exactly once semantics.
> >> Additionally,
> >> > it
> >> > > > > does not offer the
> >> > > > > ability to set iteration termination conditions. As a result,
> it's
> >> > hard
> >> > > > > for developers to
> >> > > > > build an iteration pipeline by DataStream in the practical
> >> > applications
> >> > > > > such as machine learning.
> >> > > > >
> >> > > > > FLIP-176: Unified Iteration to Support Algorithms [2] has
> >> introduced
> >> > a
> >> > > > > unified iteration library
> >> > > > > in the Flink ML repository. This library addresses all the
> issues
> >> > present
> >> > > > > in the Iteration API of
> >> > > > > DataStream and could provide solution for all the iteration
> >> > use-cases.
> >> > > > > However, maintaining two
> >> > > > > separate implementations of iteration in both the Flink
> repository
> >> > and
> >> > > > the
> >> > > > > Flink ML repository
> >> > > > > would introduce unnecessary complexity and make it difficult to
> >> > maintain
> >> > > > > the Iteration API.
> >> > > > >
> >> > > > > As such I propose deprecating the Iteration API of DataStream
> and
> >> > > > removing
> >> > > > > it completely in the next
> >> > > > > major version. In the future, if other modules in the Flink
> >> > repository
> >> > > > > require the use of the
> >> > > > > Iteration API, we can consider extracting all Iteration
> >> > implementations
> >> > > > > from the Flink ML repository
> >> > > > > into an independent module.
> >> > > > >
> >> > > > > Looking forward to your feedback.
> >> > > > >
> >> > > > >
> >> > > > > [1]
> >> > > > >
> >> > > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream
> >> > > > > [2]
> >> > > > >
> >> > > >
> >> >
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300
> >> > > > >
> >> > > > > Best regards,
> >> > > > >
> >> > > > > Wencong Liu
> >> > > >
> >> >
> >>
>

Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[Jing Ge]

Hi Wencong,

Thanks for the proposal!

"The Iteration API in DataStream is planned be deprecated in Flink 1.19 and
then finally removed in Flink 2.0. For the users that rely on the Iteration
API in DataStream, they will have to migrate to Flink ML."
- Does it make sense to migrate the iteration module into Flink directly?
Or the "independent module extraction" mentioned in the FLIP does mean an
independent module in Flink? Since the iteration will be removed in Flink,
users will have to add one more dependency of Flink ML. If iteration is the
only feature they need, it will look a little bit weird.


Best regards,
Jing

On Fri, Sep 1, 2023 at 11:05 AM weijie guo 
wrote:

> Thanks, +1 for this.
>
> Best regards,
>
> Weijie
>
>
> Yangze Guo  于2023年9月1日周五 14:29写道：
>
> > +1
> >
> > Thanks for driving this.
> >
> > Best,
> > Yangze Guo
> >
> > On Fri, Sep 1, 2023 at 2:00 PM Xintong Song 
> wrote:
> > >
> > > +1
> > >
> > > Best,
> > >
> > > Xintong
> > >
> > >
> > >
> > > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin  wrote:
> > >
> > > > Thanks Wencong for initiating the discussion.
> > > >
> > > > +1 for the proposal.
> > > >
> > > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu 
> > wrote:
> > > >
> > > > > Hi devs,
> > > > >
> > > > > I would like to start a discussion on FLIP-357: Deprecate Iteration
> > API
> > > > of
> > > > > DataStream [1].
> > > > >
> > > > > Currently, the Iteration API of DataStream is incomplete. For
> > instance,
> > > > it
> > > > > lacks support
> > > > > for iteration in sync mode and exactly once semantics.
> Additionally,
> > it
> > > > > does not offer the
> > > > > ability to set iteration termination conditions. As a result, it's
> > hard
> > > > > for developers to
> > > > > build an iteration pipeline by DataStream in the practical
> > applications
> > > > > such as machine learning.
> > > > >
> > > > > FLIP-176: Unified Iteration to Support Algorithms [2] has
> introduced
> > a
> > > > > unified iteration library
> > > > > in the Flink ML repository. This library addresses all the issues
> > present
> > > > > in the Iteration API of
> > > > > DataStream and could provide solution for all the iteration
> > use-cases.
> > > > > However, maintaining two
> > > > > separate implementations of iteration in both the Flink repository
> > and
> > > > the
> > > > > Flink ML repository
> > > > > would introduce unnecessary complexity and make it difficult to
> > maintain
> > > > > the Iteration API.
> > > > >
> > > > > As such I propose deprecating the Iteration API of DataStream and
> > > > removing
> > > > > it completely in the next
> > > > > major version. In the future, if other modules in the Flink
> > repository
> > > > > require the use of the
> > > > > Iteration API, we can consider extracting all Iteration
> > implementations
> > > > > from the Flink ML repository
> > > > > into an independent module.
> > > > >
> > > > > Looking forward to your feedback.
> > > > >
> > > > >
> > > > > [1]
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream
> > > > > [2]
> > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Wencong Liu
> > > >
> >
>

Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[weijie guo]

Thanks, +1 for this.

Best regards,

Weijie


Yangze Guo  于2023年9月1日周五 14:29写道：

> +1
>
> Thanks for driving this.
>
> Best,
> Yangze Guo
>
> On Fri, Sep 1, 2023 at 2:00 PM Xintong Song  wrote:
> >
> > +1
> >
> > Best,
> >
> > Xintong
> >
> >
> >
> > On Fri, Sep 1, 2023 at 1:11 PM Dong Lin  wrote:
> >
> > > Thanks Wencong for initiating the discussion.
> > >
> > > +1 for the proposal.
> > >
> > > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu 
> wrote:
> > >
> > > > Hi devs,
> > > >
> > > > I would like to start a discussion on FLIP-357: Deprecate Iteration
> API
> > > of
> > > > DataStream [1].
> > > >
> > > > Currently, the Iteration API of DataStream is incomplete. For
> instance,
> > > it
> > > > lacks support
> > > > for iteration in sync mode and exactly once semantics. Additionally,
> it
> > > > does not offer the
> > > > ability to set iteration termination conditions. As a result, it's
> hard
> > > > for developers to
> > > > build an iteration pipeline by DataStream in the practical
> applications
> > > > such as machine learning.
> > > >
> > > > FLIP-176: Unified Iteration to Support Algorithms [2] has introduced
> a
> > > > unified iteration library
> > > > in the Flink ML repository. This library addresses all the issues
> present
> > > > in the Iteration API of
> > > > DataStream and could provide solution for all the iteration
> use-cases.
> > > > However, maintaining two
> > > > separate implementations of iteration in both the Flink repository
> and
> > > the
> > > > Flink ML repository
> > > > would introduce unnecessary complexity and make it difficult to
> maintain
> > > > the Iteration API.
> > > >
> > > > As such I propose deprecating the Iteration API of DataStream and
> > > removing
> > > > it completely in the next
> > > > major version. In the future, if other modules in the Flink
> repository
> > > > require the use of the
> > > > Iteration API, we can consider extracting all Iteration
> implementations
> > > > from the Flink ML repository
> > > > into an independent module.
> > > >
> > > > Looking forward to your feedback.
> > > >
> > > >
> > > > [1]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream
> > > > [2]
> > > >
> > >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300
> > > >
> > > > Best regards,
> > > >
> > > > Wencong Liu
> > >
>

Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[Yangze Guo]

+1

Thanks for driving this.

Best,
Yangze Guo

On Fri, Sep 1, 2023 at 2:00 PM Xintong Song  wrote:
>
> +1
>
> Best,
>
> Xintong
>
>
>
> On Fri, Sep 1, 2023 at 1:11 PM Dong Lin  wrote:
>
> > Thanks Wencong for initiating the discussion.
> >
> > +1 for the proposal.
> >
> > On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu  wrote:
> >
> > > Hi devs,
> > >
> > > I would like to start a discussion on FLIP-357: Deprecate Iteration API
> > of
> > > DataStream [1].
> > >
> > > Currently, the Iteration API of DataStream is incomplete. For instance,
> > it
> > > lacks support
> > > for iteration in sync mode and exactly once semantics. Additionally, it
> > > does not offer the
> > > ability to set iteration termination conditions. As a result, it's hard
> > > for developers to
> > > build an iteration pipeline by DataStream in the practical applications
> > > such as machine learning.
> > >
> > > FLIP-176: Unified Iteration to Support Algorithms [2] has introduced a
> > > unified iteration library
> > > in the Flink ML repository. This library addresses all the issues present
> > > in the Iteration API of
> > > DataStream and could provide solution for all the iteration use-cases.
> > > However, maintaining two
> > > separate implementations of iteration in both the Flink repository and
> > the
> > > Flink ML repository
> > > would introduce unnecessary complexity and make it difficult to maintain
> > > the Iteration API.
> > >
> > > As such I propose deprecating the Iteration API of DataStream and
> > removing
> > > it completely in the next
> > > major version. In the future, if other modules in the Flink repository
> > > require the use of the
> > > Iteration API, we can consider extracting all Iteration implementations
> > > from the Flink ML repository
> > > into an independent module.
> > >
> > > Looking forward to your feedback.
> > >
> > >
> > > [1]
> > >
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream
> > > [2]
> > >
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300
> > >
> > > Best regards,
> > >
> > > Wencong Liu
> >

Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[Xintong Song]

+1

Best,

Xintong



On Fri, Sep 1, 2023 at 1:11 PM Dong Lin  wrote:

> Thanks Wencong for initiating the discussion.
>
> +1 for the proposal.
>
> On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu  wrote:
>
> > Hi devs,
> >
> > I would like to start a discussion on FLIP-357: Deprecate Iteration API
> of
> > DataStream [1].
> >
> > Currently, the Iteration API of DataStream is incomplete. For instance,
> it
> > lacks support
> > for iteration in sync mode and exactly once semantics. Additionally, it
> > does not offer the
> > ability to set iteration termination conditions. As a result, it's hard
> > for developers to
> > build an iteration pipeline by DataStream in the practical applications
> > such as machine learning.
> >
> > FLIP-176: Unified Iteration to Support Algorithms [2] has introduced a
> > unified iteration library
> > in the Flink ML repository. This library addresses all the issues present
> > in the Iteration API of
> > DataStream and could provide solution for all the iteration use-cases.
> > However, maintaining two
> > separate implementations of iteration in both the Flink repository and
> the
> > Flink ML repository
> > would introduce unnecessary complexity and make it difficult to maintain
> > the Iteration API.
> >
> > As such I propose deprecating the Iteration API of DataStream and
> removing
> > it completely in the next
> > major version. In the future, if other modules in the Flink repository
> > require the use of the
> > Iteration API, we can consider extracting all Iteration implementations
> > from the Flink ML repository
> > into an independent module.
> >
> > Looking forward to your feedback.
> >
> >
> > [1]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream
> > [2]
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300
> >
> > Best regards,
> >
> > Wencong Liu
>

Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

[Dong Lin]

Thanks Wencong for initiating the discussion.

+1 for the proposal.

On Fri, Sep 1, 2023 at 12:00 PM Wencong Liu  wrote:

> Hi devs,
>
> I would like to start a discussion on FLIP-357: Deprecate Iteration API of
> DataStream [1].
>
> Currently, the Iteration API of DataStream is incomplete. For instance, it
> lacks support
> for iteration in sync mode and exactly once semantics. Additionally, it
> does not offer the
> ability to set iteration termination conditions. As a result, it's hard
> for developers to
> build an iteration pipeline by DataStream in the practical applications
> such as machine learning.
>
> FLIP-176: Unified Iteration to Support Algorithms [2] has introduced a
> unified iteration library
> in the Flink ML repository. This library addresses all the issues present
> in the Iteration API of
> DataStream and could provide solution for all the iteration use-cases.
> However, maintaining two
> separate implementations of iteration in both the Flink repository and the
> Flink ML repository
> would introduce unnecessary complexity and make it difficult to maintain
> the Iteration API.
>
> As such I propose deprecating the Iteration API of DataStream and removing
> it completely in the next
> major version. In the future, if other modules in the Flink repository
> require the use of the
> Iteration API, we can consider extracting all Iteration implementations
> from the Flink ML repository
> into an independent module.
>
> Looking forward to your feedback.
>
>
> [1]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-357%3A+Deprecate+Iteration+API+of+DataStream
> [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=184615300
>
> Best regards,
>
> Wencong Liu