In general, I'll give a big +1 to deprecating the SourceFunction. That said, it is indeed worth looking into what might be missing or less easy to implement with FLIP-27 Source compared with the SourceFunction. Maybe we can just compile a list of things to do in order to fully deprecate the SourceFunction. As far as I am aware of, there are two things that need to be taken care of：
1. A simple high level API, as Jing mentioned, that makes simple cases that do not involve the split enumerator easier. Ideally this should be as simple as SourceFunction, if not simpler. Off the top of my head, I think a default no-op split enumerator will just do the work. And the data generator of FLIP-238 could be an implementation using this high level API. 2. FLIP-208, which allows users to stop the job upon receiving a record in the stream. Is there anything else that we have heard from the users / connector developers that needs some attention? Thanks, Jiangjie (Becket) Qin On Fri, Jun 10, 2022 at 3:25 PM David Anderson <dander...@apache.org> wrote: > +1 for deprecating SourceFunction from me as well. And a big THANK YOU to > Alex Fedulov for bringing forward FLIP-238. > > David > > On Fri, Jun 10, 2022 at 4:03 AM Lijie Wang <wangdachui9...@gmail.com> > wrote: > > > Hi all, > > > > Sorry for my mistake. The `StreamExecutionEnvironment#readFiles` and can > be > > easily replaced by `FileSource#forRecordStreamFormat/forBulkFileFormat`. > I > > have no other concerns. > > > > +1 to deprecate SourceFunction and deprecate the methods (in > > StreamExecutionEnvironment) based on SourceFunction . > > > > Best, > > Lijie > > > > Konstantin Knauf <kna...@apache.org> 于2022年6月10日周五 05:11写道： > > > > > Hi everyone, > > > > > > thank you Jing for redirecting the discussion back to the topic at > hand. > > I > > > agree with all of your points. > > > > > > +1 to deprecate SourceFunction > > > > > > Is there really no replacement for the > > StreamExecutionEnvironment#readXXX. > > > There is already a FLIP-27 based FileSource, right? What's missing to > > > recommend using that as opposed to the the readXXX methods? > > > > > > Cheers, > > > > > > Konstantin > > > > > > Am Do., 9. Juni 2022 um 20:11 Uhr schrieb Alexander Fedulov < > > > alexan...@ververica.com>: > > > > > > > Hi all, > > > > > > > > It seems that there is some understandable cautiousness with regard > to > > > > deprecating methods and subclasses that do not have alternatives just > > > yet. > > > > > > > > We should probably first agree if it is in general OK for Flink to > use > > > > @Deprecated > > > > annotation for parts of the code that do not have alternatives. In > that > > > > case, > > > > we could add a comment along the lines of: > > > > "This implementation is based on a deprecated SourceFunction API that > > > > will gradually be phased out from Flink. No direct substitute exists > at > > > the > > > > moment. > > > > If you want to have a more future-proof solution, consider helping > the > > > > project by > > > > contributing an implementation based on the new Source API." > > > > > > > > This should clearly communicate the message that usage of these > > > > methods/classes > > > > is discouraged and at the same time promote contributions for > > addressing > > > > the gap. > > > > What do you think? > > > > > > > > Best, > > > > Alexander Fedulov > > > > > > > > > > > > On Thu, Jun 9, 2022 at 6:27 PM Ingo Bürk <airbla...@apache.org> > wrote: > > > > > > > > > Hi, > > > > > > > > > > these APIs don't expose the underlying source directly, so I don't > > > think > > > > > we need to worry about deprecating them as well. There's also > nothing > > > > > inherently wrong with using a deprecated API internally, though > even > > > > > just for the experience of using our own new APIs I would > personally > > > say > > > > > that they should be migrated to the new Source API. It's hard to > > reason > > > > > that users must migrate to a new API if we don't do it internally > as > > > > well. > > > > > > > > > > > > > > > Best > > > > > Ingo > > > > > > > > > > On 09.06.22 15:41, Lijie Wang wrote: > > > > > > Hi Martijn, > > > > > > > > > > > > I don't mean it's a blocker. Just a information. And I'm also +1 > > for > > > > > this. > > > > > > > > > > > > Put it another way: should we migrate the `#readFile(...)` to new > > API > > > > or > > > > > > provide a similar method "readxxx“ based on the new Source API? > > > > > > > > > > > > And if we don't migrate it, does it mean that the > `#readFile(...)` > > > > should > > > > > > also be marked as deprecated? > > > > > > > > > > > > Best, > > > > > > Lijie > > > > > > > > > > > > Martijn Visser <martijnvis...@apache.org> 于2022年6月9日周四 21:03写道： > > > > > > > > > > > >> Hi Lijie, > > > > > >> > > > > > >> I don't see any problem with deprecating those methods at this > > > moment, > > > > > as > > > > > >> long as we don't remove them until the replacements are > available. > > > > > Besides > > > > > >> that, are we sure there are no replacements already, especially > > with > > > > the > > > > > >> new FileSource? > > > > > >> > > > > > >> Best regards, > > > > > >> > > > > > >> Martijn > > > > > >> > > > > > >> Op do 9 jun. 2022 om 14:23 schreef Lijie Wang < > > > > wangdachui9...@gmail.com > > > > > >: > > > > > >> > > > > > >>> Hi all, > > > > > >>> > > > > > >>> FYI, currently, some commonly used methods in > > > > > StreamExecutionEnvironment > > > > > >>> are still based on the old SourceFunction (and there is no > > > > > alternative): > > > > > >>> `StreamExecutionEnvironment#readFile(...)` > > > > > >>> `StreamExecutionEnvironment#readTextFile(...)` > > > > > >>> > > > > > >>> I think these should be migrated to the new source API before > > > > deprecate > > > > > >> the > > > > > >>> SourceFunction. > > > > > >>> > > > > > >>> Best, > > > > > >>> Lijie > > > > > >>> > > > > > >>> Martijn Visser <martijnvis...@apache.org> 于2022年6月9日周四 > 16:05写道： > > > > > >>> > > > > > >>>> Hi all, > > > > > >>>> > > > > > >>>> I think implicitly we've already considered the SourceFunction > > and > > > > > >>>> SinkFunction as deprecated. They are even marked as so on the > > > Flink > > > > > >>> roadmap > > > > > >>>> [1]. That also shows that connectors that are using these > > > interfaces > > > > > >> are > > > > > >>>> either approaching end-of-life. The fact that we're actively > > > > migrating > > > > > >>>> connectors from Source/SinkFunction to FLIP-27/FLIP-143 (plus > > > add-on > > > > > >>> FLIPs) > > > > > >>>> shows that we've already determined that target. > > > > > >>>> > > > > > >>>> With regards to the motivation of FLIP-27, I think reading up > on > > > the > > > > > >>>> original discussion thread is also worthwhile [2] to see more > > > > context. > > > > > >>>> FLIP-27 was also very important as it brought a unified > > connector > > > > > which > > > > > >>> can > > > > > >>>> support both streaming and batch (with batch being considered > a > > > > > special > > > > > >>>> case of streaming in Flink's vision). > > > > > >>>> > > > > > >>>> So +1 to deprecate SourceFunction. I would also argue that we > > > should > > > > > >>>> already mark the SinkFunction as deprecated to avoid having > this > > > > > >>> discussion > > > > > >>>> again in a couple of months. > > > > > >>>> > > > > > >>>> Best regards, > > > > > >>>> > > > > > >>>> Martijn > > > > > >>>> > > > > > >>>> [1] https://flink.apache.org/roadmap.html > > > > > >>>> [2] > > > > https://lists.apache.org/thread/334co89dbhc8qpr9nvmz8t1gp4sz2c8y > > > > > >>>> > > > > > >>>> Op do 9 jun. 2022 om 09:48 schreef Jing Ge < > j...@ververica.com > > >: > > > > > >>>> > > > > > >>>>> Hi, > > > > > >>>>> > > > > > >>>>> I am very happy to see opinions from different perspectives. > > That > > > > > >> will > > > > > >>>> help > > > > > >>>>> us understand the problem better. Thanks all for the > > informative > > > > > >>>>> discussion. > > > > > >>>>> > > > > > >>>>> Let's see the big picture and check following facts together: > > > > > >>>>> > > > > > >>>>> 1. FLIP-27 was intended to solve some technical issues that > are > > > > very > > > > > >>>>> difficult to solve with SourceFunction[1]. When we say > > > > > >> "SourceFunction > > > > > >>> is > > > > > >>>>> easy", well, it depends. If we take a look at the > > implementation > > > of > > > > > >> the > > > > > >>>>> Kafka connector, we will know how complicated it is to build > a > > > > > >> serious > > > > > >>>>> connector for production with the old SourceFunction. To > every > > > > > >> problem > > > > > >>>>> there is a solution and to every solution there is a problem. > > The > > > > > >> fact > > > > > >>> is > > > > > >>>>> that there is no perfect but a feasible solution. If we try > to > > > > solve > > > > > >>>>> complicated problems, we have to expose some complexity. > > > Comparing > > > > to > > > > > >>>>> connectors for POC, demo, training(no offense), I would also > > > solve > > > > > >>> issues > > > > > >>>>> for connectors like Kafka connector that are widely used in > > > > > >> production > > > > > >>>> with > > > > > >>>>> higher priority. I think that should be one reason why > FLIP-27 > > > has > > > > > >> been > > > > > >>>>> designed and why the new source API went public. > > > > > >>>>> > > > > > >>>>> 2. FLIP-27 and the implementation was introduced roughly at > the > > > end > > > > > >> of > > > > > >>>> 2019 > > > > > >>>>> and went public on 19.04.2021, which means Flink has provided > > two > > > > > >>>> different > > > > > >>>>> public/graduated source solutions for more than one year. On > > the > > > > day > > > > > >>> that > > > > > >>>>> the new source API went public, there should be a consensus > in > > > the > > > > > >>>>> community that we should start the migration. Old > > SourceFunction > > > > > >>>> interface, > > > > > >>>>> in the ideal case, should have been deprecated on that day, > > > > otherwise > > > > > >>> we > > > > > >>>>> should not graduate the new source API to avoid confusing > > > > (connector) > > > > > >>>>> developers[2]. > > > > > >>>>> > > > > > >>>>> 3. It is true that the new source API is hard to understand > and > > > > even > > > > > >>> hard > > > > > >>>>> to implement for simple cases. Thanks for the feedback. That > is > > > > > >>> something > > > > > >>>>> we need to improve. The current design&implementation could > be > > > > > >>> considered > > > > > >>>>> as the low level API. The next step is to create the high > level > > > API > > > > > >> to > > > > > >>>>> reduce some unnecessary complexity for those simple cases. > But, > > > > IMHO, > > > > > >>>> this > > > > > >>>>> should not be the prerequisite to postpone the deprecation of > > the > > > > old > > > > > >>>>> SourceFunction APIs. > > > > > >>>>> > > > > > >>>>> 4. As long as the old SourceFunction is not marked as > > deprecated, > > > > > >>>>> developers will continue asking which one should be used. > Let's > > > > make > > > > > >> a > > > > > >>>>> concrete example. If a new connector is developed now and the > > > > > >> developer > > > > > >>>>> asks for a suggestion of the choice between the old and new > > > source > > > > > >> API > > > > > >>> on > > > > > >>>>> the ML, which one should we suggest? I think it should be the > > new > > > > > >>> Source > > > > > >>>>> API. If a fresh new connector has been developed with the old > > > > > >>>>> SourceFunction API before asking for the consensus in the > > > community > > > > > >> and > > > > > >>>> the > > > > > >>>>> developer wants to merge it to the master. Should we allow > it? > > If > > > > the > > > > > >>>>> answer of all these questions is pointing to the new Source > > API, > > > > the > > > > > >>> old > > > > > >>>>> SourceFunction is de facto already deprecated, just has not > > been > > > > > >> marked > > > > > >>>> as > > > > > >>>>> @deprecated, which confuses developers even more. > > > > > >>>>> > > > > > >>>>> Best regards, > > > > > >>>>> Jing > > > > > >>>>> > > > > > >>>>> [1] > > > > > >>>>> > > > > > >>>>> > > > > > >>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface > > > > > >>>>> [2] > > > > https://lists.apache.org/thread/7okp4y46n3o3rx5mn0t3qobrof8zxwqs > > > > > >>>>> > > > > > >>>>> On Wed, Jun 8, 2022 at 2:21 AM Alexander Fedulov < > > > > > >>>> alexan...@ververica.com> > > > > > >>>>> wrote: > > > > > >>>>> > > > > > >>>>>> Hey Austin, > > > > > >>>>>> > > > > > >>>>>> Since we are getting deeper into the implementation details > of > > > the > > > > > >>>>>> DataGeneratorSource > > > > > >>>>>> and it is not the main topic of this thread, I propose to > move > > > our > > > > > >>>>>> discussion to where it belongs: [DISCUSS] FLIP-238 [1]. > Could > > > you > > > > > >>>> please > > > > > >>>>>> briefly formulate your requirements to make it easier for > the > > > > > >> others > > > > > >>> to > > > > > >>>>>> follow? I am happy to continue this conversation there. > > > > > >>>>>> > > > > > >>>>>> [1] > > > > > >> > https://lists.apache.org/thread/7gjxto1rmkpff4kl54j8nlg5db2rqhkt > > > > > >>>>>> > > > > > >>>>>> Best, > > > > > >>>>>> Alexander Fedulov > > > > > >>>>>> > > > > > >>>>>> On Tue, Jun 7, 2022 at 6:14 PM Austin Cawley-Edwards < > > > > > >>>>>> austin.caw...@gmail.com> wrote: > > > > > >>>>>> > > > > > >>>>>>>> @Austin, in the FLIP I mentioned above [1], the user is > > > > > >> expected > > > > > >>> to > > > > > >>>>>>> pass a MapFunction<Long, > > > > > >>>>>>> OUT> > > > > > >>>>>>> to the generator. I wonder if you could have your external > > > client > > > > > >>> and > > > > > >>>>>>> polling logic wrapped in a custom > > > > > >>>>>>> MapFunction implementation class? Would that answer your > > needs > > > or > > > > > >>> do > > > > > >>>>> you > > > > > >>>>>>> have some > > > > > >>>>>>> more sophisticated scenario in mind? > > > > > >>>>>>> > > > > > >>>>>>> At first glance, the FLIP looks good but for this case in > > > regards > > > > > >>> to > > > > > >>>>> the > > > > > >>>>>>> map function, but leaves out 1) ability to control polling > > > > > >>> intervals > > > > > >>>>> and > > > > > >>>>>> 2) > > > > > >>>>>>> ability to produce an unknown number of records, both > > per-poll > > > > > >> and > > > > > >>>>>> overall > > > > > >>>>>>> boundedness. Do you think something like this could be > built > > > from > > > > > >>> the > > > > > >>>>>> same > > > > > >>>>>>> pieces? > > > > > >>>>>>> I'm also wondering what handles threading, is that on the > > user > > > or > > > > > >>> is > > > > > >>>>> that > > > > > >>>>>>> part of the DataGeneratorSource? > > > > > >>>>>>> > > > > > >>>>>>> Best, > > > > > >>>>>>> Austin > > > > > >>>>>>> > > > > > >>>>>>> On Tue, Jun 7, 2022 at 9:34 AM Alexander Fedulov < > > > > > >>>>>> alexan...@ververica.com> > > > > > >>>>>>> wrote: > > > > > >>>>>>> > > > > > >>>>>>>> Hi everyone, > > > > > >>>>>>>> > > > > > >>>>>>>> Thanks for all the input and a lively discussion. It seems > > > that > > > > > >>>> there > > > > > >>>>>> is > > > > > >>>>>>> a > > > > > >>>>>>>> consensus that due to > > > > > >>>>>>>> the inherent complexity of FLIP-27 sources we should > provide > > > > > >> more > > > > > >>>>>>>> user-facing utilities to bridge > > > > > >>>>>>>> the gap between the existing SourceFunction-based > > > functionality > > > > > >>> and > > > > > >>>>> the > > > > > >>>>>>> new > > > > > >>>>>>>> APIs. > > > > > >>>>>>>> > > > > > >>>>>>>> To start addressing this I picked the issue that David > > raised > > > > > >> and > > > > > >>>>> many > > > > > >>>>>>>> upvoted. Here is a proposal > > > > > >>>>>>>> for the new DataGeneratorSource: FLIP-238 [1]. Please > take > > a > > > > > >>>> look, I > > > > > >>>>>> am > > > > > >>>>>>>> going to open a separate > > > > > >>>>>>>> discussion thread on it shortly. > > > > > >>>>>>>> > > > > > >>>>>>>> Jing also raised some great points regarding the > interfaces > > > and > > > > > >>>>>>> subclasses. > > > > > >>>>>>>> It seems to me that > > > > > >>>>>>>> what might actually help is some sort of a "soft > > deprecation" > > > > > >>>> concept > > > > > >>>>>> and > > > > > >>>>>>>> annotation. It could be > > > > > >>>>>>>> used in places where we do not have an alternative > > > > > >> implementation > > > > > >>>>> yet, > > > > > >>>>>>> but > > > > > >>>>>>>> we clearly want > > > > > >>>>>>>> to indicate that continuing to build on top of these > > > interfaces > > > > > >>> is > > > > > >>>>>>>> discouraged. The area of > > > > > >>>>>>>> impact of deprecating all SourceFunction subclasses is > > rather > > > > > >>> big, > > > > > >>>>> and > > > > > >>>>>> we > > > > > >>>>>>>> can expect it to > > > > > >>>>>>>> take a while. The hope would be that if in the meantime > > > someone > > > > > >>>> finds > > > > > >>>>>>>> themselves using one of > > > > > >>>>>>>> such old APIs, the "soft deprecation" annotation will be a > > > > > >> clear > > > > > >>>>>>> indication > > > > > >>>>>>>> and encouragement to > > > > > >>>>>>>> work on introducing an alternative FLIP-27-based > > > implementation > > > > > >>>>>> instead. > > > > > >>>>>>>> > > > > > >>>>>>>> @Austin, in the FLIP I mentioned above [1], the user is > > > > > >> expected > > > > > >>> to > > > > > >>>>>>>> pass a MapFunction<Long, > > > > > >>>>>>>> OUT> > > > > > >>>>>>>> to the generator. I wonder if you could have your external > > > > > >> client > > > > > >>>> and > > > > > >>>>>>>> polling logic wrapped in a custom > > > > > >>>>>>>> MapFunction implementation class? Would that answer your > > needs > > > > > >> or > > > > > >>>> do > > > > > >>>>>> you > > > > > >>>>>>>> have some > > > > > >>>>>>>> more sophisticated scenario in mind? > > > > > >>>>>>>> > > > > > >>>>>>>> [1] https://cwiki.apache.org/confluence/x/9Av1D > > > > > >>>>>>>> Best, > > > > > >>>>>>>> Alexander Fedulov > > > > > >>>>>>>> > > > > > >>>>>>>> On Mon, Jun 6, 2022 at 7:08 PM Austin Cawley-Edwards < > > > > > >>>>>>>> austin.caw...@gmail.com> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>>> Thanks for the nice discussion all. > > > > > >>>>>>>>> > > > > > >>>>>>>>> I was recently trying to implement a very simple polling > > > > > >> source > > > > > >>>> and > > > > > >>>>>>>>> would've loved a higher-level base to work from. I'm > > > > > >> wondering > > > > > >>> if > > > > > >>>>> in > > > > > >>>>>>>>> addition to the data generator use cases, it would be > good > > to > > > > > >>>>>> support a > > > > > >>>>>>>>> simple non-parallel polling abstraction to make it easier > > to, > > > > > >>> for > > > > > >>>>>>>> instance, > > > > > >>>>>>>>> start prototyping with data in existing APIs without > > adding a > > > > > >>>> Kafka > > > > > >>>>>> or > > > > > >>>>>>>> such > > > > > >>>>>>>>> in the middle. > > > > > >>>>>>>>> > > > > > >>>>>>>>> Best, > > > > > >>>>>>>>> Austin > > > > > >>>>>>>>> > > > > > >>>>>>>>> On Mon, Jun 6, 2022 at 10:02 AM tison < > > wander4...@gmail.com> > > > > > >>>>> wrote: > > > > > >>>>>>>>> > > > > > >>>>>>>>>> Well. It's a bit off-topic. For deprecating > SourceFunction > > > > > >> as > > > > > >>>>>> FLIP-27 > > > > > >>>>>>>>>> series works go ahead, +1 from my side. It's a > significant > > > > > >>> work > > > > > >>>>>>> towards > > > > > >>>>>>>>> the > > > > > >>>>>>>>>> unification of batch and streaming effort :) > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> Best, > > > > > >>>>>>>>>> tison. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> tison <wander4...@gmail.com> 于2022年6月6日周一 21:54写道： > > > > > >>>>>>>>>> > > > > > >>>>>>>>>>> The starting point of the version bump and removal > > > > > >> question > > > > > >>>> is > > > > > >>>>>> that > > > > > >>>>>>>>>>> downstream projects may experience a tough time to > adapt > > > > > >>> new > > > > > >>>>>>>> interfaces > > > > > >>>>>>>>>>> while Flink keeps in 1.x versions so that users may > > > > > >> expect > > > > > >>> it > > > > > >>>>> as > > > > > >>>>>> an > > > > > >>>>>>>>> easy > > > > > >>>>>>>>>>> task. From my experience, it's really challenge to > > > > > >> maintain > > > > > >>>>>>>>>>> compatibility between multiple versions of Flink while > > > > > >>>>>> significant > > > > > >>>>>>>>>> changes > > > > > >>>>>>>>>>> made but sharing 1.x version series - users may not be > > > > > >>> aware > > > > > >>>>> that > > > > > >>>>>>>> it's > > > > > >>>>>>>>>>> almost a major version bump. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Best, > > > > > >>>>>>>>>>> tison. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> tison <wander4...@gmail.com> 于2022年6月6日周一 21:51写道： > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>> One question from my side: > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> As SourceFunction a @Public interface, we cannot > remove > > > > > >> it > > > > > >>>>>> before > > > > > >>>>>>>>> doing > > > > > >>>>>>>>>>>> a major version bump (Flink 2.0). > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Of course it's not a blocker to make such deprecation > > > > > >> and > > > > > >>>> let > > > > > >>>>>> the > > > > > >>>>>>>> new > > > > > >>>>>>>>>>>> interface step in. My question is whether we have a > plan > > > > > >>> to > > > > > >>>>>>> finally > > > > > >>>>>>>>>> remove > > > > > >>>>>>>>>>>> the deprecated interfaces, or postpone it until a > clear > > > > > >>> plan > > > > > >>>>> of > > > > > >>>>>>>> Flink > > > > > >>>>>>>>>> 2.0? > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>> tison. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> David Anderson <dander...@apache.org> 于2022年6月6日周一 > > > > > >>> 21:35写道： > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> David, can you elaborate why you need watermark > > > > > >>>> generation > > > > > >>>>> in > > > > > >>>>>>> the > > > > > >>>>>>>>>>>>> source > > > > > >>>>>>>>>>>>>> for your data generators? > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> The training exercises should strive to provide > > > > > >> examples > > > > > >>> of > > > > > >>>>>> best > > > > > >>>>>>>>>>>>> practices. > > > > > >>>>>>>>>>>>> If the exercises and their solutions use > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> env.fromSource(source, > > > > > >> WatermarkStrategy.noWatermarks(), > > > > > >>>>>>>>>>>>> "name-of-source") > > > > > >>>>>>>>>>>>> .map(...) > > > > > >>>>>>>>>>>>> .assignTimestampsAndWatermarks(...) > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> this will help establish this anti-pattern as the > > > > > >> normal > > > > > >>>> way > > > > > >>>>> of > > > > > >>>>>>>> doing > > > > > >>>>>>>>>>>>> things. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Most new Flink users are using a KafkaSource with a > > > > > >>>>>> noWatermarks > > > > > >>>>>>>>>> strategy > > > > > >>>>>>>>>>>>> and a SimpleStringSchema, followed by a map that does > > > > > >> the > > > > > >>>>> real > > > > > >>>>>>>>>>>>> deserialization, followed by the real watermarking -- > > > > > >>>> because > > > > > >>>>>>> they > > > > > >>>>>>>>>> aren't > > > > > >>>>>>>>>>>>> seeing examples that teach how these interfaces are > > > > > >> meant > > > > > >>>> to > > > > > >>>>> be > > > > > >>>>>>>> used. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> When we redo the sources used in training exercises, > I > > > > > >>> want > > > > > >>>>> to > > > > > >>>>>>>> avoid > > > > > >>>>>>>>>>>>> these > > > > > >>>>>>>>>>>>> pitfalls. > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> David > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Mon, Jun 6, 2022 at 9:12 AM Konstantin Knauf < > > > > > >>>>>>> kna...@apache.org > > > > > >>>>>>>>> > > > > > >>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Hi everyone, > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> very interesting thread. The proposal for > deprecation > > > > > >>>> seems > > > > > >>>>>> to > > > > > >>>>>>>> have > > > > > >>>>>>>>>>>>> sparked > > > > > >>>>>>>>>>>>>> a very important discussion. Do we what users > > > > > >> struggle > > > > > >>>> with > > > > > >>>>>>>>>>>>> specifically? > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Speaking for myself, when I upgrade flink-faker to > > > > > >> the > > > > > >>>> new > > > > > >>>>>>> Source > > > > > >>>>>>>>> API > > > > > >>>>>>>>>>>>> an > > > > > >>>>>>>>>>>>>> unbounded version of the NumberSequenceSource would > > > > > >>> have > > > > > >>>>> been > > > > > >>>>>>>> all I > > > > > >>>>>>>>>>>>> needed, > > > > > >>>>>>>>>>>>>> but that's just the data generator use case. I > think, > > > > > >>>> that > > > > > >>>>>> one > > > > > >>>>>>>>> could > > > > > >>>>>>>>>> be > > > > > >>>>>>>>>>>>>> solved quite easily. David, can you elaborate why > you > > > > > >>>> need > > > > > >>>>>>>>> watermark > > > > > >>>>>>>>>>>>>> generation in the source for your data generators? > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Cheers, > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Konstantin > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Am So., 5. Juni 2022 um 17:48 Uhr schrieb Piotr > > > > > >>> Nowojski > > > > > >>>> < > > > > > >>>>>>>>>>>>>> pnowoj...@apache.org>: > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Also +1 to what David has written. But it doesn't > > > > > >>> mean > > > > > >>>> we > > > > > >>>>>>>> should > > > > > >>>>>>>>> be > > > > > >>>>>>>>>>>>>> waiting > > > > > >>>>>>>>>>>>>>> indefinitely to deprecate SourceFunction. > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>>>>> Piotrek > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> niedz., 5 cze 2022 o 16:46 Jark Wu < > > > > > >> imj...@gmail.com > > > > > >>>> > > > > > >>>>>>>>> napisał(a): > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> +1 to David's point. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Usually, when we deprecate some interfaces, we > > > > > >>> should > > > > > >>>>>> point > > > > > >>>>>>>>> users > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>> use > > > > > >>>>>>>>>>>>>>>> the recommended alternatives. > > > > > >>>>>>>>>>>>>>>> However, implementing the new Source interface > > > > > >> for > > > > > >>>> some > > > > > >>>>>>>> simple > > > > > >>>>>>>>>>>>>> scenarios > > > > > >>>>>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>>>> too challenging and complex. > > > > > >>>>>>>>>>>>>>>> We also found it isn't easy to push the internal > > > > > >>>>>> connector > > > > > >>>>>>> to > > > > > >>>>>>>>>>>>> upgrade > > > > > >>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>> the new Source because > > > > > >>>>>>>>>>>>>>>> "FLIP-27 are hard to understand, while > > > > > >>> SourceFunction > > > > > >>>>> is > > > > > >>>>>>>> easy". > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> +1 to make implementing a simple Source easier > > > > > >>> before > > > > > >>>>>>>>> deprecating > > > > > >>>>>>>>>>>>>>>> SourceFunction. > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>>>>>> Jark > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> On Sun, 5 Jun 2022 at 07:29, Jingsong Lee < > > > > > >>>>>>>>>> lzljs3620...@apache.org > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> +1 to David and Ingo. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Before deprecate and remove SourceFunction, we > > > > > >>>> should > > > > > >>>>>>> have > > > > > >>>>>>>>> some > > > > > >>>>>>>>>>>>>> easier > > > > > >>>>>>>>>>>>>>>> APIs > > > > > >>>>>>>>>>>>>>>>> to wrap new Source, the cost to write a new > > > > > >>> Source > > > > > >>>> is > > > > > >>>>>> too > > > > > >>>>>>>>> high > > > > > >>>>>>>>>>>>> now. > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> Ingo Bürk <airbla...@apache.org>于2022年6月5日 > > > > > >>>>> 周日05:32写道： > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> I +1 everything David said. The new Source > > > > > >> API > > > > > >>>>> raised > > > > > >>>>>>> the > > > > > >>>>>>>>>>>>>> complexity > > > > > >>>>>>>>>>>>>>>>>> significantly. It's great to have such a > > > > > >> rich, > > > > > >>>>>> powerful > > > > > >>>>>>>> API > > > > > >>>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>>>> do > > > > > >>>>>>>>>>>>>>>>>> everything, but in the process we lost the > > > > > >>>> ability > > > > > >>>>> to > > > > > >>>>>>>>> onboard > > > > > >>>>>>>>>>>>>> people > > > > > >>>>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>>> the APIs. > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> Best > > > > > >>>>>>>>>>>>>>>>>> Ingo > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> On 04.06.22 21:21, David Anderson wrote: > > > > > >>>>>>>>>>>>>>>>>>> I'm in favor of this, but I think we need > > > > > >> to > > > > > >>>> make > > > > > >>>>>> it > > > > > >>>>>>>>> easier > > > > > >>>>>>>>>>>>> to > > > > > >>>>>>>>>>>>>>>>> implement > > > > > >>>>>>>>>>>>>>>>>>> data generators and test sources. As things > > > > > >>>> stand > > > > > >>>>>> in > > > > > >>>>>>>>> 1.15, > > > > > >>>>>>>>>>>>> unless > > > > > >>>>>>>>>>>>>>> you > > > > > >>>>>>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>>>>>>>> be satisfied with using a > > > > > >>> NumberSequenceSource > > > > > >>>>>>> followed > > > > > >>>>>>>>> by > > > > > >>>>>>>>>> a > > > > > >>>>>>>>>>>>> map, > > > > > >>>>>>>>>>>>>>>>> things > > > > > >>>>>>>>>>>>>>>>>>> get quite complicated. I looked into > > > > > >>> reworking > > > > > >>>>> the > > > > > >>>>>>> data > > > > > >>>>>>>>>>>>>> generators > > > > > >>>>>>>>>>>>>>>> used > > > > > >>>>>>>>>>>>>>>>>> in > > > > > >>>>>>>>>>>>>>>>>>> the training exercises, and got discouraged > > > > > >>> by > > > > > >>>>> the > > > > > >>>>>>>> amount > > > > > >>>>>>>>>> of > > > > > >>>>>>>>>>>>> work > > > > > >>>>>>>>>>>>>>>>>> involved. > > > > > >>>>>>>>>>>>>>>>>>> (The sources used in the training want to > > > > > >> be > > > > > >>>>>>> unbounded, > > > > > >>>>>>>>> and > > > > > >>>>>>>>>>>>> need > > > > > >>>>>>>>>>>>>>>>>>> watermarking in the sources, which means > > > > > >> that > > > > > >>>>> using > > > > > >>>>>>>>>>>>>>>>> NumberSequenceSource > > > > > >>>>>>>>>>>>>>>>>>> isn't an option.) > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> I think the proposed deprecation will be > > > > > >>> better > > > > > >>>>>>>> received > > > > > >>>>>>>>> if > > > > > >>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>> can > > > > > >>>>>>>>>>>>>>> be > > > > > >>>>>>>>>>>>>>>>>>> accompanied by something that makes > > > > > >>>> implementing > > > > > >>>>> a > > > > > >>>>>>>> simple > > > > > >>>>>>>>>>>>> Source > > > > > >>>>>>>>>>>>>>>> easier > > > > > >>>>>>>>>>>>>>>>>>> than it is now. People are continuing to > > > > > >>>>> implement > > > > > >>>>>>> new > > > > > >>>>>>>>>>>>>>>> SourceFunctions > > > > > >>>>>>>>>>>>>>>>>>> because the interfaces defined by FLIP-27 > > > > > >> are > > > > > >>>>> hard > > > > > >>>>>> to > > > > > >>>>>>>>>>>>> understand, > > > > > >>>>>>>>>>>>>>>> while > > > > > >>>>>>>>>>>>>>>>>>> SourceFunction is easy. Alex, I believe you > > > > > >>>> were > > > > > >>>>>>>> looking > > > > > >>>>>>>>>> into > > > > > >>>>>>>>>>>>>>>>>> implementing > > > > > >>>>>>>>>>>>>>>>>>> an easier-to-use building block that could > > > > > >> be > > > > > >>>>> used > > > > > >>>>>> in > > > > > >>>>>>>>>>>>> situations > > > > > >>>>>>>>>>>>>>> like > > > > > >>>>>>>>>>>>>>>>>> this. > > > > > >>>>>>>>>>>>>>>>>>> Can we get something like that in place > > > > > >>> first? > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> David > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> On Fri, Jun 3, 2022 at 4:52 PM Jing Ge < > > > > > >>>>>>>>> j...@ververica.com > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> Hi, > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> Thanks Alex for driving this! > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> +1 To give the Flink developers, > > > > > >> especially > > > > > >>>>>>> Connector > > > > > >>>>>>>>>>>>> developers > > > > > >>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>> clear > > > > > >>>>>>>>>>>>>>>>>>>> signal that the new Source API is > > > > > >>> recommended > > > > > >>>>>>>> according > > > > > >>>>>>>>> to > > > > > >>>>>>>>>>>>>>> FLIP-27, > > > > > >>>>>>>>>>>>>>>> we > > > > > >>>>>>>>>>>>>>>>>>>> should mark them as deprecated. > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> There are some open questions to discuss: > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> 1. Do we need to mark all > > > > > >>>>> subinterfaces/subclasses > > > > > >>>>>>> as > > > > > >>>>>>>>>>>>>> deprecated? > > > > > >>>>>>>>>>>>>>>> e.g. > > > > > >>>>>>>>>>>>>>>>>>>> FromElementsFunction, etc. there are many. > > > > > >>>> What > > > > > >>>>>> are > > > > > >>>>>>>> the > > > > > >>>>>>>>>>>>>>>> replacements? > > > > > >>>>>>>>>>>>>>>>>>>> 2. Do we need to mark all subclasses that > > > > > >>> have > > > > > >>>>>>>>> replacement > > > > > >>>>>>>>>>>>> as > > > > > >>>>>>>>>>>>>>>>>> deprecated? > > > > > >>>>>>>>>>>>>>>>>>>> e.g. ExternallyInducedSource whose > > > > > >>> replacement > > > > > >>>>>>> class, > > > > > >>>>>>>>> if I > > > > > >>>>>>>>>>>>> am > > > > > >>>>>>>>>>>>>> not > > > > > >>>>>>>>>>>>>>>>>> mistaken, > > > > > >>>>>>>>>>>>>>>>>>>> ExternallyInducedSourceReader is > > > > > >>> @Experimental > > > > > >>>>>>>>>>>>>>>>>>>> 3. Do we need to mark all related test > > > > > >>> utility > > > > > >>>>>>> classes > > > > > >>>>>>>>> as > > > > > >>>>>>>>>>>>>>>> deprecated? > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> I think it might make sense to create an > > > > > >>>>> umbrella > > > > > >>>>>>>> ticket > > > > > >>>>>>>>>> to > > > > > >>>>>>>>>>>>>> cover > > > > > >>>>>>>>>>>>>>>> all > > > > > >>>>>>>>>>>>>>>>> of > > > > > >>>>>>>>>>>>>>>>>>>> these with the following process: > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> 1. Mark SourceFunction as deprecated asap. > > > > > >>>>>>>>>>>>>>>>>>>> 2. Mark subinterfaces and subclasses as > > > > > >>>>>> deprecated, > > > > > >>>>>>> if > > > > > >>>>>>>>>>>>> there are > > > > > >>>>>>>>>>>>>>>>>> graduated > > > > > >>>>>>>>>>>>>>>>>>>> replacements. Good example is that > > > > > >>> KafkaSource > > > > > >>>>>>>> replaced > > > > > >>>>>>>>>>>>>>>> KafkaConsumer > > > > > >>>>>>>>>>>>>>>>>> which > > > > > >>>>>>>>>>>>>>>>>>>> has been marked as deprecated. > > > > > >>>>>>>>>>>>>>>>>>>> 3. Do not mark subinterfaces and > > > > > >> subclasses > > > > > >>> as > > > > > >>>>>>>>> deprecated, > > > > > >>>>>>>>>>>>> if > > > > > >>>>>>>>>>>>>>>>>> replacement > > > > > >>>>>>>>>>>>>>>>>>>> classes are still experimental, check if > > > > > >> it > > > > > >>> is > > > > > >>>>>> time > > > > > >>>>>>> to > > > > > >>>>>>>>>>>>> graduate > > > > > >>>>>>>>>>>>>>>> them. > > > > > >>>>>>>>>>>>>>>>>> After > > > > > >>>>>>>>>>>>>>>>>>>> graduation, go to step 2. It might take a > > > > > >>>> while > > > > > >>>>>> for > > > > > >>>>>>>>>>>>> graduation. > > > > > >>>>>>>>>>>>>>>>>>>> 4. Do not mark subinterfaces and > > > > > >> subclasses > > > > > >>> as > > > > > >>>>>>>>> deprecated, > > > > > >>>>>>>>>>>>> if > > > > > >>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>>>> replacement classes are experimental and > > > > > >> are > > > > > >>>> too > > > > > >>>>>>> young > > > > > >>>>>>>>> to > > > > > >>>>>>>>>>>>>>> graduate. > > > > > >>>>>>>>>>>>>>>> We > > > > > >>>>>>>>>>>>>>>>>> have > > > > > >>>>>>>>>>>>>>>>>>>> to wait. But in this case we could create > > > > > >>> new > > > > > >>>>>>> tickets > > > > > >>>>>>>>>> under > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>> umbrella > > > > > >>>>>>>>>>>>>>>>>>>> ticket. > > > > > >>>>>>>>>>>>>>>>>>>> 5. Do not mark subinterfaces and > > > > > >> subclasses > > > > > >>> as > > > > > >>>>>>>>> deprecated, > > > > > >>>>>>>>>>>>> if > > > > > >>>>>>>>>>>>>>> there > > > > > >>>>>>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>>>>>> no > > > > > >>>>>>>>>>>>>>>>>>>> replacement at all. We have to create new > > > > > >>>>> tickets > > > > > >>>>>>> and > > > > > >>>>>>>>> wait > > > > > >>>>>>>>>>>>> until > > > > > >>>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>>>>>>>>> implementation has been done and > > > > > >> graduated. > > > > > >>> It > > > > > >>>>>> will > > > > > >>>>>>>>> take a > > > > > >>>>>>>>>>>>>> longer > > > > > >>>>>>>>>>>>>>>>> time, > > > > > >>>>>>>>>>>>>>>>>>>> roughly 1,5 years. > > > > > >>>>>>>>>>>>>>>>>>>> 6. For test classes, we could follow the > > > > > >>> same > > > > > >>>>>> rule. > > > > > >>>>>>>> But > > > > > >>>>>>>>> I > > > > > >>>>>>>>>>>>> think > > > > > >>>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>> some > > > > > >>>>>>>>>>>>>>>>>>>> cases, we could consider doing the > > > > > >>> replacement > > > > > >>>>>>>> directly > > > > > >>>>>>>>>>>>> without > > > > > >>>>>>>>>>>>>>>> going > > > > > >>>>>>>>>>>>>>>>>>>> through the deprecation phase. > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> When we look back on all of these, we can > > > > > >>>>> realize > > > > > >>>>>> it > > > > > >>>>>>>> is > > > > > >>>>>>>>> a > > > > > >>>>>>>>>>>>> big > > > > > >>>>>>>>>>>>>> epic > > > > > >>>>>>>>>>>>>>>>> (even > > > > > >>>>>>>>>>>>>>>>>>>> bigger than an epic). It needs someone to > > > > > >>>> drive > > > > > >>>>> it > > > > > >>>>>>> and > > > > > >>>>>>>>>> keep > > > > > >>>>>>>>>>>>>> focus > > > > > >>>>>>>>>>>>>>> on > > > > > >>>>>>>>>>>>>>>>> it > > > > > >>>>>>>>>>>>>>>>>>>> continuously with support from the > > > > > >> community > > > > > >>>> and > > > > > >>>>>>> push > > > > > >>>>>>>>> the > > > > > >>>>>>>>>>>>>>>> development > > > > > >>>>>>>>>>>>>>>>>>>> towards the new Source API of FLIP-27. > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> If we could have consensus for this, Alex > > > > > >>>> and I > > > > > >>>>>>> could > > > > > >>>>>>>>>>>>> create > > > > > >>>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>>>>>> umbrella > > > > > >>>>>>>>>>>>>>>>>>>> ticket to kick it off. > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> Best regards, > > > > > >>>>>>>>>>>>>>>>>>>> Jing > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> On Fri, Jun 3, 2022 at 3:54 PM Alexander > > > > > >>>>> Fedulov < > > > > > >>>>>>>>>>>>>>>>>> alexan...@ververica.com> > > > > > >>>>>>>>>>>>>>>>>>>> wrote: > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Hi everyone, > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> I would like to start the discussion > > > > > >> about > > > > > >>>>>> marking > > > > > >>>>>>>>>>>>>>>>> SourceFunction-based > > > > > >>>>>>>>>>>>>>>>>>>>> interfaces as deprecated. With the > > > > > >> FLIP-27 > > > > > >>>> APIs > > > > > >>>>>>>>> becoming > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>> new > > > > > >>>>>>>>>>>>>>>>>>>> standard, > > > > > >>>>>>>>>>>>>>>>>>>>> the old ones have to be eventually phased > > > > > >>>> out. > > > > > >>>>>>>> Although > > > > > >>>>>>>>>>>>> this > > > > > >>>>>>>>>>>>>>> state > > > > > >>>>>>>>>>>>>>>> is > > > > > >>>>>>>>>>>>>>>>>>>> well > > > > > >>>>>>>>>>>>>>>>>>>>> known within the community and no new > > > > > >>>>> connectors > > > > > >>>>>>>> based > > > > > >>>>>>>>> on > > > > > >>>>>>>>>>>>> the > > > > > >>>>>>>>>>>>>> old > > > > > >>>>>>>>>>>>>>>>>>>>> interfaces can be accepted into the > > > > > >>> project, > > > > > >>>>> the > > > > > >>>>>>>>>> footprint > > > > > >>>>>>>>>>>>> of > > > > > >>>>>>>>>>>>>>>>>>>>> SourceFunction in the user code still > > > > > >> keeps > > > > > >>>>>> growing > > > > > >>>>>>>>>>>>> (primarily > > > > > >>>>>>>>>>>>>>> for > > > > > >>>>>>>>>>>>>>>>> data > > > > > >>>>>>>>>>>>>>>>>>>>> generators and test utilities). I believe > > > > > >>> it > > > > > >>>> is > > > > > >>>>>>> best > > > > > >>>>>>>> to > > > > > >>>>>>>>>>>>> mark > > > > > >>>>>>>>>>>>>>>>>>>> SourceFunction > > > > > >>>>>>>>>>>>>>>>>>>>> as deprecated as soon as possible. What > > > > > >> do > > > > > >>>> you > > > > > >>>>>>> think? > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>>> Best, > > > > > >>>>>>>>>>>>>>>>>>>>> Alexander Fedulov > > > > > >>>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> -- > > > > > >>>>>>>>>>>>>> https://twitter.com/snntrable > > > > > >>>>>>>>>>>>>> https://github.com/knaufk > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>>> > > > > > >>>>>> > > > > > >>>>> > > > > > >>>> > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > https://twitter.com/snntrable > > > https://github.com/knaufk > > > > > >
