Re: [DISUCSS] Deprecate multiple APIs in 1.18

got it, thanks for the clarification. Best regards, Jing On Fri, Jul 21, 2023 at 5:14 AM Jane Chan wrote: > Hi, Jing, > > I share the same opinion with Xintong. The base class `TimestampExtractor` > for these two classes was deprecated in release-1.12[1][2], so deprecating > these two

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi, Jing, I share the same opinion with Xintong. The base class `TimestampExtractor` for these two classes was deprecated in release-1.12[1][2], so deprecating these two subclasses is relatively straightforward. [1] https://issues.apache.org/jira/browse/FLINK-19453 [2]

Re: [DISUCSS] Deprecate multiple APIs in 1.18

I personally don't think it's necessary. IIUC, we have already decided to deprecate TableSource / TableSink, and not marking StreamRecordTimestamp and ExistingField deprecated is just an oversight in carrying out that decision. Best, Xintong On Fri, Jul 21, 2023 at 7:48 AM Jing Ge wrote: >

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Jane, Hi Xintong, The specific discussion of deprecating `StreamRecordTimestamp` & `ExistingField` was kind of hiding in the discussion of the umbrella topic that people might miss out. Does it make sense to start an individual Discussion thread for it like we deprecate any other APIs? For

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thank you, Jane. What you said (preparing a FLIP for ManagedTable related classes, not deprecating SourceFunctionProvider, and starting a dedicated discussion for the missing annotations) sounds good to me. In addition, if there's no objections from the community on marking

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xintong, > IIUC, you are suggesting to mark the classes with a red background as `@Deprecated` in 1.18? Exactly. They are not marked at the code level but suppose to be deprecated. > - For the ManagedTable related classes, is there any FLIP explicitly that decides to deprecate them? If not,

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks for the beautiful sheets, Jane. 1. This sheet < > https://docs.google.com/spreadsheets/d/1dZBNHLuAHYJt3pFU8ZtfUzrYyf2ZFQ6wybDXGS1bHno/edit?usp=sharing> > summarizes the user-facing classes and methods that need to be deprecated > under the flink-table module, some of which are marked with

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xintong, Thanks for driving this topic. Regarding the Table API deprecation, I can provide some details to help with the process. 1. This sheet summarizes the user-facing classes

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks for bringing this up, Matthias. I've replied to you in the other thread[1]. Best, Xintong [1] https://lists.apache.org/thread/l3dkdypyrovd3txzodn07lgdwtwvhgk4 On Mon, Jul 10, 2023 at 8:37 PM Matthias Pohl wrote: > @Xintong did you come across FLINK-3957 [1] when looking for

Re: [DISUCSS] Deprecate multiple APIs in 1.18

@Xintong did you come across FLINK-3957 [1] when looking for deprecated issues? There are a few subtasks that would require deprecations as well as far as I can see. For some I'm wondering whether we should add them to the release 2.0 feature list [2] in some way and (as a consequence) missed them

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Alex, I would follow FLIP-197 and try to release them asap depending on dev resources and how difficult those issues are. The fastest timeline is the period defined in FLIP-197 in ideal conditions. Best regards, Jing On Fri, Jul 7, 2023 at 12:20 PM Alexander Fedulov <

Re: [DISUCSS] Deprecate multiple APIs in 1.18

@Xintong > - IIUC, the testing scenario you described is like blocking the source for > proceeding (emit data, finish, etc.) until a checkpoint is finished. It is more tricky than that - we need to prevent the Sink from receiving a checkpoint barrier until the Source is done emitting a given set

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks all for the discussion. I've created FLINK-32557 for this. Best, Xintong On Thu, Jul 6, 2023 at 1:00 AM Jing Ge wrote: > Hi Alex, > > > > > 3. remove SinkFunction. > > Which steps do you imply for the 1.18 release and for the 2.0 release? > > > > for 2.0 release. 1.18 will be

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Alex, > > 3. remove SinkFunction. > Which steps do you imply for the 1.18 release and for the 2.0 release? > for 2.0 release. 1.18 will be released soon. Best regards, Jing On Wed, Jul 5, 2023 at 1:08 PM Alexander Fedulov < alexander.fedu...@gmail.com> wrote: > @Jing > Just to clarify,

Re: [DISUCSS] Deprecate multiple APIs in 1.18

@Chesnay, shouldn't the decision to deprecate an API be part of the FLIP discussion? > Exactly. I agree that deprecation of an old API should be part of the FLIP where the new API is introduced. And I appreciate that many APIs that are listed to be removed in release 2.0 are already deprecated

Re: [DISUCSS] Deprecate multiple APIs in 1.18

@Jing Just to clarify, when you say: > 3. remove SinkFunction. Which steps do you imply for the 1.18 release and for the 2.0 release? @Xintong A side note - with the new Source API we lose the ability to control checkpointing from the source since there is no lock anymore. This functionality is

Re: [DISUCSS] Deprecate multiple APIs in 1.18

There's a whole bunch of metric APIs that would need to be deprecated. That is of course if the metric FLIPs are being accepted. Which makes me wonder if we aren't doing things the wrong way around; shouldn't the decision to deprecate an API be part of the FLIP discussion? On 05/07/2023

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks all for the discussion. It seems to me there's a consensus on marking the following as deprecated in 1.18: - DataSet API - SourceFunction - Queryable State - All Scala APIs More time is needed for deprecating SinkFunction. I'll leave this discussion open for a few more days. And if

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks for the input, Jing. I'd also be +1 for option 1. Best, Xintong On Mon, Jul 3, 2023 at 7:20 PM Jing Ge wrote: > Hi Xingtong, > > Option 1, secure plan would be: > > 1. graduate kafka, File, JDBC connectors to @Public > 2. graduate SinkV2 to @Public > 3. remove SinkFunction. > >

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xingtong, Option 1, secure plan would be: 1. graduate kafka, File, JDBC connectors to @Public 2. graduate SinkV2 to @Public 3. remove SinkFunction. Option 2, risky plan but at a fast pace: 1. graduate SinkV2 to @Public and expecting more maintenance effort since there are many known and

Re: [DISUCSS] Deprecate multiple APIs in 1.18

I see. Thanks for the explanation. I may have not looked into this deeply enough, and would trust the decision from you and the community members who participated in the discussion & vote. Best, Xintong On Thu, Jun 29, 2023 at 10:28 PM Alexander Fedulov < alexander.fedu...@gmail.com> wrote:

Re: [DISUCSS] Deprecate multiple APIs in 1.18

> However, I'm not sure about 2. I am not aware of a bylaw that states the specific requirements in order to mark something as @Deprecated. My understanding from the discussion and the vote was that the community recognizes the necessity to make it explicit that the usage of the SourceFunction

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks for the explanation, Alex. Not blocking the deprecation on 1 & 3 makes sense to me. However, I'm not sure about 2. It sounds to me that, without FLINK-28051 & FLINK-28054, some of the connectors cannot migrate to the new Source API, or at least further investigation is needed to

Re: [DISUCSS] Deprecate multiple APIs in 1.18

@Xintong The original discussion [1] and vote [2] converged on the idea that it is better to make it clear to the users that they should stop using SourceFunction since it is going away. The longer we do not have this indication, the more user implementations will be based on it and the more pain

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks for the inputs, Martijn, Jing and Alex. @Martijn, Regarding the Scala supports, I personally don't think "a fully striked through experience in the IDE" is something we want to avoid, especially given that we are planning to remove the deprecated APIs soon, unlike when FLINK-29740 was

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xintong, Thanks for bringing up this topic. I can provide some details regarding the SourceFunction deprecation efforts. Marking SourceFunction as deprecated was not possible until now since we have stringent compiler checks in flink-examples against using any deprecated APIs. We actually

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xintong, 2. SourceFunction / SinkFunction This is one with challenges. There were many discussions in the past. 2.1 SourceFunction The voting was passed[1], but there are still some ongoing issues[2]. The SourceFunction is not marked as @deprecated yet[3], afaik. 2.2 SinkFunction SinkV2 is a

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xintong, With regards to the deprecation of the Scala APIs, during the PR review it was requested to not mark all APIs as deprecated but only the entry point [1], to avoid a fully striked through experience in the IDE. I think the same idea was applicable on the DataSet API. I think it depends

[DISUCSS] Deprecate multiple APIs in 1.18

Hi devs, Looking at the release 2.0 proposals [1], I noticed that many APIs that are proposed to be removed in 2.0 are not (fully) deprecated yet. We might want to properly mark them as `@Deprecated` in 1.18 if we agree they should be removed in 2.0. Moreover, according to FLIP-321 [2] (not voted