Hi Martijn,
Thanks for the reply. Regarding the behavioral stability guarantee, by
definition, an "API" always consists of the signature + behavior. So, the
behavior is a native part of an API. Therefore, behavioral changes should
be treated with the same guarantee as signature changes.
+1 on hav
Hi all,
Thanks for the lively and good discussion. Given the length of the
discussion, I skimmed through and then did a deep dive on the latest state
of the FLIP. I think the FLIP is overall in a good state and ready to bring
to a vote.
One thing that I did notice while skimming through the discu
Hi Xintong, Jark and Jing,
Thanks for the reply. Yes, we can only mark the DataStream API as
@Deprecated after the ProcessFunction API is fully functional and mature.
It is a fair point that the condition of marking a @Public API as
deprecated should also be a part of this FLIP. I just added that
>
> Considering DataStream API is the most fundamental and complex API of
> Flink, I think it is worth a longer time than the general process for the
> deprecation period to wait for the new API be mature.
>
This inspires me. In this specific case, compared to how long should
DataStream API be rem
Hi all,
Just want to make sure we are on the same page. There is another example[1]
I was aware of recently that shows why more factors need to be taken care
of than just the migration period. Thanks Galen for your hint.
To put it simply, the concern about API deprecation is not that deprecated
A
I agree with Jingsong and Becket.
Look at the legacy SourceFunction (a small part of DataStream API),
the SourceFunction is still not and can't be marked deprecated[1] until
now after the new Source was released 2 years ago, because the new Source
still can't fully consume the abilities of legacy
Hi Jingsong,
Thanks for the reply. I completely agree with you.
The above 2 options are based on the assumption that the community cannot
afford to maintain the deprecated DataStream API for long. I'd say we
should try everything we can to maintain it for as much time as possible.
DataStream API
Thanks Becket and all for your discussion.
> 1. We say this FLIP is enforced starting release 2.0. For current 1.x APIs,
we provide a migration period with best effort, while allowing exceptions
for immediate removal in 2.0. That means we will still try with best effort
to get the ProcessFuncion A
Thanks much for the input, John, Stefan and Jing.
I think Xingtong has well summarized the pros and cons of the two options.
Let's collect a few more opinions here and we can move forward with the one
more people prefer.
Thanks,
Jiangjie (Becket) Qin
On Wed, Jun 21, 2023 at 3:20 AM Jing Ge wro
Hi all,
Thanks Xingtong for the summary. If I could only choose one of the given
two options, I would go with option 1. I understood that option 2 worked
great with Kafka. But the bridge release will still confuse users and my
gut feeling is that many users will skip 2.0 and be waiting for 3.0 or
Hi Xintong,
Thanks for the summary, most of the points that you agreed upon also make sense
to me.
> 2. Dropping deprecated Public APIs in minor releases, or demoting APIs
> from Public to PublicEvolving / Experimental / Retired in any version bump,
> are not good practices.
I hope we can
Hi Xintong,
Thanks for the update!
FWIW, I'm also in agreement with the points that you and Becket agree on. As
you can probably guess, I'd also vote for option (2) on how to handle removal
of the DataStream API.
This is actually similar to how Kafka handled the removal of Zookeeper, and
it's
Becket and I had an offline voice call earlier today. We have reached
consensus on some of the arguments, but still hold different opinions on
some others. I'd like to post the outcome here for transparency.
We both agree that:
1. Providing a migration period would be beneficial for our users,
Hi Becket and Xintong,
I hope you don't mind if I chime in a little.
Once an API is marked Public, we're committing to support it until it's
deprecated, and once it's deprecated, to leave it in place for at least two
minor releases and then only remove it in the following major release.
As a u
Hi Xintong,
Let's compare the following cases:
A. If the maintenance overhead of the deprecated API is high, and we want
to remove it after two minor releases. Then there are two options:
A1: Demote the API in the major version bump and remove the code with a
minor version bump.
A2: Do ex
>
> The part I don't understand is if we are willing to have a migration
> period, and do a minor version bump to remove an API, what do we lose to do
> a major version bump instead, so we don't break the common versioning
> semantic?
>
I think we are talking about the cases where a major version
Hi Xintong,
Please see the replies below.
I see your point, that users would feel surprised if they find things no
> longer work when upgrading to another 2.x minor release. However, I'd like
> to point out that PublicEvolving APIs would have the similar problem
> anyway. So the question is, how
>
> As an end user who only uses Public APIs, if I don't change my code at
> all, my expectation is the following:
> 1. Upgrading from 1.x to 2.x may have issues.
> 2. If I can upgrade from 1.x to 2.x without an issue, I am fine with all
> the 2.x versions.
> Actually I think there are some depende
Hi Becket,
Thanks for the reply! I’d like to continue the conversation about compatibility
outside of this FLIP thread, but for now, I can accept your decision. It’s
certainly an improvement.
Thanks again,
John
On Sun, Jun 18, 2023, at 21:42, Becket Qin wrote:
> Hi John,
>
> Completely agree
Hi John,
Completely agree with all you said.
Can we consider only dropping deprecated APIs in major releases across the
> board? I understand that Experimental and PublicEvolving APIs are by
> definition less stable, but it seems like this should be reflected in the
> required deprecation period
Hi Becket,
Thanks for this FLIP! Having a deprecation process is really important. I
understand some people’s concerns about the additional burden for project
maintainers, but my personal experience with Kafka has been that it’s very
liveable and that it’s well worth the benefit to users. In fa
Hi All,
The @Public -> @PublicEvolving proposed by Xintong is a great idea.
Especially, after he suggest @PublicRetired, i.e. @PublicEvolving --(2
minor release)--> @Public --> @deprecated --(1 major
release)--> @PublicRetired. It will provide a lot of flexibility without
breaking any rules we had
Hi Becket,
Thanks for the clarification. Sorry that I didn't make myself clear enough.
Let me share more thoughts.
> Deprecating an API is just a more elegant way of replacing an API with a
> new one. The only difference between the two is whether the old API is kept
> and coexists with the new
Hi Xintong,
Thanks for the explanation. Please see the replies inline below.
I agree. And from my understanding, demoting a Public API is also a kind of
> such change, just like removing one, which can only happen with major
> version bumps. I'm not proposing to allow demoting Public APIs anytime
Hi Chesnay,
I think that there are two things we are discussing here:
1. The API stability story we WANT to have.
2. The API stability guarantees we CAN have.
We can only design for what we want. Good API stability with affordable
maintenance overhead does demand careful design from the high leve
>
> Public API is a well defined common concept, and one of its
> convention is that it only changes with a major version change.
>
I agree. And from my understanding, demoting a Public API is also a kind of
such change, just like removing one, which can only happen with major
version bumps. I'm n
On 13/06/2023 17:26, Becket Qin wrote:
It would be valuable if we can avoid releasing minor versions for previous
major versions.
On paper, /absolutely /agree, but I'm not sure how viable that is in
practice.
On the current 2.0 agenda is potentially dropping support for Java 8/11,
which may
Hi Xintong,
I think the key of stability guarantees is about not breaking the
> commitments we made to users. From my understanding, when we mark an API as
> Public in for example 1.18, we make the following two commitments.
> a) The API will not be removed in all the 1.x serials.
> b) The API wil
Hi all,
First off, thanks for opening the discussion on this topic. I think it's an
important one.
>From my perspective, I think that one of the major pain points that users
have expressed over time, is the API stability of Flink overall. I think
that every Flink user wants to have reliable, stab
I agree that Public APIs should require a longer migration period. I think
that is why the FLIP requires at least 2 minor releases (compared to 1
minor release for PublicEvolving and 1 patch release for Experimental).
I think the key of stability guarantees is about not breaking the
commitments we
Thanks for the explanation, Matthias.
In the example you raised, would it be better to just keep both YARN and
K8S support in the new major version, but with YARN support deprecated if
we want to? We can say for YARN we will only provide bug fixes but no
feature development anymore. Given these tw
One (made-up) example from the top of my head would have been that the
community decides to focus fully on Kubernetes without considering Yarn
anymore because of some must-have feature on the Kubernetes side. At the
same time there are still some users for whom it would be tricky to migrate
from Ya
Hi Matthias,
Thanks for the feedback.
Do you have an example of behavioral change in mind? Not sure I fully
understand the concern for behavioral change here. From what I understand,
any user sensible change in an existing API, regardless of its kind (API
signature or behavioral change), can alwa
Hi Xintong,
Thanks for the comment. Please see the replies below:
1. Do we allow deprecation & removal of APIs without adding a new one as a
> replacement? The examples in the table give me an impression that marking
> an API as `@Deprecated` should only happen at the same time of introducing
> a
Thanks for starting this discussion, Becket. A few good points were raised.
Here's what I want to add:
Stefan raised the point of behavioral stability (in contrast to API
stability). That might be a reason for users to not be able to go ahead
with a major version bump. Working around behavioral ch
Thanks for bringing up this discussion, Becket.
My two cents:
1. Do we allow deprecation & removal of APIs without adding a new one as a
replacement? The examples in the table give me an impression that marking
an API as `@Deprecated` should only happen at the same time of introducing
a new repla
Hi Jing,
Thanks for the feedback. Please see the answers to your questions below:
*"Always add a "Since X.X.X" comment to indicate when was a class /
> interface / method marked as deprecated."*
> Could you describe it with a code example? Do you mean Java comments?
It is just a comment such as
> This is by design. Most of these are @Public APIs that we had to carry
> around until Flink 2.0, because that was the initial guarantee that we
> gave people.
>
True, I knew @Public APIs could not be removed before the next major
release. I meant house cleaning without violation of these annotat
Hi Chesnay and Stefan,
Thanks for the feedback and sorry for the confusion about Public API
deprecation. I just noticed that there was a mistake in the NOTES part for
Public API due to a copy-paste error... I just fixed it.
To clarify on the deprecation of "Public" APIs.
The FLIP does not propos
On 13/06/2023 12:50, Jing Ge wrote:
One major issue we have, afaiu, is caused by the lack of housekeeping/house
cleaning, there are many APIs that were marked as deprecated a few years
ago and still don't get removed. Some APIs should be easy to remove and
others will need some more clear rules,
Hi Becket,
> *1. What is the scope of public interfaces?*
> We actually already have a definition of public interfaces in the FLIP[1]
> wiki, it is basically anything that is sensible to the users, including
> packages, classes, method signature and behavior (blocking v.s.
> non-blocking, for ex
Hi Becket,
Thanks for driving this important topic! There were many discussions
previously that ended up with waiting up for a clear API deprecation
process definition. This FLIP will help a lot.
I'd like to ask some questions to understand your thoughts.
Speaking of the FLIP,
*"Always add a "S
Hi Stefan,
Thanks for the comments. I agree the API stability and compatibility
guarantees should be clear.
I think we need to answer the following four questions in order to complete
the the public API stability and compatibility story:
*1. What is the scope of public interfaces?*
We actually al
Hi,
Thanks a lot for bringing up this topic and for the initial proposal. As more
and more people are looking into running Flink as a continuous service this
discussion is becoming very relevant.
What I would like to see is a clearer definition for what we understand by
stability and compatibi
Hi folks,
As one of the release 2.0 efforts, the release managers were discussing our
API lifecycle policies. There have been FLIP-196[1] and FLIP-197[2] that
are relevant to this topic. These two FLIPs defined the stability guarantee
of the programming APIs with various different stability annota
45 matches
Mail list logo
