Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-16 Thread Dawid Wysakowicz
In addition to fixing the annotation consistency formally, I think it is equally, if not more, important to see whether the public APIs we expose tell a good story. For example, if we say StreamConfig should be internal, some fair questions to ask is why our own

Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-13 Thread Jing Ge
Hi Becket, Speaking of tools, we have ArchUnit integrated in Flink. Extending the defined ArchRules [1] a little bit, you will get the wished scan result. [1]

Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-13 Thread Becket Qin
I don't have an overview of all the holes in our public API surface at the moment. It would be great if there's some tool to do a scan. In addition to fixing the annotation consistency formally, I think it is equally, if not more, important to see whether the public APIs we expose tell a good

Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-13 Thread Konstantin Knauf
Hi Becket, > It is a basic rule of public API that anything exposed by a public interface should also be public. I agree with this in general. Did you get an overview of where we currently violate this? Is this something that the Arc42 architecture tests could test for so that as a first measure

Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-13 Thread Becket Qin
Hi Dawid, Thanks for the reply. I am currently looking at the Beam Flink runner, and there are quite some hacks the Beam runner has to do in order to deal with the backwards incompatible changes in AbstractStreamOperator and some of the classes exposed by it. Regardless of what we think, the fact

Re: [DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-13 Thread Dawid Wysakowicz
Hi Becket, May I ask what is the motivation for the change? I'm really skeptical about making any of those classes `Public` or `PublicEvolving`. As far as I am concerned there is no agreement in the community if StreamOperator is part of the `Public(Evolving)` API. At least I think it should

[DISCUSS] FLIP-286: Fix the API stability/scope annotation inconsistency in AbstractStreamOperator

2023-01-12 Thread Becket Qin
Hi flink devs, I'd like to start a discussion thread for FLIP-286[1]. As a recap, currently while AbstractStreamOperator is a class marked as @PublicEvolving, some classes exposed via its methods / fields are marked as @Internal. This FLIP wants to fix this inconsistency of stability / scope