Hi, There were many discussions wrt Source-/Sink-Function's deprecation. Especially after FLIP-321 has been newly introduced. One reason, which causes those discussions to happen back and forth, is that we have two different understandings of how and when to use @deprecated.
This is the focus of this discussion thread. The expected output of this discussion is the consensus of using @deprecated. Option 1: the standard @deprecated usage. This is the common case widely used in the software development industry. According to the official Javadoc[1]: "A program element annotated @Deprecated is one that programmers are discouraged from using. An element may be deprecated for any of several reasons, for example, its usage is likely to lead to errors; it may be changed incompatibly or removed in a future version; it has been superseded by a newer, usually preferable alternative; or it is obsolete." FLIP-321[2] shrunk the cases: " In addition, a @Public interface can only be marked as deprecated if - There are one or more @Public interfaces that serve as replacement of its functionality, or - The functionality of the @Public interface is going to be removed completely without any full or partial replacement. " In summary, before APIs can be marked as @deprecated. features behind deprecated APIs can either be abandoned or a fully worked replacement has been provided and users are able to migrate to the replacement with no function regression. Option 2: the adapted @deprecated usage, which is also the de facto standard in the Flink community now With the adapted @deprecated usage, we will mark old APIs as deprecated before the replacement is ready. The replacement will be implemented iteratively until it can fully replace old APIs. There are many cases that fall into this option. The most up-to-date one that many contributors in the community were aware of could be found at [3][4]. We will deprecate the sourceFunction before the replacement is ready. There will be many other cases, e.g. SinkFunction, SinkV1, SinkV2. At the first glance, it seems to break the standard usage described in option 1. But it ends up with a better, flexible, and (more important) feasible solution for the Flink community. Let's see why. Some modules like source or sink in Flink are complex. They defined the core feature with many different downstream modules implementations, i.e. connectors. Different contributors have been working on different modules. Users split into many different cohorts and each only uses a small part of the features. A typical fragmentation issue. It is quite difficult (almost impossible) to know if the new APIs can replace the old ones gracefully following the rules in option 1. Not to mention that in the open source community, contributors for one single module will come and go, which turns out that it is sometimes not easy to have a clear picture even within one single (complex) module. Marking APIs as deprecated in advance became the better practices with the following benefits: 1. Users will be aware of the deprecations in advance. They will have more time to prepare and upgrade their architecture, e.g. isolate features that depend on deprecated APIs or avoid using them. 2. Flink developers can get more attention from users and other developers in the ecosystem. More feedback will be collected(because of the explicit deprecation mark) and will be used to improve the replacement iteratively in a distributed development and consumption environment. 3. Extended deprecation time and migration period. Users or eco modules developers who do not observe ML closely will have a better chance to get informed, join the discussion, and share more insight within their own domain, which will be very valuable to increase the robustness of the replacement before removing deprecated APIs . A win-win situation. Just because option 1 is the standard or officially suggested way to use @deprecated, does not mean it is the best fit for the Flink community. I personally prefer option 2, i.e. the adapted one. And I think there should be more reasons than I knew about why it has been used in the community. One additional (strong) suggestion is that we should force developers to always use "since" and "forRemoval" with @deprecated, at least from now on. Looking forward to your feedback and suggestions. If you think it makes sense to create a FLIP for it, please let me know. I'd like to do it. Best regards, Jing [1] https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/lang/Deprecated.html [2] https://cwiki.apache.org/confluence/display/FLINK/FLIP-321%3A+Introduce+an+API+deprecation+process [3] https://lists.apache.org/thread/yyw52k45x2sp1jszldtdx7hc98n72w7k [4] https://lists.apache.org/thread/kv9rj3w2rmkb8jtss5bqffhw57or7v8v
