Hi Martijn, Thanks for bringing up this discussion.
>From my point of view, Flink Bylaws should also be applied to the connectors. I don't think connectors are just implementations, they provide many APIs for end-users, including DataStream API, and SQL DDL options, SQL metadata columns. There are many FLIPs just discussing the connector APIs, e.g. FLIP-86[1], FLIP-125[2], FLIP-105[3], FLIP-107[4]. Best, Jark [1]: https://cwiki.apache.org/confluence/display/FLINK/FLIP-86%3A+Improve+Connector+Properties [2]: https://cwiki.apache.org/confluence/display/FLINK/FLIP-125%3A+Confluent+Schema+Registry+Catalog [3]: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=147427289 [4]: https://cwiki.apache.org/confluence/display/FLINK/FLIP-107%3A+Handling+of+metadata+in+SQL+connectors#FLIP107:HandlingofmetadatainSQLconnectors-Kafka On Thu, 5 May 2022 at 20:00, Martijn Visser <[email protected]> wrote: > Hi everyone, > > I would like to open a discussion on the process regarding new connectors. > As you know from previous updates [1] we are making a lot of progress on > externalizing connectors that are currently hosted inside in the Flink > repository. > > One topic I would like to bring up for discussion is how the Flink > community wants to deal with new connectors. I've been contacted by many > contributors who are interested in working on one or more connectors. Most > of these connectors are not yet made public or development hasn't started > yet. > > When reading up on the Flink Bylaws [2] I would argue that connectors that > are currently already existing (but not under a Flink project scope) would > fall under 'Adoption of New Codebase' which would require a 2/3 majority > vote by PMC members. Looking at the FLIP requirements [3] you could argue > that any new connector is 'a major new feature, subsystem, or piece of > functionality'. A pro of needing to create a (small) FLIP for a new > connector is that someone needs to think about the design, implementation > and requires a vote, so there is more control. The downside of it is that a > FLIP is considered a drawback, given that a connector normally should be > using only the public interfaces provided by Flink so you could argue it's > just an implementation. > > I'm looking forward to your input to see if we can reach consensus on this > topic, so it can be included in the documentation for contributors that > want to work and maintain a new connector. > > Best regards, > > Martijn Visser > https://twitter.com/MartijnVisser82 > https://github.com/MartijnVisser > > [1] https://lists.apache.org/thread/8k1xonqt7hn0xldbky1cxfx3fzh6sj7h > [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws > [3] > > https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals#FlinkImprovementProposals-Whatisconsidereda%22majorchange%22thatneedsaFLIP > ? >
