Thanks tison for starting the discussion here. Both Apache Inlong and Flink CDC Connectors are under Apache License 2.0 protocol, It’s legal even someone copy/distribute all code only if he/she keeps the license files. But from the open source project’s long term maintaining perspective, personally I like upstream first way instead of copy/distribute way. I really hope Flink CDC project can help Apache InLong more and we’re welcome to open issue to statisfy the user requirements from inLong community.
Best, Leonard Xu > 2022年7月24日 下午4:56，Charles Zhang <[email protected]> 写道： > > Hi, tison and Jark, > Thanks for your advice. > Different from Flink CDC connectors, InLong's changes to MySQL CDC > connector mainly involve entire database migration and support for rich > metadata fields (like canal), which are very important for Apache InLong to > provide external services as data integration. For example, for the entire > database migration, if one schema is used for one table, it will be a waste > of resources and difficult to maintain, and it will also put pressure on the > upstream database.We also particularly hope to work with Flink CDC to see how > to support the entire database migration and support richer metadata fields, > so that we can directly rely on it. We don't know if Flink CDC has plans and > technical solutions for these two scenarios. We can talk about it together or > contribute directly. > In addition, in order to speed up discussion and follow-up cooperation, I > have created a WeChat group [1], and interested contributors can join the > direct discussion. > Thanks again. > [1] > <image.png> > > Jark Wu <[email protected] <mailto:[email protected]>> 于2022年7月23日周六 22:45写道： > Thanks for starting this discussion, tison! > > It's great to see InLong is using flink-cdc-connectors to extract data from > various databases. > I think tison raised a good point about contributing back instead of > forking. I believe this can > help the InLong community in the long term by leveraging improvements and > bug fixes from > upstream. Flink CDC can also benefit from more contributions from the > InLong community. > I believe it's the only way that we can improve collaboration between > different OSS projects. > > In a word, I highly appreciate learning the missing part of Flink CDC that > InLong needs. > We can discuss the missing features here or in the issues. > > Cheers, > Jark > > > On Fri, 22 Jul 2022 at 23:56, tison <[email protected] > <mailto:[email protected]>> wrote: > > > Hi, > > > > I notice that inlong-sort mysql-cdc connector[1] is almost copied from > > Flink CDC mysql connector[2]. > > > > With a short discussion with Charles I learned that InLong developers > > added metadata supports and migration of a whole database. > > > > It could be better to see if InLong community can contribute such supports > > back to Flink CDC upstream, and avoid maintaining a whole fork as well as > > periodically catch up upstream updates. > > > > Maintainers of Flink CDC projects are in cc (Leonard & Jark). > > > > Perhaps we can start with creating an issue on Flink CDC repo to describe > > the case, contribute back the improvements, and switch dependency to the > > upstream. > > > > Best, > > tison. > > > > [1] > > https://github.com/apache/inlong/tree/master/inlong-sort/sort-connectors/mysql-cdc > > > > <https://github.com/apache/inlong/tree/master/inlong-sort/sort-connectors/mysql-cdc> > > [2] > > https://github.com/ververica/flink-cdc-connectors/tree/master/flink-connector-mysql-cdc > > > > <https://github.com/ververica/flink-cdc-connectors/tree/master/flink-connector-mysql-cdc> > > > > > > > -- > Best wishes, > Charles Zhang
