How about creating a new repository flink-connector-aws and merging dynamodb, kinesis firehouse into it? This can reduce the maintenance for complex dependencies and make the release easy. I think the maintainers of aws-releated connectors are the same people.
Best, Jark > 2022年10月21日 17:41，Chesnay Schepler <ches...@apache.org> 写道： > > I would not go with 2); I think it'd just be messy . > > Here's another option: > > Create another repository (aws-connector-base) (following the externalization > model), add it as a sub-module to the downstream repositories, and make it > part of the release process of said connector. > > I.e., we never create a release for aws-connector-bose, but release it as > part of the connector. > This main benefit here is that we'd always be able to make changes to the > aws-base code without delaying connector releases. > I would assume that any added overhead due to _technically_ releasing the aws > code multiple times to be negligible. > > > On 20/10/2022 22:38, Danny Cranmer wrote: >> Hello all, >> >> Currently we have 2 AWS Flink connectors in the main Flink codebase >> (Kinesis Data Streams and Kinesis Data Firehose) and one new externalized >> connector in progress (DynamoDB). Currently all three of these use common >> AWS utilities from the flink-connector-aws-base module. Common code >> includes client builders, property keys, validation, utils etc. >> >> Once we externalize the connectors, leaving flink-connector-aws-base in the >> main Flink repository will restrict our ability to evolve the connectors >> quickly. For example, as part of the DynamoDB connector build we are >> considering adding a general retry strategy config that can be leveraged by >> all connectors. We would need to block on Flink 1.17 for this. >> >> In the past we have tried to keep the AWS SDK version consistent across >> connectors, with the externalization this is more likely to diverge. >> >> Option 1: I propose we create a new repository, flink-connector-aws, which >> we can move the flink-connector-aws-base module to and create a new >> flink-connector-aws-parent to manage SDK versions. Each of the externalized >> AWS connectors will depend on this new module and parent. Downside is an >> additional module to release per Flink version, however I will volunteer to >> manage this. >> >> Option 2: We can move the flink-connector-aws-base module and create >> flink-connector-parent within the flink-connector-shared-utils repo [2] >> >> Option 3: We do nothing. >> >> For option 1+2 we will follow the general externalized connector versioning >> strategy and rules. >> >> I am inclined towards option 1, and appreciate feedback from the community. >> >> [1] >> https://github.com/apache/flink/tree/master/flink-connectors/flink-connector-aws-base >> [2] https://github.com/apache/flink-connector-shared-utils >> >> Thanks, >> Danny >> >
