dannycranmer commented on code in PR #668: URL: https://github.com/apache/flink-web/pull/668#discussion_r1283535822
########## docs/content/posts/2023-08-03-externalized-connectors.md: ########## @@ -0,0 +1,77 @@ +--- +title: "Announcing three new Apache Flink connectors, the new connector versioning strategy and externalization" +date: "2023-08-03T12:00:00Z" +authors: +- elphastori: + name: "Elphas Toringepi" + twitter: "elphastori" +aliases: +- /news/2023/08/03/externalized-connectors.html +--- + +## New connectors + +We're excited to announce that Apache Flink now supports three new connectors: [Amazon DynamoDB](https://aws.amazon.com/dynamodb), [MongoDB](https://www.mongodb.com/) and [OpenSearch](https://opensearch.org/)! The connectors are available for both the [DataStream](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/overview/) and [Table/SQL](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/overview/) APIs. + +- **[Amazon DynamoDB](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/dynamodb/)** - This connector includes a sink that provides at-least-once delivery guarantees. +- **[MongoDB connector](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/mongodb/)** - This connector includes a source and sink that provide at-least-once guarantees. +- **[OpenSearch sink](https://github.com/apache/flink-connector-opensearch/blob/main/docs/content/docs/connectors/datastream/opensearch.md)** - This connector includes a sink that provides at-least-once guarantees Review Comment: ```suggestion - **[OpenSearch sink](https://github.com/apache/flink-connector-opensearch/blob/main/docs/content/docs/connectors/datastream/opensearch.md)** - This connector includes a sink that provides at-least-once guarantees. ``` ########## docs/content/posts/2023-08-03-externalized-connectors.md: ########## @@ -0,0 +1,77 @@ +--- +title: "Announcing three new Apache Flink connectors, the new connector versioning strategy and externalization" +date: "2023-08-03T12:00:00Z" +authors: +- elphastori: + name: "Elphas Toringepi" + twitter: "elphastori" +aliases: +- /news/2023/08/03/externalized-connectors.html +--- + +## New connectors + +We're excited to announce that Apache Flink now supports three new connectors: [Amazon DynamoDB](https://aws.amazon.com/dynamodb), [MongoDB](https://www.mongodb.com/) and [OpenSearch](https://opensearch.org/)! The connectors are available for both the [DataStream](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/overview/) and [Table/SQL](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/overview/) APIs. + +- **[Amazon DynamoDB](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/dynamodb/)** - This connector includes a sink that provides at-least-once delivery guarantees. +- **[MongoDB connector](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/mongodb/)** - This connector includes a source and sink that provide at-least-once guarantees. +- **[OpenSearch sink](https://github.com/apache/flink-connector-opensearch/blob/main/docs/content/docs/connectors/datastream/opensearch.md)** - This connector includes a sink that provides at-least-once guarantees + +|Connector|Date Released|Supported Flink Versions| +|---|---|---| +|Amazon DynamoDB sink|12/2/2022|1.15+| +|MongoDB connector|3/31/2023|1.16+| +|OpenSearch sink|12/21/2022|1.16+| + +## Externalized connectors + +The community have externalized connectors from [Flink’s main repository](https://github.com/apache/flink). This was driven to realise the following benefits: +- **Faster releases of connectors:** New features can be added more quickly, bugs can be fixed immediately, and we can have faster security patches in case of direct or indirect (through dependencies) security flaws. +- **Adding newer connector features to older Flink versions:** By having stable connector APIs, the same connector artifact may be used with different Flink versions. Thus, new features can also immediately be used with older Flink versions. +- **More activity and contributions around connectors:** By easing the contribution and development process around connectors, we will see faster development and also more connectors. +- **Documentation:** Standardized documentation and user experience for the connectors, regardless of where they are maintained. +- **A faster Flink CI:** By not needing to build and test connectors, the Flink CI pipeline will be faster and Flink developers will experience fewer build stabilities (which mostly come from connectors). That should speed up Flink development. + +The following connectors have been moved to individual repositories: + +- [Kafka / Upsert-Kafka](https://github.com/apache/flink-connector-kafka) +- [Cassandra](https://github.com/apache/flink-connector-cassandra) +- [Elasticsearch](https://github.com/apache/flink-connector-elasticsearch/) +- [**MongoDB**](https://github.com/apache/flink-connector-mongodb) +- **[OpenSearch](https://github.com/apache/flink-connector-opensearch)** +- [RabbitMQ](https://github.com/apache/flink-connector-rabbitmq) +- [Google Cloud PubSub](https://github.com/apache/flink-connector-gcp-pubsub) +- [Pulsar](https://github.com/apache/flink-connector-pulsar/) +- [JDBC](https://github.com/apache/flink-connector-jdbc) +- [HBase](https://github.com/apache/flink-connector-hbase) +- [Hive](https://github.com/apache/flink-connector-hive) +- [AWS connectors](https://github.com/apache/flink-connector-aws): + - Firehose + - Kinesis + - **DynamoDB** + +### Versioning + +Connectors continue to use the same maven dependency `groupId` and `artificatId`, however, the JAR artifact `version` has changed and now uses the format, `<major>.<minor>.<patch>-<flink-major>.<flink-minor>`. For example, to use the DynamoDB connector for Flink 1.17, add the following dependency to your project: Review Comment: ```suggestion Connectors continue to use the same Maven dependency `groupId` and `artificatId`, however, the JAR artifact `version` has changed and now uses the format, `<major>.<minor>.<patch>-<flink-major>.<flink-minor>`. For example, to use the DynamoDB connector for Flink 1.17, add the following dependency to your project: ``` ########## docs/content/posts/2023-08-03-externalized-connectors.md: ########## @@ -0,0 +1,77 @@ +--- +title: "Announcing three new Apache Flink connectors, the new connector versioning strategy and externalization" +date: "2023-08-03T12:00:00Z" +authors: +- elphastori: + name: "Elphas Toringepi" + twitter: "elphastori" +aliases: +- /news/2023/08/03/externalized-connectors.html +--- + +## New connectors + +We're excited to announce that Apache Flink now supports three new connectors: [Amazon DynamoDB](https://aws.amazon.com/dynamodb), [MongoDB](https://www.mongodb.com/) and [OpenSearch](https://opensearch.org/)! The connectors are available for both the [DataStream](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/overview/) and [Table/SQL](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/overview/) APIs. + +- **[Amazon DynamoDB](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/dynamodb/)** - This connector includes a sink that provides at-least-once delivery guarantees. +- **[MongoDB connector](https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/mongodb/)** - This connector includes a source and sink that provide at-least-once guarantees. +- **[OpenSearch sink](https://github.com/apache/flink-connector-opensearch/blob/main/docs/content/docs/connectors/datastream/opensearch.md)** - This connector includes a sink that provides at-least-once guarantees + +|Connector|Date Released|Supported Flink Versions| +|---|---|---| +|Amazon DynamoDB sink|12/2/2022|1.15+| +|MongoDB connector|3/31/2023|1.16+| +|OpenSearch sink|12/21/2022|1.16+| Review Comment: nit: Are these date formats used elsewhere? Would prefer yyyy-mm-dd personally -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
