PeterZh6 opened a new pull request, #11501: URL: https://github.com/apache/inlong/pull/11501
Fixes [#11494 [Feature][Sort] Add Kafka connector on Flink 1.18](https://github.com/apache/inlong/issues/11494) ### Motivation This PR adds a Kafka connector to `inlong-sort` for Flink 1.18 support. ### Modifications - Migrated the Kafka connector from [flink-kafka-connector 3.2.0](https://github.com/apache/flink-connector-kafka/releases/tag/3.2.0) to support Flink 1.18. - Updated the relevant Maven `pom.xml` to reflect the changes. ### Verifying this Change - [ ] This change is a trivial rework/code cleanup without any test coverage. - [ ] This change is already covered by existing tests, such as: *(please describe tests)* - [x] This change added tests and can be verified as follows: Further formal testing will be introduced in [#11495](https://github.com/apache/inlong/issues/11495). Currently, a `KafkaProducer` is used to send messages to Kafka, and these messages are consumed by `sort-connectors-v1.18`, which is introduced in this PR. Since there is no verified functional sink yet, the built-in `file-system` sink is used in this test. In the example below, we can see that the messages in Kafka are successfully extracted and written to `/tmp/output_file`. #### Example Flink SQL: ```sql CREATE TABLE kafka_source ( `message` STRING ) WITH ( 'connector' = 'kafka-inlong', 'topic' = 'test-topic', 'properties.bootstrap.servers' = 'kafka:9092', 'properties.group.id' = 'flink-group', 'scan.startup.mode' = 'earliest-offset', 'format' = 'json' ); CREATE TABLE file_sink ( `message` STRING ) WITH ( 'connector' = 'filesystem', 'path' = 'file:///tmp/output_file', 'format' = 'json' ); INSERT INTO file_sink SELECT * FROM kafka_source; ``` **Output:** The messages in Kafka have been successfully extracted and transported to `/tmp/output_file`. See the screenshot below for the result:  ### Documentation - **Does this pull request introduce a new feature?** Yes - **If yes, how is the feature documented?** N/A - **If a feature is not applicable for documentation, explain why:** The version change of the sort connectors is not reflected on the official website. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
