kriti-sc opened a new pull request, #2886: URL: https://github.com/apache/iggy/pull/2886
## Which issue does this PR close? Closes #2539 ## Rationale Clickhouse is a real-time data analytics engine, and very popular in modern analytics architectures. ## What changed? This PR introduces a Clickhouse Sink Connector that enables writing data from Iggy to Clickhouse. The Clickhouse writing logic is heavily inspired by the [official Clickhouse Kafka Connector](https://github.com/ClickHouse/clickhouse-kafka-connect). ## Local Execution - Produced messages 30456 + 29060 rows with schema user_id: String, user_type: u8, email: String, source: String, state: String, created_at: DateTime<Utc>, message: String using [sample data producer](https://github.com/apache/iggy/tree/master/examples/rust/src/sink-data-producer). - Consumed messages using the Clickhouse sink and into the particular Clickhouse table. - Verified schema and number of rows in Clickhouse. - Added unit tests and e2e tests, both passing. Images 1&2: Produced 30456 + 29060 rows into Iggy in two batches Image 3: Verified schema and number of rows in Clickhouse <img width="400" alt="image" src="https://github.com/user-attachments/assets/80bdcf91-2858-4eda-99f7-828cdf220e27"; /> <img width="400" alt="image" src="https://github.com/user-attachments/assets/da970c9d-478a-451e-a04f-0faf3ec38895"; /> <img width="500" alt="image" src="https://github.com/user-attachments/assets/3ae0f6d1-200c-4e4c-af89-6a9c83163d52"; /> ## AI Usage 1. Which tools? (e.g., GitHub Copilot, Claude, ChatGPT) Claude Code 2. Scope of usage? (e.g., autocomplete, generated functions, entire implementation) generated functions 3. How did you verify the generated code works correctly? Manual testing by producing data into Iggy and then running the sink and verifying insertion into Clickhouse, unit tests and e2e tests for different Clickhouse insert configurations. 4. Can you explain every line of the code if asked? Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
