realonbebeto opened a new pull request, #3654: URL: https://github.com/apache/iggy/pull/3654
## Summary Adds a Redshift sink connector for writing Apache Iggy messages into Amazon Redshift using S3-staged Parquet files and Redshift's COPY command. The connector batches messages into Parquet files, uploads them to a configurable S3 bucket/prefix, and performs bulk COPY loads into the target Redshift table. It supports configurable batch sizing, connection pooling, optional metadata/checksum/origin timestamp fields, payload encoding modes, retry/backoff handling for failed loads, optional archival of staged Parquet files, and runtime metrics logging. The implementation treats S3 purely as a staging layer for efficient bulk ingestion rather than a destination system. Staged files can either be deleted after a successful load or retained under an archive prefix for replay, auditing, or downstream processing. It also integrates the connector throughout the project, including workspace membership, connector documentation, release and binary artifact pipelines, version bump automation, and Docker-based integration test infrastructure. ### Disclosures 1. `pgwire`, `sqlparser`, and `tokio-postgres `are introduced as workspace dependencies solely to support the RedshiftFixture used by the integration test suite. 2. The Redshift integration tests are inspired by and closely follow the existing PostgreSQL test setup to maintain consistency across sink connector implementations. Closes #2540 ## AI Usage A blend of ChatGPT and Claude was used to help build, refactor the connector and associated test suite -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
