ethanlin01x opened a new pull request, #3996: URL: https://github.com/apache/iggy/pull/3996
## Which issue does this PR address? Closes #3731 ## Rationale Iggy can write to several databases, but not to Amazon DynamoDB, so users have to write their own consumer for it. ## What changed? Stream messages could not reach a DynamoDB table without custom code. This adds a sink connector that maps each message to a DynamoDB item and writes it with `BatchWriteItem`, splitting batches at the 25 item limit and retrying throttled requests and unprocessed items with exponential backoff. The item key is built from the stream, topic, partition and message id when the payload does not carry the configured key field, so a redelivered message overwrites the same item. Records that DynamoDB would reject, such as items over 400 KB or an invalid key type, are skipped with a warning instead of failing the whole batch. The sink uses the official `aws-sdk-dynamodb`, so the repo now carries the AWS SDK next to the `rust-s3` stack that the S3 sink uses. DynamoDB has no comparable third-party client. The commits are split so that each one is a working increment: base sink, keys, skipped records, retries, key schema check, then tests and docs. ## Local Execution - Passed - Pre-commit hooks ran ## AI Usage 1. Which tools? Claude Code 2. Scope of usage? Implementation of connector 3. How did you verify the generated code works correctly? Verified with the unit tests, the container-backed integration tests, and a manual end-to-end run against DynamoDB Local 4. Can you explain every line of the code if asked? Yes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
