davsclaus opened a new pull request, #24861: URL: https://github.com/apache/camel/pull/24861
_Claude Code on behalf of davsclaus_ ## Summary Fixes four related defects in `ShardIteratorHandler` that cause silent data loss when DynamoDB rotates or splits shards (every ~4 hours): 1. **Stale shard tree** — `shardTree.populate()` was only called when `currentShardIterators` was empty. Shards created after startup were never discovered, causing all subsequent writes to a rotated partition to be silently dropped. 2. **ConcurrentModificationException** — `getShardIterators()` returned the internal `HashMap` by reference while `updateShardIterator()` structurally modified it during the consumer's iteration loop. 3. **Missing `describeStream` pagination** — streams with >100 shards lost partitions because `lastEvaluatedShardId` was ignored. 4. **Stuck consumer on expired iterator** — `requestFreshShardIterator` sent `AFTER_SEQUENCE_NUMBER` with a null sequence number when the iterator expired before any record was read. ## Changes - `ShardIteratorHandler`: track closed shards in `pendingClosedShards`; refresh the tree via paginated `describeStream` when a shard closes; return a defensive copy; fall back to `TRIM_HORIZON` on null sequence number. - `AmazonDDBStreamsClientMock`: replace-by-shardId support and request tracking for test assertions. - `ShardIteratorHandlerTest`: 3 new tests covering resharding discovery, defensive copy, and null-sequence fallback. ## Test plan - [x] All 65 existing + new unit tests pass (`mvn test` in `camel-aws2-ddb`) - [ ] CI green 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.6 <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
