Federico Mariani created CAMEL-24156:
----------------------------------------
Summary: camel-aws2 - Umbrella: high-severity findings from July
2026 review (S3, SQS, SNS, Kinesis, DDB)
Key: CAMEL-24156
URL: https://issues.apache.org/jira/browse/CAMEL-24156
Project: Camel
Issue Type: Bug
Components: camel-aws
Reporter: Federico Mariani
Attachments: camel-aws-review-2026-07.md
Umbrella for the high-severity findings of a deep code review of camel-aws2-s3,
-sqs, -sns, -kinesis (incl. firehose) and -ddb (incl. ddbstream) on main
(4.22.0-SNAPSHOT). The full findings document (with failure scenarios and
suggested fixes per item) is attached. The two critical findings are tracked
separately with failing reproducer tests attached: the S3 streaming truncation
and the DDB Streams resharding data loss.
Individual findings can be split out into their own issues as they are picked
up.
*camel-aws2-s3*
# {{AWS2S3Producer:645}} — {{operation=getObject&ignoreBody=true}} leaks the
ResponseInputStream; connection pool exhausted after ~50 exchanges (CAMEL-21116
follow-up).
# {{AWS2S3Producer:222-305}} — {{processMultiPart}} computes the dynamic bucket
name ({{CamelAwsS3OverrideBucketName}} / simple expressions) but builds all
four multipart requests with the raw configured bucket; behavior silently
diverges by payload size.
# {{AWS2S3Producer:187-217}} — FileInputStream opened for File bodies is never
closed on the {{contentLength < partSize}} shortcut; one fd leaked per small
file in multiPartUpload mode.
*camel-aws2-sqs*
# {{Sqs2Producer:270-300}} — per-message {{DelaySeconds}} is applied even when
{{delayQueue=true}}; on FIFO queues every send fails with
InvalidParameterValue, on standard queues the queue-level option is defeated.
*camel-aws2-sns*
# {{Sns2Endpoint:147-150}} — auto-created FIFO topics never get
{{ContentBasedDeduplication}}, so
{{messageDeduplicationIdStrategy=useContentBasedDeduplication}} makes every
publish fail (SQS twin handles this correctly).
*camel-aws2-kinesis*
# {{Kinesis2Configuration:71}} — {{shardClosed}} default is only declared in
the annotation; the field is null and {{handleClosedShard}} NPEs on the first
shard rotation with default config.
# {{Kinesis2Consumer:176-297}} — expired shard iterators are never invalidated
(no ExpiredIteratorException handling in the module); any >5-minute gap stalls
the consumer permanently.
# {{KinesisResumeAction:62-67}} + {{KinesisResumeStrategy:60-63}} — resume
support broken for multi-shard streams: evalEntry overwrites the request with
every cache entry, KinesisOffset values hit a String cast (ClassCastException),
and the single registry action is mutated concurrently from parallelStream.
# {{KinesisConnection:38-64}} + {{Kinesis2Endpoint:80-93}} — the component-wide
connection caches the first endpoint's client for all endpoints (second
endpoint's region/credentials silently ignored) and {{doStop}} closes the
shared client without resetting the cache.
# {{KclKinesis2Consumer:193-232}} — KCL consumer never checkpoints during
processing, swallows all processing Throwables (not even logged with stack
trace), then checkpoints unconditionally at shutdown — data loss on failure,
mass reprocessing on crash.
# {{KclKinesis2Consumer:94-133}} — session-credentials branch is unreachable
({{accessKey && secretKey && sessionToken}} tested after {{accessKey &&
secretKey}}); KCL DynamoDB/CloudWatch clients fail with STS temporary
credentials.
*camel-aws2-ddb (ddbstream)*
# {{Ddb2StreamConsumer:65,93}} + {{ShardIteratorHandler:77-86}} — internal
iterator map returned by reference and structurally modified during poll
iteration -> ConcurrentModificationException whenever a shard retires while >=2
shards are tracked.
# {{ShardIteratorHandler:55-57}} — DescribeStream pagination
({{lastEvaluatedShardId}}) ignored; streams with >100 shards silently lose
partitions.
# {{ShardIteratorHandler:88-92}} — expired-iterator recovery sends
{{AFTER_SEQUENCE_NUMBER}} with null sequence when no record was seen; consumer
permanently stuck failing.
*camel-aws2-ddb (transformer)*
# {{Ddb2JsonDataTypeTransformer:197-247}} — Long/BigInteger/BigDecimal values
(e.g. epoch millis) and numeric arrays are stored as DynamoDB String/StringSet
types instead of N/NS (CAMEL-22038 only fixed scalar Double).
_This issue was researched and filed by Claude Code (AI) on behalf of Federico
Mariani (fmariani). Full findings document from the review is attached to the
umbrella issues._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
