Federico Mariani created CAMEL-24153:
----------------------------------------
Summary: camel-aws2-s3 - Streaming upload silently truncates
bodies larger than partSize and orphans multipart uploads
Key: CAMEL-24153
URL: https://issues.apache.org/jira/browse/CAMEL-24153
Project: Camel
Issue Type: Bug
Components: camel-aws2-s3
Reporter: Federico Mariani
Attachments: AWS2S3StreamUploadProducerTruncationTest.java
Found during a code review of camel-aws2-s3 on main (4.22.0-SNAPSHOT).
In {{AWS2S3StreamUploadProducer.processWithoutTimestampGrouping}} (lines
~179-297), {{maxRead}} starts at {{partSize}} and is decremented after every
read. It is reset back to {{partSize}} only on the *part-flush* branches (lines
198 and 288). When the *batch-flush* branch fires instead (line 271:
{{totalSize >= batchSize}} -> {{uploadPart(state); completeUpload(state); state
= null; continue;}}), {{maxRead}} stays {{0}}, so the next loop iteration calls
{{AWS2S3Utils.toByteArray(is, 0)}}, which reads nothing, and the loop exits
with the remainder of the input stream unread. The exchange still completes
successfully with the ETag of the truncated object.
Note this happens with the *default* configuration ({{batchSize=1000000}} <
{{partSize=26214400}}): any body larger than partSize is truncated to the first
part.
Additionally, line 183 ({{maxRead -= uploadAggregate.buffer.size()}}) can drive
{{maxRead}} negative when {{batchSize > bufferSize}}, dropping an entire
subsequent exchange body.
*Secondary bug in the same loop*: since CAMEL-20728 changed {{state.index++}}
to {{state.index = 1}} (line 224), the guard {{if (state.index == 1)}} at line
265 no longer means "first iteration", so {{createMultipartUpload}} is invoked
on *every* read iteration. Parts of a single logical upload are spread across
several uploadIds; {{completeMultipartUpload}} then fails with {{InvalidPart}},
and the superseded uploads are never aborted (incomplete multipart uploads
accrue storage charges indefinitely).
*Reproducer*: attached {{AWS2S3StreamUploadProducerTruncationTest.java}} (pure
unit test, mocked S3Client, goes under
{{components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/stream/}}).
It sends a 12288-byte body with {{partSize=8192}}, {{batchSize=4096}} and
fails with:
{noformat}
expected: <12288> but was: <8192>
{noformat}
The existing IT (S3StreamUploadS3MultipartIT) only sends bodies smaller than
partSize, which is why this path is untested.
*Suggested fix*: reset {{maxRead}} after every flush (and clamp to a positive
read size), and guard the upload creation with {{if (state.initResponse ==
null)}} instead of {{state.index == 1}}.
_This issue was researched and filed by Claude Code (AI) on behalf of Federico
Mariani (fmariani). Full findings document from the review is attached to the
umbrella issues._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
