[
https://issues.apache.org/jira/browse/CAMEL-25743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128798#comment-18128798
]
Claus Ibsen commented on CAMEL-25743:
-------------------------------------
Merged to main via https://github.com/apache/camel/pull/28070 (66a07e331869).
_Claude Code on behalf of Claus Ibsen_
> camel-aws2-s3 - the producer uploads a body of known length with the length
> from a stale CamelAwsS3ContentLength header, so a body the route made longer
> is stored cut to the old length
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25743
> URL: https://issues.apache.org/jira/browse/CAMEL-25743
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-s3
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> The aws2-s3 producer takes the content length from the
> {{CamelAwsS3ContentLength}} header whenever it is set, and only looks at the
> body when it is not ({{AWS2S3Producer.processSingleOp}}, main 55ed20f14e90,
> lines 354 and 371-375; {{processMultiPart}}, lines 178 and 193-197):
> {code:java}
> long contentLength =
> exchange.getIn().getHeader(AWS2S3Constants.CONTENT_LENGTH, -1, Long.class);
> ...
> if (contentLength <= 0) {
> // such as a java.nio.file.Path, byte[] or stream cache
> contentLength = PayloadHelper.getLength(obj);
> }
> {code}
> The length is then given to the SDK
> ({{RequestBody.fromInputStream(inputStream, contentLength)}} and
> {{PutObjectRequest.contentLength}}, lines 461 and 510), which sends exactly
> that many bytes: {{MakeHttpRequestStage}} (sdk-core) wraps the content in
> {{software.amazon.awssdk.utils.io.LengthAwareInputStream}}, which returns end
> of stream after {{contentLength}} bytes and throws "The request content has
> fewer bytes than the specified content-length" for a shorter body (SDK
> 2.55.15, the version on main). Checked end to end with the real {{S3Client}}
> against a local HTTP server: a 31-byte String body with the header 10 arrives
> as the 10 bytes {{a body mad}}, and the exchange completes without an error.
> The aws2-s3 consumer sets {{CamelAwsS3ContentLength}} to the length of the
> object it read ({{AWS2S3Consumer}} line 584; {{getObject}} does the same,
> producer line 1553), and the header stays on the message when the route
> changes the body. So a route like
> {code:java}
> from("aws2-s3://incoming?...")
> .convertBodyTo(String.class)
> .transform(simple("${body}\n# processed"))
> .to("aws2-s3://processed?...");
> {code}
> (or an enrich, marshal or template step between the two) stores the new body
> cut to the length of the original object, and the exchange completes
> normally. A body made shorter fails in the SDK. The truncation is silent and
> the object in the target bucket is corrupt.
> The header exists for a stream whose length is not known: CAMEL-4126 added it
> because "With InputStreams, Amazon's Java libraries aren't able to generate
> the content length", and CAMEL-13433 stopped copying a big stream into memory
> when the header was set. CAMEL-25167 says "Existing length headers
> ({{CamelAwsS3ContentLength}}, {{CamelAzureStorageBlobUploadSize}}, ...) keep
> precedence"; its change kept the order that was there and did not discuss a
> header that no longer matches the body. This issue proposes to narrow that
> precedence to bodies whose length is not known; if the precedence is meant to
> hold for every body, the alternative is a documentation note (remove the
> {{CamelAwsS3*}} headers before writing a changed body). For a {{byte[]}},
> {{String}}, file, {{Path}} or cached stream the length is known without
> reading the body, and CAMEL-25167's own length helper
> ({{PayloadHelper.getBodyLength}}) does not use {{CamelFileLength}} "as it can
> be stale once the route changed the body".
> h3. Reproduction
> New tests in {{AWS2S3ProducerPayloadLengthTest}} (mocked {{S3Client}}):
> * {{bodyLongerThanAStaleContentLengthHeaderIsStoredWhole}}: a 31-character
> String body with the header 10 is uploaded with length 10 ({{expected: <31>
> but was: <10>}}).
> * {{bodyShorterThanAStaleContentLengthHeaderIsStoredWithItsLength}}: a 5-byte
> body with the header 1000 ({{expected: <5> but was: <1000>}}).
> * {{multiPartUploadOfABodyLongerThanAStaleContentLengthHeader}}:
> {{multiPartUpload=true}}, a 1 MB {{byte[]}} with the header 10 ({{expected:
> <1048576> but was: <10>}}).
> All three fail on main, 3 of 3 runs. A fourth test,
> {{streamOfUnknownLengthUsesTheContentLengthHeader}}, passes on main and with
> the fix; it guards the case the header is for. The end-to-end probe above
> (real SDK client, local HTTP server; not part of the change) shows the same
> with the fix: the 31 bytes arrive.
> h3. Proposed fix
> When the length of the body is known without reading it
> ({{PayloadHelper.getLength}} of the body or of its input stream), that length
> is used; the header is only used for a stream whose length is not known, as
> before. Same change in the single and the multi-part upload. Upgrade note in
> the 4.24 guide.
> With the fix the camel-aws2-s3 tests pass: 108 run, 0 failures, 0 errors, 0
> skipped.
> Not changed: a stale header on a stream whose length is not known (for
> example {{includeBody=false}} and a route that wraps the stream) is still
> used, as there is nothing else to measure without copying the stream. Same
> pattern, not changed here: camel-minio ({{MinioProducer}}), camel-ibm-cos and
> camel-google-storage give their own content-length header precedence too
> (possible follow-ups); camel-azure-files with {{CamelFileLength}} is a
> separate issue.
> Affected: main (tested); camel-4.23.x, camel-4.22.x and camel-4.18.x give the
> header the same precedence (read, not run).
> Duplicate check (2026-10-17): JIRA text "CamelAwsS3ContentLength", "aws2-s3
> content-length header stale": CAMEL-4126, CAMEL-12121 (Not A Problem, a
> constant header in a route), CAMEL-13433, CAMEL-25167; none about a stale
> header from the consumer. GitHub pull requests "AWS2S3Producer content
> length": #27132 (CAMEL-25167), #21122.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
