shashank created CAMEL-25743:
--------------------------------
Summary: camel-aws2-s3 - the producer uploads a body of known
length with the length from a stale CamelAwsS3ContentLength header, so a body
the route made longer is stored cut to the old length
Key: CAMEL-25743
URL: https://issues.apache.org/jira/browse/CAMEL-25743
Project: Camel
Issue Type: Bug
Components: camel-aws2-s3
Reporter: shashank
Assignee: shashank
The aws2-s3 producer takes the content length from the
{{CamelAwsS3ContentLength}} header whenever it is set, and only looks at the
body when it is not ({{AWS2S3Producer.processSingleOp}}, main 55ed20f14e90,
lines 354 and 371-375; {{processMultiPart}}, lines 178 and 193-197):
{code:java}
long contentLength = exchange.getIn().getHeader(AWS2S3Constants.CONTENT_LENGTH,
-1, Long.class);
...
if (contentLength <= 0) {
// such as a java.nio.file.Path, byte[] or stream cache
contentLength = PayloadHelper.getLength(obj);
}
{code}
The length is then given to the SDK ({{RequestBody.fromInputStream(inputStream,
contentLength)}} and {{PutObjectRequest.contentLength}}, lines 461 and 510),
which sends exactly that many bytes: {{MakeHttpRequestStage}} (sdk-core) wraps
the content in {{software.amazon.awssdk.utils.io.LengthAwareInputStream}},
which returns end of stream after {{contentLength}} bytes and throws "The
request content has fewer bytes than the specified content-length" for a
shorter body (SDK 2.55.15, the version on main). Checked end to end with the
real {{S3Client}} against a local HTTP server: a 31-byte String body with the
header 10 arrives as the 10 bytes {{a body mad}}, and the exchange completes
without an error.
The aws2-s3 consumer sets {{CamelAwsS3ContentLength}} to the length of the
object it read ({{AWS2S3Consumer}} line 584; {{getObject}} does the same,
producer line 1553), and the header stays on the message when the route changes
the body. So a route like
{code:java}
from("aws2-s3://incoming?...")
.convertBodyTo(String.class)
.transform(simple("${body}\n# processed"))
.to("aws2-s3://processed?...");
{code}
(or an enrich, marshal or template step between the two) stores the new body
cut to the length of the original object, and the exchange completes normally.
A body made shorter fails in the SDK. The truncation is silent and the object
in the target bucket is corrupt.
The header exists for a stream whose length is not known: CAMEL-4126 added it
because "With InputStreams, Amazon's Java libraries aren't able to generate the
content length", and CAMEL-13433 stopped copying a big stream into memory when
the header was set. CAMEL-25167 says "Existing length headers
({{CamelAwsS3ContentLength}}, {{CamelAzureStorageBlobUploadSize}}, ...) keep
precedence"; its change kept the order that was there and did not discuss a
header that no longer matches the body. This issue proposes to narrow that
precedence to bodies whose length is not known; if the precedence is meant to
hold for every body, the alternative is a documentation note (remove the
{{CamelAwsS3*}} headers before writing a changed body). For a {{byte[]}},
{{String}}, file, {{Path}} or cached stream the length is known without reading
the body, and CAMEL-25167's own length helper ({{PayloadHelper.getBodyLength}})
does not use {{CamelFileLength}} "as it can be stale once the route changed the
body".
h3. Reproduction
New tests in {{AWS2S3ProducerPayloadLengthTest}} (mocked {{S3Client}}):
* {{bodyLongerThanAStaleContentLengthHeaderIsStoredWhole}}: a 31-character
String body with the header 10 is uploaded with length 10 ({{expected: <31> but
was: <10>}}).
* {{bodyShorterThanAStaleContentLengthHeaderIsStoredWithItsLength}}: a 5-byte
body with the header 1000 ({{expected: <5> but was: <1000>}}).
* {{multiPartUploadOfABodyLongerThanAStaleContentLengthHeader}}:
{{multiPartUpload=true}}, a 1 MB {{byte[]}} with the header 10 ({{expected:
<1048576> but was: <10>}}).
All three fail on main, 3 of 3 runs. A fourth test,
{{streamOfUnknownLengthUsesTheContentLengthHeader}}, passes on main and with
the fix; it guards the case the header is for. The end-to-end probe above (real
SDK client, local HTTP server; not part of the change) shows the same with the
fix: the 31 bytes arrive.
h3. Proposed fix
When the length of the body is known without reading it
({{PayloadHelper.getLength}} of the body or of its input stream), that length
is used; the header is only used for a stream whose length is not known, as
before. Same change in the single and the multi-part upload. Upgrade note in
the 4.24 guide.
With the fix the camel-aws2-s3 tests pass: 108 run, 0 failures, 0 errors, 0
skipped.
Not changed: a stale header on a stream whose length is not known (for example
{{includeBody=false}} and a route that wraps the stream) is still used, as
there is nothing else to measure without copying the stream. Same pattern, not
changed here: camel-minio ({{MinioProducer}}), camel-ibm-cos and
camel-google-storage give their own content-length header precedence too
(possible follow-ups); camel-azure-files with {{CamelFileLength}} is a separate
issue.
Affected: main (tested); camel-4.23.x, camel-4.22.x and camel-4.18.x give the
header the same precedence (read, not run).
Duplicate check (2026-10-17): JIRA text "CamelAwsS3ContentLength", "aws2-s3
content-length header stale": CAMEL-4126, CAMEL-12121 (Not A Problem, a
constant header in a route), CAMEL-13433, CAMEL-25167; none about a stale
header from the consumer. GitHub pull requests "AWS2S3Producer content length":
#27132 (CAMEL-25167), #21122.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
