shashank created CAMEL-25903:
--------------------------------
Summary: camel-aws2-s3 - with multiPartUpload=true, an object
uploaded in parts is stored without the user metadata of the CamelAwsS3Metadata
header
Key: CAMEL-25903
URL: https://issues.apache.org/jira/browse/CAMEL-25903
Project: Camel
Issue Type: Bug
Components: camel-aws2-s3
Reporter: shashank
Assignee: shashank
With {{multiPartUpload=true}}, {{AWS2S3Producer.processMultiPart}} (main
74ee085f7d64, lines 172-347) uploads a body that is at least {{partSize}} long
(default 25 MB) with {{CreateMultipartUpload}} / {{UploadPart}} /
{{CompleteMultipartUpload}}, and a smaller body with a single {{PutObject}}
({{doPutObject}}). Both start from the same user metadata, computed from the
{{CamelAwsS3Metadata}} header:
{code:java}
Map<String, String> objectMetadata = determineMetadata(exchange);
...
objectMetadata.put("Content-Length", "" + contentLength);
...
CreateMultipartUploadRequest.Builder createMultipartUploadRequest
=
CreateMultipartUploadRequest.builder().bucket(bucketName).key(keyName);
{code}
{{doPutObject}} sets it on the request
({{putObjectRequest...metadata(objectMetadata)}}), but the multipart path never
puts the map on the {{CreateMultipartUploadRequest}}: an object uploaded in
parts is stored without any user metadata. The upload succeeds and nothing is
logged. Storage class, ACL, content type, cache control, content disposition,
content encoding and the encryption options are all applied on that path; only
the metadata is left out.
So with {{multiPartUpload=true}} whether an object gets its metadata depends on
its size. The S3 consumer sets {{CamelAwsS3Metadata}} to the user metadata of
the object it read, so a route that copies objects from one bucket to another
with multipart upload keeps the metadata of the small objects and silently
drops it for the large ones. CAMEL-17713 added user metadata to the producer
for integrations that rely on it; the multipart path has never had it (the same
lines are in 3.22.x).
*Fix*: {{processMultiPart}} sets the metadata on the
{{CreateMultipartUploadRequest}}. The {{Content-Length}} entry that the
multipart path added to the map (never sent anywhere) is removed, so it does
not become an {{x-amz-meta-content-length}} entry. No option or documentation
change; no upgrade note (objects now get the metadata the route asked for).
*Tests*: new {{AWS2S3ProducerMultipartMetadataTest}} (2 tests, Mockito
{{S3Client}} as the other producer tests of the module, {{partSize=5}}).
{{multipartUploadStoresTheUserMetadata}} (a 12 byte body, 3 parts) fails on
main with {{expected: {"checksum"="abc123", "origin"="camel"} but was: {}}}.
{{bodySmallerThanThePartSizeStoresTheUserMetadata}} is a control: the single
put keeps the metadata on main and with the fix. camel-aws2-s3 unit tests: 118
run, 0 failures (run with the AWS SDK 2.55.15 jars of the local repository,
{{-Daws-java-sdk2-version=2.55.15}}; main uses 2.56.0).
*How it was found*: comparing, per producer, the request options applied on
each code path of one operation (single vs multipart, sync vs async, batch vs
single). The JUnit tests above are the evidence; nothing was run against AWS S3.
*Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x
(the same {{CreateMultipartUploadRequest}} without metadata; read, not run).
Not changed, possible follow-ups: with {{useCustomerKey=true}} (SSE-C) the
{{UploadPartRequest}} of the multipart path does not carry the customer key
either, which S3 requires for each part, so such uploads fail (loudly).
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
