allthingssecurity opened a new pull request, #29122: URL: https://github.com/apache/camel/pull/29122
## CAMEL-26539: camel-aws2-s3 - streaming upload writes concurrent exchanges one at a time With `streamingUploadMode=true` (no timestamp grouping), an exchange that finds no open upload starts one of its own outside the producer lock and publishes it only at its end (CAMEL-24195 moved the check under the lock, but not this). Two exchanges sent at the same time (concurrent consumers, parallel splits, `threads()`) can both do that: - with progressive naming (the default) both uploads get the same key, so one object replaces the other; with the defaults and two threads the 10 messages of the first object are lost, although their exchanges completed without an error (the one that completed the object even got its ETag); - with `multiPartUpload=true` the first part of a large body goes to an upload that is neither completed nor aborted. Fix: `processWithoutTimestampGrouping` holds the lock for the whole exchange, so the "potential race condition" copy branch is gone. Appended exchanges already made their S3 calls under the lock; what is new is that reading a body and the S3 calls of an exchange that starts an upload no longer run in parallel within one producer (upgrade guide). A stop (and the `streamingUploadTimeout` task) now waits for an exchange that is still reading its body instead of completing the upload under it. Tests: `AWS2S3StreamUploadConcurrentExchangesTest` holds the first `createMultipartUpload` on a latch while other exchanges arrive; both tests fail on main and pass with the fix (class repeated 20x), module unit tests pass. Found with a SPIN model; a 4-thread stress probe lost 53-114 of 1000 messages per run on main, none with the fix. Follow-ups: CAMEL-YYYYY (`doStop` does not clear the open upload, so a producer stopped and started again appends to a completed upload), CAMEL-ZZZZZ (the timestamp grouping path counts the concurrent exchanges of a window outside the lock and can interleave their bodies). Affected: main, 4.23.x, 4.22.x (cherry-pick of the code change); 4.18.x/4.14.x need an adapted backport. _Claude Code on behalf of allthingssecurity_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
