This is an automated email from the ASF dual-hosted git repository.

davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git


The following commit(s) were added to refs/heads/main by this push:
     new 2e2bb4771ed8 CAMEL-27015: camel-aws2-s3, camel-azure-storage-blob, 
camel-aws2-sqs - the consumers delete or move the object they consumed, 
whatever the route did with its headers (#30168)
2e2bb4771ed8 is described below

commit 2e2bb4771ed886f173f07fe5ab60012b9f8f8424
Author: Federico Mariani <[email protected]>
AuthorDate: Mon Nov 16 16:08:58 2026 +0100

    CAMEL-27015: camel-aws2-s3, camel-azure-storage-blob, camel-aws2-sqs - the 
consumers delete or move the object they consumed, whatever the route did with 
its headers (#30168)
    
    - With `deleteAfterRead` or `moveAfterRead`, the `aws2-s3` and 
`azure-storage-blob` consumers took the key, the bucket and the blob name from 
the headers when the exchange completed, and `aws2-sqs` took the receipt handle 
from its header. A route that set these headers (for example to write a copy of 
the object under another name) deleted or moved that copy. A route that removed 
them, or replaced the message, left the object or the message in place, and it 
was consumed again.
    - The consumers now take the key (from the exchange property the S3 
consumer sets), the blob name and the receipt handle before the exchange is 
routed, and pass them to a new `processCommit(Exchange, String)`. The S3 move 
copies from the bucket of the endpoint, as the delete already did. 
`processCommit(Exchange)` is kept for subclasses and deprecated, since the 
consumers no longer call it. The WARN of a blob without a name no longer says 
that the name is missing from the headers.
    - `AWS2S3Consumer.processCommit` catches `SdkException` instead of 
`AwsServiceException`, so a failure raised by the client itself reaches the 
exception handler of the consumer.
    - Same approach as CAMEL-25945 for `azure-storage-queue`.
    
    _Claude Code on behalf of Federico Mariani_
    
    Co-Authored-By: Claude <[email protected]>
---
 .../camel/component/aws2/s3/AWS2S3Consumer.java    |  37 +--
 .../s3/AWS2S3ConsumerDeleteMoveAfterReadTest.java  | 200 ++++++++++++++++
 .../camel/component/aws2/s3/FakeS3Server.java      |  57 ++++-
 .../camel/component/aws2/sqs/Sqs2Consumer.java     |  23 +-
 .../aws2/sqs/SqsConsumerDeleteMessageTest.java     | 105 ++++++++
 .../component/azure/storage/blob/BlobConsumer.java |  22 +-
 .../blob/BlobConsumerDeleteMoveAfterReadTest.java  | 265 +++++++++++++++++++++
 .../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc    |  20 ++
 8 files changed, 702 insertions(+), 27 deletions(-)

diff --git 
a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Consumer.java
 
b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Consumer.java
index 18c778104f87..b4ce3c482970 100644
--- 
a/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Consumer.java
+++ 
b/components/camel-aws/camel-aws2-s3/src/main/java/org/apache/camel/component/aws2/s3/AWS2S3Consumer.java
@@ -44,6 +44,7 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import software.amazon.awssdk.awscore.exception.AwsServiceException;
 import software.amazon.awssdk.core.ResponseInputStream;
+import software.amazon.awssdk.core.exception.SdkException;
 import software.amazon.awssdk.services.s3.S3Client;
 import software.amazon.awssdk.services.s3.model.CopyObjectRequest;
 import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
@@ -401,12 +402,15 @@ public class AWS2S3Consumer extends 
ScheduledBatchPollingConsumer {
             exchange.setProperty(ExchangePropertyKey.BATCH_SIZE, total);
             exchange.setProperty(ExchangePropertyKey.BATCH_COMPLETE, index == 
total - 1);
 
+            // the object to delete or move once the exchange is done. It is 
taken now, as the route may change or
+            // remove the key and bucket headers before it ends (the producer 
takes the key of the object to write from
+            // the key header)
+            final String key = exchange.getProperty(AWS2S3Constants.KEY, 
String.class);
             // add on completion to handle after work when the exchange is done
             exchange.getExchangeExtension().addOnCompletion(new 
Synchronization() {
                 public void onComplete(Exchange exchange) {
-                    final String key = 
exchange.getProperty(AWS2S3Constants.KEY, String.class);
                     try {
-                        processCommit(exchange);
+                        processCommit(exchange, key);
                     } finally {
                         if (key != null) {
                             
getEndpoint().getInProgressRepository().remove(key);
@@ -415,7 +419,6 @@ public class AWS2S3Consumer extends 
ScheduledBatchPollingConsumer {
                 }
 
                 public void onFailure(Exchange exchange) {
-                    final String key = 
exchange.getProperty(AWS2S3Constants.KEY, String.class);
                     try {
                         processRollback(exchange);
                     } finally {
@@ -481,16 +484,26 @@ public class AWS2S3Consumer extends 
ScheduledBatchPollingConsumer {
     }
 
     /**
-     * Strategy to delete the message after being processed.
+     * Strategy to delete or move the object after being processed, the object 
the exchange was created from.
      *
-     * @param exchange the exchange
+     * @param      exchange the exchange
+     * @deprecated          use {@link #processCommit(Exchange, String)}, 
which the consumer calls instead
      */
+    @Deprecated(since = "4.24.0")
     protected void processCommit(Exchange exchange) {
+        processCommit(exchange, exchange.getProperty(AWS2S3Constants.KEY, 
String.class));
+    }
+
+    /**
+     * Strategy to delete or move the object after being processed.
+     *
+     * @param exchange the exchange
+     * @param key      the key of the object the exchange was created from, in 
the bucket of the endpoint
+     */
+    protected void processCommit(Exchange exchange, String key) {
+        final String bucketName = getConfiguration().getBucketName();
         try {
             if (getConfiguration().isMoveAfterRead()) {
-                String bucketName = 
exchange.getIn().getHeader(AWS2S3Constants.BUCKET_NAME, String.class);
-                String key = exchange.getIn().getHeader(AWS2S3Constants.KEY, 
String.class);
-
                 LOG.trace("Moving object from bucket {} with key {} to bucket 
{}...", bucketName, key,
                         getConfiguration().getDestinationBucket());
 
@@ -517,17 +530,13 @@ public class AWS2S3Consumer extends 
ScheduledBatchPollingConsumer {
                         getConfiguration().getDestinationBucket());
             }
             if (getConfiguration().isDeleteAfterRead()) {
-                String bucketName = 
exchange.getIn().getHeader(AWS2S3Constants.BUCKET_NAME, String.class);
-                String key = exchange.getIn().getHeader(AWS2S3Constants.KEY, 
String.class);
-
                 LOG.trace("Deleting object from bucket {} with key {}...", 
bucketName, key);
 
-                getAmazonS3Client().deleteObject(
-                        
DeleteObjectRequest.builder().bucket(getConfiguration().getBucketName()).key(key).build());
+                
getAmazonS3Client().deleteObject(DeleteObjectRequest.builder().bucket(bucketName).key(key).build());
 
                 LOG.trace("Deleted object from bucket {} with key {}...", 
bucketName, key);
             }
-        } catch (AwsServiceException e) {
+        } catch (SdkException e) {
             getExceptionHandler().handleException("Error occurred during 
moving or deleting object. This exception is ignored.",
                     exchange, e);
         }
diff --git 
a/components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/AWS2S3ConsumerDeleteMoveAfterReadTest.java
 
b/components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/AWS2S3ConsumerDeleteMoveAfterReadTest.java
new file mode 100644
index 000000000000..b2e3e78ff5ef
--- /dev/null
+++ 
b/components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/AWS2S3ConsumerDeleteMoveAfterReadTest.java
@@ -0,0 +1,200 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.aws2.s3;
+
+import java.util.List;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.function.Consumer;
+
+import org.apache.camel.CamelContext;
+import org.apache.camel.Exchange;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.model.RouteDefinition;
+import org.apache.camel.spi.ExceptionHandler;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+import software.amazon.awssdk.core.exception.SdkClientException;
+
+import static org.apache.camel.builder.Builder.constant;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
+
+/**
+ * With deleteAfterRead or moveAfterRead the consumer deletes or moves the 
object it consumed once the exchange is done,
+ * whatever the route did with the CamelAwsS3Key and CamelAwsS3BucketName 
headers in the meantime: a route may set them
+ * to write a copy of the object under another key, or remove the Camel 
headers before it calls another system.
+ * <p>
+ * The scheduler of the consumer is not started: the tests call {@code poll()} 
themselves. The routes are synchronous,
+ * so the delete or the move is done when {@code poll()} returns.
+ */
+class AWS2S3ConsumerDeleteMoveAfterReadTest extends CamelTestSupport {
+
+    private FakeS3Server s3;
+
+    @Override
+    protected CamelContext createCamelContext() throws Exception {
+        s3 = new FakeS3Server("bkt");
+        s3.start();
+        return super.createCamelContext();
+    }
+
+    @AfterEach
+    void stopServer() {
+        if (s3 != null) {
+            s3.stop();
+        }
+    }
+
+    @Test
+    void 
deleteAfterReadDeletesTheConsumedObjectWhenTheRouteChangesTheKeyHeader() throws 
Exception {
+        s3.put("a.txt", "a");
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("a");
+        // for example the key under which the route writes a copy of the 
object
+        AWS2S3Consumer consumer = startRoute("", route -> 
route.setHeader(AWS2S3Constants.KEY, constant("out/a.txt")));
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("a.txt"), s3.deletes());
+        // so the next poll does not consume it again
+        assertEquals(0, consumer.poll());
+    }
+
+    @Test
+    void 
deleteAfterReadDeletesTheConsumedObjectWhenTheRouteRemovesTheHeaders() throws 
Exception {
+        s3.put("a.txt", "a");
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("a");
+        AWS2S3Consumer consumer = startRoute("", route -> 
route.removeHeaders("CamelAwsS3*"));
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("a.txt"), s3.deletes());
+        assertEquals(0, consumer.poll());
+    }
+
+    @Test
+    void moveAfterReadMovesTheConsumedObjectWhenTheRouteChangesTheHeaders() 
throws Exception {
+        s3.put("a.txt", "a");
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("a");
+        AWS2S3Consumer consumer = 
startRoute("&moveAfterRead=true&destinationBucket=archive",
+                route -> route.setHeader(AWS2S3Constants.BUCKET_NAME, 
constant("other"))
+                        .setHeader(AWS2S3Constants.KEY, 
constant("out/a.txt")));
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("bkt/a.txt -> archive/a.txt"), s3.copies());
+        assertEquals(List.of("a.txt"), s3.deletes());
+    }
+
+    @Test
+    void deleteAfterReadDeletesTheConsumedObject() throws Exception {
+        // control: the route keeps the headers the consumer set
+        s3.put("a.txt", "a");
+        s3.put("b.txt", "b");
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("a", "b");
+        AWS2S3Consumer consumer = startRoute("", route -> {
+        });
+
+        assertEquals(2, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("a.txt", "b.txt"), s3.deletes());
+        assertEquals(0, consumer.poll());
+    }
+
+    @Test
+    void moveAfterReadMovesTheConsumedObject() throws Exception {
+        // control: the destination key is built from the key of the consumed 
object
+        s3.put("a.txt", "a");
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("a");
+        AWS2S3Consumer consumer = startRoute(
+                
"&moveAfterRead=true&destinationBucket=archive&destinationBucketPrefix=done/&destinationBucketSuffix=.bak",
+                route -> {
+                });
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("bkt/a.txt -> archive/done/a.txt.bak"), 
s3.copies());
+        assertEquals(List.of("a.txt"), s3.deletes());
+    }
+
+    @Test
+    void deleteThatCannotBeSentIsReportedToTheExceptionHandler() throws 
Exception {
+        s3.put("a.txt", "a");
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("a");
+        // S3 cannot be reached when the exchange is done: the client fails 
without an answer from S3
+        AWS2S3Consumer consumer = startRoute("", route -> 
route.process(exchange -> s3.stop()));
+        RecordingExceptionHandler handler = new RecordingExceptionHandler();
+        consumer.setExceptionHandler(handler);
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("Error occurred during moving or deleting object. 
This exception is ignored."),
+                handler.messages);
+        assertInstanceOf(SdkClientException.class, handler.exceptions.get(0));
+    }
+
+    private AWS2S3Consumer startRoute(String options, 
Consumer<RouteDefinition> steps) throws Exception {
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                RouteDefinition route
+                        = from(
+                        
"aws2-s3://bkt?accessKey=xxx&secretKey=yyy&region=us-east-1&overrideEndpoint=true"
+                                             + 
"&forcePathStyle=true&uriEndpointOverride=" + s3.endpoint()
+                                             + "&startScheduler=false" + 
options)
+                        .routeId("s3");
+                steps.accept(route);
+                route.convertBodyTo(String.class).to("mock:result");
+            }
+        });
+        return (AWS2S3Consumer) context.getRoute("s3").getConsumer();
+    }
+
+    private static final class RecordingExceptionHandler implements 
ExceptionHandler {
+        final List<String> messages = new CopyOnWriteArrayList<>();
+        final List<Throwable> exceptions = new CopyOnWriteArrayList<>();
+
+        @Override
+        public void handleException(Throwable exception) {
+            handleException(null, null, exception);
+        }
+
+        @Override
+        public void handleException(String message, Throwable exception) {
+            handleException(message, null, exception);
+        }
+
+        @Override
+        public void handleException(String message, Exchange exchange, 
Throwable exception) {
+            messages.add(String.valueOf(message));
+            exceptions.add(exception);
+        }
+    }
+}
diff --git 
a/components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/FakeS3Server.java
 
b/components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/FakeS3Server.java
index 0cee9c222835..48d7a820f4f0 100644
--- 
a/components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/FakeS3Server.java
+++ 
b/components/camel-aws/camel-aws2-s3/src/test/java/org/apache/camel/component/aws2/s3/FakeS3Server.java
@@ -29,6 +29,7 @@ import java.util.NavigableMap;
 import java.util.Set;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentSkipListMap;
+import java.util.concurrent.CopyOnWriteArrayList;
 import java.util.concurrent.atomic.AtomicInteger;
 
 import com.sun.net.httpserver.HttpExchange;
@@ -36,9 +37,9 @@ import com.sun.net.httpserver.HttpServer;
 
 /**
  * A minimal in-process fake of the S3 API calls the consumer makes for a 
single bucket (path-style requests): head
- * bucket, list objects (v2, paginated with max-keys and 
start-after/continuation-token), head, get and delete object.
- * An object can be removed right after it was listed, the gets (and heads) of 
an object can be refused, and the body of
- * an object can be cut short.
+ * bucket, list objects (v2, paginated with max-keys and 
start-after/continuation-token), head, get, delete and copy
+ * object. A copy to another bucket is only recorded. An object can be removed 
right after it was listed, the gets (and
+ * heads) of an object can be refused, and the body of an object can be cut 
short.
  */
 final class FakeS3Server {
 
@@ -47,6 +48,8 @@ final class FakeS3Server {
     private final NavigableMap<String, byte[]> objects = new 
ConcurrentSkipListMap<>();
     private final Map<String, AtomicInteger> gets = new ConcurrentHashMap<>();
     private final Map<String, AtomicInteger> heads = new ConcurrentHashMap<>();
+    private final List<String> deletes = new CopyOnWriteArrayList<>();
+    private final List<String> copies = new CopyOnWriteArrayList<>();
     private final Set<String> removedWhenListed = 
ConcurrentHashMap.newKeySet();
     private final Set<String> refused = ConcurrentHashMap.newKeySet();
     private final Map<String, AtomicInteger> refusedGets = new 
ConcurrentHashMap<>();
@@ -128,9 +131,26 @@ final class FakeS3Server {
         return n != null ? n.get() : 0;
     }
 
+    /**
+     * The keys of the delete object requests, in order.
+     */
+    List<String> deletes() {
+        return List.copyOf(deletes);
+    }
+
+    /**
+     * The copy object requests, in order, as {@code sourceBucket/sourceKey -> 
destinationBucket/destinationKey}.
+     */
+    List<String> copies() {
+        return List.copyOf(copies);
+    }
+
     private void handle(HttpExchange http) throws IOException {
-        String path = http.getRequestURI().getRawPath();
-        String key = path.length() > bucket.length() + 2 ? 
decode(path.substring(bucket.length() + 2)) : null;
+        // path-style requests: /bucket/key
+        String path = http.getRequestURI().getRawPath().substring(1);
+        int slash = path.indexOf('/');
+        String requestBucket = decode(slash >= 0 ? path.substring(0, slash) : 
path);
+        String key = slash >= 0 && slash < path.length() - 1 ? 
decode(path.substring(slash + 1)) : null;
         String method = http.getRequestMethod();
         if (key == null) {
             if ("GET".equals(method) && query(http).containsKey("list-type")) {
@@ -141,6 +161,11 @@ final class FakeS3Server {
             }
             return;
         }
+        String copySource = 
http.getRequestHeaders().getFirst("x-amz-copy-source");
+        if ("PUT".equals(method) && copySource != null) {
+            copy(http, decode(copySource), requestBucket, key);
+            return;
+        }
         byte[] data = objects.get(key);
         switch (method) {
             case "HEAD" -> {
@@ -181,6 +206,7 @@ final class FakeS3Server {
                 }
             }
             case "DELETE" -> {
+                deletes.add(key);
                 objects.remove(key);
                 http.sendResponseHeaders(204, -1);
                 http.close();
@@ -189,6 +215,27 @@ final class FakeS3Server {
         }
     }
 
+    private void copy(HttpExchange http, String source, String 
destinationBucket, String destinationKey)
+            throws IOException {
+        String sourcePath = source.startsWith("/") ? source.substring(1) : 
source;
+        int slash = sourcePath.indexOf('/');
+        String sourceBucket = sourcePath.substring(0, slash);
+        String sourceKey = sourcePath.substring(slash + 1);
+        copies.add(sourceBucket + "/" + sourceKey + " -> " + destinationBucket 
+ "/" + destinationKey);
+        byte[] data = bucket.equals(sourceBucket) ? objects.get(sourceKey) : 
null;
+        if (data == null) {
+            noSuchKey(http, sourceKey);
+            return;
+        }
+        if (bucket.equals(destinationBucket)) {
+            objects.put(destinationKey, data);
+        }
+        respond(http, 200, "application/xml",
+                "<?xml version=\"1.0\" encoding=\"UTF-8\"?><CopyObjectResult>"
+                                              + 
"<LastModified>2026-10-09T00:00:00.000Z</LastModified>"
+                                              + 
"<ETag>\"0123456789abcdef0123456789abcdef\"</ETag></CopyObjectResult>");
+    }
+
     private void list(HttpExchange http, Map<String, String> query) throws 
IOException {
         int maxKeys = query.containsKey("max-keys") ? 
Integer.parseInt(query.get("max-keys")) : 1000;
         String after = query.get("continuation-token");
diff --git 
a/components/camel-aws/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Consumer.java
 
b/components/camel-aws/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Consumer.java
index 567111b473c0..fdde6c3a05a8 100644
--- 
a/components/camel-aws/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Consumer.java
+++ 
b/components/camel-aws/camel-aws2-sqs/src/main/java/org/apache/camel/component/aws2/sqs/Sqs2Consumer.java
@@ -191,11 +191,14 @@ public class Sqs2Consumer extends 
ScheduledBatchPollingConsumer {
             // update pending number of exchanges
             pendingExchanges = total - index - 1;
 
+            // the receipt handle to delete the message with once the exchange 
is done. It is taken now, as the route
+            // may remove the headers or replace the message before it ends
+            final String receiptHandle = 
exchange.getIn().getHeader(Sqs2Constants.RECEIPT_HANDLE, String.class);
             // add on completion to handle after work when the exchange is done
             exchange.getExchangeExtension().addOnCompletion(new 
Synchronization() {
                 @Override
                 public void onComplete(Exchange exchange) {
-                    processCommit(exchange);
+                    processCommit(exchange, receiptHandle);
                 }
 
                 @Override
@@ -254,14 +257,26 @@ public class Sqs2Consumer extends 
ScheduledBatchPollingConsumer {
     }
 
     /**
-     * Strategy to delete the message after being processed.
+     * Strategy to delete the message after being processed, with the receipt 
handle of the
+     * {@link Sqs2Constants#RECEIPT_HANDLE} header.
      *
-     * @param exchange the exchange
+     * @param      exchange the exchange
+     * @deprecated          use {@link #processCommit(Exchange, String)}, 
which the consumer calls instead
      */
+    @Deprecated(since = "4.24.0")
     protected void processCommit(Exchange exchange) {
+        processCommit(exchange, 
exchange.getIn().getHeader(Sqs2Constants.RECEIPT_HANDLE, String.class));
+    }
+
+    /**
+     * Strategy to delete the message after being processed.
+     *
+     * @param exchange      the exchange
+     * @param receiptHandle the receipt handle the message of the exchange was 
received with
+     */
+    protected void processCommit(Exchange exchange, String receiptHandle) {
         try {
             if (shouldDelete(exchange)) {
-                String receiptHandle = 
exchange.getIn().getHeader(Sqs2Constants.RECEIPT_HANDLE, String.class);
                 DeleteMessageRequest.Builder deleteRequest
                         = 
DeleteMessageRequest.builder().queueUrl(getQueueUrl()).receiptHandle(receiptHandle);
 
diff --git 
a/components/camel-aws/camel-aws2-sqs/src/test/java/org/apache/camel/component/aws2/sqs/SqsConsumerDeleteMessageTest.java
 
b/components/camel-aws/camel-aws2-sqs/src/test/java/org/apache/camel/component/aws2/sqs/SqsConsumerDeleteMessageTest.java
new file mode 100644
index 000000000000..dd804478ba08
--- /dev/null
+++ 
b/components/camel-aws/camel-aws2-sqs/src/test/java/org/apache/camel/component/aws2/sqs/SqsConsumerDeleteMessageTest.java
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.aws2.sqs;
+
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.function.Consumer;
+
+import org.apache.camel.BindToRegistry;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.model.RouteDefinition;
+import org.apache.camel.support.DefaultMessage;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.Test;
+import software.amazon.awssdk.services.sqs.model.DeleteMessageRequest;
+import software.amazon.awssdk.services.sqs.model.Message;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * With deleteAfterRead the consumer deletes the message it received once the 
exchange is done, with the receipt handle
+ * it received the message with, whatever the route did with the 
CamelAwsSqsReceiptHandle header in the meantime.
+ */
+class SqsConsumerDeleteMessageTest extends CamelTestSupport {
+
+    @BindToRegistry("client")
+    private final AmazonSQSClientMock client = new AmazonSQSClientMock();
+
+    @Override
+    protected void doPreSetup() {
+        
client.addMessage(Message.builder().messageId("m-1").receiptHandle("handle-1").body("hello").build());
+    }
+
+    @Test
+    void messageIsDeletedWhenTheRouteRemovesTheHeaders() throws Exception {
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("hello");
+        startRoute(route -> route.removeHeaders("*"));
+
+        mock.assertIsSatisfied();
+        assertDeleted("handle-1");
+    }
+
+    @Test
+    void messageIsDeletedWhenTheRouteReplacesTheMessage() throws Exception {
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("reply");
+        // the new message has none of the headers of the received one
+        startRoute(route -> route.process(exchange -> {
+            DefaultMessage message = new DefaultMessage(exchange);
+            message.setBody("reply");
+            exchange.setMessage(message);
+        }));
+
+        mock.assertIsSatisfied();
+        assertDeleted("handle-1");
+    }
+
+    @Test
+    void messageIsDeletedWithTheReceiptHandleItWasReceivedWith() throws 
Exception {
+        // control: the route keeps the headers the consumer set
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedBodiesReceived("hello");
+        mock.expectedHeaderReceived(Sqs2Constants.RECEIPT_HANDLE, "handle-1");
+        startRoute(route -> {
+        });
+
+        mock.assertIsSatisfied();
+        assertDeleted("handle-1");
+    }
+
+    private void startRoute(Consumer<RouteDefinition> steps) throws Exception {
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                RouteDefinition route = 
from("aws2-sqs://MyQueue?amazonSQSClient=#client&delay=50");
+                steps.accept(route);
+                route.to("mock:result");
+            }
+        });
+    }
+
+    private void assertDeleted(String receiptHandle) {
+        // the delete is sent once the exchange is done, after the mock 
endpoint got it
+        await().atMost(10, TimeUnit.SECONDS).until(() -> 
!client.getDeleteMessageRequests().isEmpty());
+        assertEquals(List.of(receiptHandle),
+                
client.getDeleteMessageRequests().stream().map(DeleteMessageRequest::receiptHandle).toList());
+    }
+}
diff --git 
a/components/camel-azure/camel-azure-storage-blob/src/main/java/org/apache/camel/component/azure/storage/blob/BlobConsumer.java
 
b/components/camel-azure/camel-azure-storage-blob/src/main/java/org/apache/camel/component/azure/storage/blob/BlobConsumer.java
index 26e802fa75b5..295345a3eb79 100644
--- 
a/components/camel-azure/camel-azure-storage-blob/src/main/java/org/apache/camel/component/azure/storage/blob/BlobConsumer.java
+++ 
b/components/camel-azure/camel-azure-storage-blob/src/main/java/org/apache/camel/component/azure/storage/blob/BlobConsumer.java
@@ -253,13 +253,15 @@ public class BlobConsumer extends 
ScheduledBatchPollingConsumer {
             // update pending number of exchanges
             pendingExchanges = total - index - 1;
 
+            // the blob to delete or move once the exchange is done. It is 
taken now, as the route may change or remove
+            // the blob name header before it ends (the producer takes the 
name of the blob to upload from it)
             final String blobName = 
exchange.getIn().getHeader(BlobConstants.BLOB_NAME, String.class);
             // add on completion to handle after work when the exchange is done
             exchange.getExchangeExtension().addOnCompletion(new 
Synchronization() {
                 @Override
                 public void onComplete(Exchange exchange) {
                     try {
-                        processCommit(exchange);
+                        processCommit(exchange, blobName);
                     } finally {
                         releaseInProgress(blobName);
                     }
@@ -303,18 +305,30 @@ public class BlobConsumer extends 
ScheduledBatchPollingConsumer {
         }
     }
 
+    /**
+     * Strategy to move/delete the blob after being processed, the blob named 
by the {@link BlobConstants#BLOB_NAME}
+     * header.
+     *
+     * @param      exchange the exchange
+     * @deprecated          use {@link #processCommit(Exchange, String)}, 
which the consumer calls instead
+     */
+    @Deprecated(since = "4.24.0")
+    protected void processCommit(Exchange exchange) {
+        processCommit(exchange, 
exchange.getIn().getHeader(BlobConstants.BLOB_NAME, String.class));
+    }
+
     /**
      * Strategy to move/delete the blob after being processed.
      *
      * @param exchange the exchange
+     * @param blobName the name of the blob the exchange was created from, in 
the container of the endpoint
      */
-    protected void processCommit(Exchange exchange) {
+    protected void processCommit(Exchange exchange, String blobName) {
         try {
             final String containerName = 
getEndpoint().getConfiguration().getContainerName();
-            final String blobName = 
exchange.getIn().getHeader(BlobConstants.BLOB_NAME, String.class);
 
             if (blobName == null) {
-                LOG.warn("Cannot perform move/delete after read because blob 
name is not available in exchange headers");
+                LOG.warn("Cannot perform move/delete after read because the 
consumed blob has no name");
                 return;
             }
 
diff --git 
a/components/camel-azure/camel-azure-storage-blob/src/test/java/org/apache/camel/component/azure/storage/blob/BlobConsumerDeleteMoveAfterReadTest.java
 
b/components/camel-azure/camel-azure-storage-blob/src/test/java/org/apache/camel/component/azure/storage/blob/BlobConsumerDeleteMoveAfterReadTest.java
new file mode 100644
index 000000000000..9cf81003bbe2
--- /dev/null
+++ 
b/components/camel-azure/camel-azure-storage-blob/src/test/java/org/apache/camel/component/azure/storage/blob/BlobConsumerDeleteMoveAfterReadTest.java
@@ -0,0 +1,265 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.azure.storage.blob;
+
+import java.time.Duration;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentSkipListSet;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.function.Consumer;
+
+import com.azure.core.http.HttpHeaders;
+import com.azure.core.http.HttpResponse;
+import com.azure.core.http.rest.PagedIterable;
+import com.azure.core.http.rest.PagedResponseBase;
+import com.azure.core.http.rest.SimpleResponse;
+import com.azure.core.util.Context;
+import com.azure.core.util.polling.LongRunningOperationStatus;
+import com.azure.core.util.polling.PollResponse;
+import com.azure.core.util.polling.SyncPoller;
+import com.azure.storage.blob.BlobClient;
+import com.azure.storage.blob.BlobContainerClient;
+import com.azure.storage.blob.BlobServiceClient;
+import com.azure.storage.blob.models.BlobCopyInfo;
+import com.azure.storage.blob.models.BlobItem;
+import com.azure.storage.blob.models.BlobProperties;
+import com.azure.storage.blob.models.BlobRange;
+import com.azure.storage.blob.models.BlobRequestConditions;
+import com.azure.storage.blob.models.BlobStorageException;
+import com.azure.storage.blob.models.ListBlobsOptions;
+import com.azure.storage.blob.options.BlockBlobSimpleUploadOptions;
+import com.azure.storage.blob.specialized.BlobInputStream;
+import com.azure.storage.blob.specialized.BlockBlobClient;
+import org.apache.camel.BindToRegistry;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.mock.MockEndpoint;
+import org.apache.camel.model.RouteDefinition;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.Test;
+
+import static org.apache.camel.builder.Builder.constant;
+import static org.apache.camel.builder.Builder.simple;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyString;
+import static org.mockito.ArgumentMatchers.nullable;
+import static org.mockito.Mockito.doAnswer;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * With deleteAfterRead or moveAfterRead the consumer deletes or moves the 
blob it consumed once the exchange is done,
+ * whatever the route did with the CamelAzureStorageBlobBlobName header in the 
meantime: a route may set it to upload a
+ * copy of the blob under another name (the producer takes the name from that 
header), or remove the Camel headers
+ * before it calls another system.
+ * <p>
+ * The scheduler of the consumer is not started: the tests call {@code poll()} 
themselves. The routes are synchronous,
+ * so the delete or the move is done when {@code poll()} returns.
+ */
+class BlobConsumerDeleteMoveAfterReadTest extends CamelTestSupport {
+
+    private static final String ACCOUNT_URL = 
"https://account.blob.core.windows.net/";;
+
+    @BindToRegistry("client")
+    private final BlobServiceClient client = mock(BlobServiceClient.class);
+
+    private final Map<String, Set<String>> containers = new 
ConcurrentHashMap<>();
+    private final Map<String, BlobContainerClient> containerClients = new 
ConcurrentHashMap<>();
+    private final List<String> uploads = new CopyOnWriteArrayList<>();
+    private final List<String> copies = new CopyOnWriteArrayList<>();
+    private final List<String> deletes = new CopyOnWriteArrayList<>();
+
+    @Override
+    protected void doPreSetup() {
+        when(client.getBlobContainerClient(anyString()))
+                .thenAnswer(i -> 
containerClients.computeIfAbsent(i.getArgument(0), this::containerClient));
+        blobs("in").add("todo/a.txt");
+    }
+
+    @Test
+    void deleteAfterReadDeletesTheConsumedBlobWhenTheRouteRenamesIt() throws 
Exception {
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedMessageCount(1);
+        BlobConsumer consumer = startRoute("&deleteAfterRead=true", 
this::uploadCopy);
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("in/out/todo/a.txt"), uploads);
+        assertEquals(List.of("in/todo/a.txt"), deletes);
+        // the copy the route uploaded is kept
+        assertTrue(blobs("in").contains("out/todo/a.txt"));
+        // and the next poll does not consume the blob again
+        assertEquals(0, consumer.poll());
+    }
+
+    @Test
+    void deleteAfterReadDeletesTheConsumedBlobWhenTheRouteRemovesTheHeaders() 
throws Exception {
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedMessageCount(1);
+        BlobConsumer consumer = startRoute("&deleteAfterRead=true", route -> 
route.removeHeaders("CamelAzureStorageBlob*"));
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("in/todo/a.txt"), deletes);
+        assertEquals(0, consumer.poll());
+    }
+
+    @Test
+    void moveAfterReadMovesTheConsumedBlobWhenTheRouteRenamesIt() throws 
Exception {
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedMessageCount(1);
+        BlobConsumer consumer = 
startRoute("&moveAfterRead=true&destinationContainer=archive", 
this::uploadCopy);
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("in/todo/a.txt -> archive/todo/a.txt"), copies);
+        assertEquals(List.of("in/todo/a.txt"), deletes);
+        assertTrue(blobs("in").contains("out/todo/a.txt"));
+        assertEquals(0, consumer.poll());
+    }
+
+    @Test
+    void deleteAfterReadDeletesTheConsumedBlob() throws Exception {
+        // control: the route keeps the header the consumer set
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedMessageCount(1);
+        mock.expectedHeaderReceived(BlobConstants.BLOB_NAME, "todo/a.txt");
+        BlobConsumer consumer = startRoute("&deleteAfterRead=true", route -> {
+        });
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("in/todo/a.txt"), deletes);
+        assertEquals(0, consumer.poll());
+    }
+
+    @Test
+    void moveAfterReadMovesTheConsumedBlob() throws Exception {
+        // control: the destination name is built from the name of the 
consumed blob
+        MockEndpoint mock = getMockEndpoint("mock:result");
+        mock.expectedMessageCount(1);
+        BlobConsumer consumer = 
startRoute("&moveAfterRead=true&destinationContainer=archive&removePrefixOnMove=true"
+                                           + 
"&destinationBlobPrefix=done/&destinationBlobSuffix=.bak",
+                route -> {
+                });
+
+        assertEquals(1, consumer.poll());
+
+        mock.assertIsSatisfied();
+        assertEquals(List.of("in/todo/a.txt -> archive/done/a.txt.bak"), 
copies);
+        assertEquals(List.of("in/todo/a.txt"), deletes);
+    }
+
+    /**
+     * Uploads a copy of the blob to out/ in the same container: the producer 
takes the name of the blob from the
+     * header.
+     */
+    private void uploadCopy(RouteDefinition route) {
+        route.setHeader(BlobConstants.BLOB_NAME, 
simple("out/${header.CamelAzureStorageBlobBlobName}"))
+                .setBody(constant("copy"))
+                
.to("azure-storage-blob://account/in?serviceClient=#client&operation=uploadBlockBlob");
+    }
+
+    private BlobConsumer startRoute(String options, Consumer<RouteDefinition> 
steps) throws Exception {
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                RouteDefinition route = 
from("azure-storage-blob://account/in?serviceClient=#client&prefix=todo/"
+                                             + "&startScheduler=false" + 
options)
+                        .routeId("blob");
+                steps.accept(route);
+                route.to("mock:result");
+            }
+        });
+        return (BlobConsumer) context.getRoute("blob").getConsumer();
+    }
+
+    private Set<String> blobs(String container) {
+        return containers.computeIfAbsent(container, k -> new 
ConcurrentSkipListSet<>());
+    }
+
+    private BlobContainerClient containerClient(String container) {
+        BlobContainerClient containerClient = mock(BlobContainerClient.class);
+        when(containerClient.listBlobs(nullable(ListBlobsOptions.class), 
nullable(Duration.class))).thenAnswer(i -> {
+            ListBlobsOptions options = i.getArgument(0);
+            String prefix = options != null && options.getPrefix() != null ? 
options.getPrefix() : "";
+            List<BlobItem> items = blobs(container).stream().filter(name -> 
name.startsWith(prefix))
+                    .map(name -> new BlobItem().setName(name)).toList();
+            return new PagedIterable<>(() -> new PagedResponseBase<Void, 
BlobItem>(null, 200, null, items, null, null));
+        });
+        when(containerClient.getBlobClient(anyString())).thenAnswer(i -> 
blobClient(container, i.getArgument(0)));
+        return containerClient;
+    }
+
+    @SuppressWarnings("unchecked")
+    private BlobClient blobClient(String container, String name) {
+        BlobClient blobClient = mock(BlobClient.class);
+        when(blobClient.getBlobUrl()).thenReturn(ACCOUNT_URL + container + "/" 
+ name);
+        when(blobClient.openInputStream(nullable(BlobRange.class), 
nullable(BlobRequestConditions.class))).thenAnswer(i -> {
+            if (!blobs(container).contains(name)) {
+                throw notFound();
+            }
+            BlobInputStream stream = mock(BlobInputStream.class);
+            
when(stream.getProperties()).thenReturn(mock(BlobProperties.class));
+            return stream;
+        });
+        doAnswer(i -> {
+            deletes.add(container + "/" + name);
+            if (!blobs(container).remove(name)) {
+                throw notFound();
+            }
+            return null;
+        }).when(blobClient).delete();
+        when(blobClient.beginCopy(anyString(), 
any(Duration.class))).thenAnswer(i -> {
+            String source = i.<String> 
getArgument(0).substring(ACCOUNT_URL.length());
+            String sourceContainer = source.substring(0, source.indexOf('/'));
+            String sourceName = source.substring(source.indexOf('/') + 1);
+            copies.add(source + " -> " + container + "/" + name);
+            boolean found = blobs(sourceContainer).contains(sourceName);
+            if (found) {
+                blobs(container).add(name);
+            }
+            SyncPoller<BlobCopyInfo, Void> poller = mock(SyncPoller.class);
+            when(poller.waitForCompletion(any(Duration.class))).thenReturn(new 
PollResponse<>(
+                    found ? LongRunningOperationStatus.SUCCESSFULLY_COMPLETED 
: LongRunningOperationStatus.FAILED, null));
+            return poller;
+        });
+        BlockBlobClient blockBlobClient = mock(BlockBlobClient.class);
+        
when(blockBlobClient.uploadWithResponse(any(BlockBlobSimpleUploadOptions.class),
 nullable(Duration.class),
+                nullable(Context.class))).thenAnswer(i -> {
+                    uploads.add(container + "/" + name);
+                    blobs(container).add(name);
+                    return new SimpleResponse<>(null, 201, new HttpHeaders(), 
null);
+                });
+        when(blobClient.getBlockBlobClient()).thenReturn(blockBlobClient);
+        return blobClient;
+    }
+
+    private static BlobStorageException notFound() {
+        HttpResponse response = mock(HttpResponse.class);
+        when(response.getStatusCode()).thenReturn(404);
+        return new BlobStorageException("The specified blob does not exist.", 
response, null);
+    }
+}
diff --git 
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc 
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index 0b23c1b1d3da..2577b3f8f95b 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -1819,6 +1819,26 @@ section of 
xref:components::aws2-kinesis-component.adoc[AWS Kinesis].
 `copyBlob` now rejects invalid source account or container names (headers 
`CamelAzureStorageBlobSourceBlobAccountName`
 and `CamelAzureStorageBlobSourceBlobContainerName`) with an 
`IllegalArgumentException`.
 
+=== camel-aws2-s3, camel-azure-storage-blob, camel-aws2-sqs - the consumer 
deletes or moves what it consumed
+
+With `deleteAfterRead` or `moveAfterRead`, the `aws2-s3` and 
`azure-storage-blob` consumers now delete or move the
+object they consumed, in the bucket or container of the endpoint. Before, they 
took the key and the bucket from the
+`CamelAwsS3Key` and `CamelAwsS3BucketName` headers, and the name of the blob 
from the `CamelAzureStorageBlobBlobName`
+header, as the route left them. A route that set these headers, for example to 
write a copy of the object under
+another name, deleted or moved that copy, and the consumed object was consumed 
again by the next poll. A route that
+removed them deleted nothing.
+
+The `aws2-sqs` consumer now deletes the message with the receipt handle it 
received it with. Before, it took the
+`CamelAwsSqsReceiptHandle` header as the route left it, so a route that 
removed the headers or replaced the message
+did not delete it, and the message was received again after its visibility 
timeout.
+
+A failure of the `aws2-s3` consumer to delete or move an object that the AWS 
client raises itself (for example when S3
+cannot be reached) now goes to the exception handler of the consumer, as a 
failure that S3 answers does.
+
+The consumers now call `processCommit(Exchange, String)` with the key, the 
name of the blob or the receipt handle. A
+subclass of `AWS2S3Consumer`, `BlobConsumer` or `Sqs2Consumer` that overrides 
`processCommit(Exchange)` must override
+the new method instead. `processCommit(Exchange)` is deprecated.
+
 === camel-stitch - a response with an empty body, and the default sequence of 
a message
 
 A response of Stitch with an empty body now sets the `CamelStitchCode`, 
`CamelStitchStatus` and `CamelStitchHeaders`

Reply via email to