This is an automated email from the ASF dual-hosted git repository.

davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git


The following commit(s) were added to refs/heads/main by this push:
     new 6ec5f70e7f02 CAMEL-26967: camel-kafka - breakOnFirstError with 
synchronous manual commits: a commitSync timeout in the break skips the failed 
record and the rest of the poll (#30132)
6ec5f70e7f02 is described below

commit 6ec5f70e7f02eec2f694f6278b32579cf18fa4db
Author: allthingssecurity <[email protected]>
AuthorDate: Tue Nov 17 20:38:06 2026 +0530

    CAMEL-26967: camel-kafka - breakOnFirstError with synchronous manual 
commits: a commitSync timeout in the break skips the failed record and the rest 
of the poll (#30132)
    
    With breakOnFirstError and the SyncCommitManager, a failed record makes
    processException commit the records routed before it and return the
    break, so the consumer reconnects and polls again from the committed
    offset. When that commitSync threw (a TimeoutException when the timeout
    expires first, or CommitFailedException / RebalanceInProgressException),
    the exception left the poll and reached the poll exception strategy.
    With pollOnError ERROR_HANDLER (the default), DISCARD or RETRY the
    fetcher went on polling with the same Kafka consumer, whose position was
    after the whole poll: the failed record and the records after it were
    never routed, and the next manual commit committed past them.
    
    A KafkaException of that commit (not WakeupException or InterruptException)
    is now reported to the exception handler of the consumer, and the break is
    returned as before: the new Kafka consumer polls from the committed offset,
    which is the failed record when the commit was applied, or an earlier one.
    So that commit failure no longer goes through pollOnError: it reaches the
    exception handler of the consumer (logged, or routed with
    bridgeErrorHandler=true) instead of the error handler of the route through
    the ERROR_HANDLER strategy, and pollOnError=STOP no longer stops the
    consumer on it (the consumer reconnects, as for the break).
    
    Found by the Kafka DST world of CAMEL-26947 with
    -Ddst.kafka.syncBreakOnFirstError=true (seed 95). New
    KafkaConsumerBreakOnFirstErrorCommitTimeoutTest.
    
    Co-authored-by: Claude Opus 5.5 <[email protected]>
---
 .../org/apache/camel/catalog/components/kafka.json |   2 +-
 .../org/apache/camel/catalog/docs/test-dst.adoc    |   2 +-
 .../org/apache/camel/component/kafka/kafka.json    |   2 +-
 .../streaming/KafkaRecordStreamingProcessor.java   |  26 +-
 ...ConsumerBreakOnFirstErrorCommitTimeoutTest.java | 269 +++++++++++++++++++++
 .../camel/component/kafka/dst/KafkaWorld.java      |  16 +-
 .../camel-test-dst/src/main/docs/test-dst.adoc     |   2 +-
 .../ROOT/pages/camel-4x-upgrade-guide-4_24.adoc    |  11 +
 8 files changed, 312 insertions(+), 18 deletions(-)

diff --git 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/kafka.json
 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/kafka.json
index 1a9e7f4c66ef..54d4737a98ef 100644
--- 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/kafka.json
+++ 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/kafka.json
@@ -301,6 +301,6 @@
     "sslTruststoreType": { "index": 121, "kind": "parameter", "displayName": 
"Ssl Truststore Type", "group": "security", "label": "common,security", 
"required": false, "type": "string", "javaType": "java.lang.String", 
"deprecated": false, "autowired": false, "secret": false, "defaultValue": 
"JKS", "configurationClass": 
"org.apache.camel.component.kafka.KafkaConfiguration", "configurationField": 
"configuration", "description": "The file format of the trust store file. The 
default value is [...]
   },
   "dstWorlds": {
-    "KafkaWorld": { "javaType": 
"org.apache.camel.component.kafka.dst.KafkaWorld", "fake": "FakeKafka: a topic 
with one or two partitions and a consumer group (classic protocol, eager 
rebalance), behind the Consumer interface of KafkaClientFactory", "covers": [ 
"commits by Kafka's auto-commit, by manual synchronous or asynchronous commits 
in the route, and per batch with batching=true", "breakOnFirstError", 
"pollOnError ERROR_HANDLER, DISCARD, RETRY and RECONNECT", "a poll that fails 
whi [...]
+    "KafkaWorld": { "javaType": 
"org.apache.camel.component.kafka.dst.KafkaWorld", "fake": "FakeKafka: a topic 
with one or two partitions and a consumer group (classic protocol, eager 
rebalance), behind the Consumer interface of KafkaClientFactory", "covers": [ 
"commits by Kafka's auto-commit, by manual synchronous or asynchronous commits 
in the route, and per batch with batching=true", "breakOnFirstError", 
"pollOnError ERROR_HANDLER, DISCARD, RETRY and RECONNECT", "a poll that fails 
whi [...]
   }
 }
diff --git 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/test-dst.adoc
 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/test-dst.adoc
index d6ee0ac24ad9..a78c6d18b373 100644
--- 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/test-dst.adoc
+++ 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/test-dst.adoc
@@ -121,7 +121,7 @@ a|* commits by Kafka's auto-commit, by manual synchronous 
or asynchronous commit
 * route restart, also while the consumer polls
 * route suspend and resume, with a second request while the consumer handles 
the first
 * partitions revoked and assigned again by a rebalance while records are 
processed
-|CAMEL-25313, CAMEL-25756
+|CAMEL-25313, CAMEL-25756, CAMEL-26967
 
 |camel-kafka-share
 |`KafkaShareWorld` (`KafkaShareConsumerDstTest`)
diff --git 
a/components/camel-kafka/src/generated/resources/META-INF/org/apache/camel/component/kafka/kafka.json
 
b/components/camel-kafka/src/generated/resources/META-INF/org/apache/camel/component/kafka/kafka.json
index 1a9e7f4c66ef..54d4737a98ef 100644
--- 
a/components/camel-kafka/src/generated/resources/META-INF/org/apache/camel/component/kafka/kafka.json
+++ 
b/components/camel-kafka/src/generated/resources/META-INF/org/apache/camel/component/kafka/kafka.json
@@ -301,6 +301,6 @@
     "sslTruststoreType": { "index": 121, "kind": "parameter", "displayName": 
"Ssl Truststore Type", "group": "security", "label": "common,security", 
"required": false, "type": "string", "javaType": "java.lang.String", 
"deprecated": false, "autowired": false, "secret": false, "defaultValue": 
"JKS", "configurationClass": 
"org.apache.camel.component.kafka.KafkaConfiguration", "configurationField": 
"configuration", "description": "The file format of the trust store file. The 
default value is [...]
   },
   "dstWorlds": {
-    "KafkaWorld": { "javaType": 
"org.apache.camel.component.kafka.dst.KafkaWorld", "fake": "FakeKafka: a topic 
with one or two partitions and a consumer group (classic protocol, eager 
rebalance), behind the Consumer interface of KafkaClientFactory", "covers": [ 
"commits by Kafka's auto-commit, by manual synchronous or asynchronous commits 
in the route, and per batch with batching=true", "breakOnFirstError", 
"pollOnError ERROR_HANDLER, DISCARD, RETRY and RECONNECT", "a poll that fails 
whi [...]
+    "KafkaWorld": { "javaType": 
"org.apache.camel.component.kafka.dst.KafkaWorld", "fake": "FakeKafka: a topic 
with one or two partitions and a consumer group (classic protocol, eager 
rebalance), behind the Consumer interface of KafkaClientFactory", "covers": [ 
"commits by Kafka's auto-commit, by manual synchronous or asynchronous commits 
in the route, and per batch with batching=true", "breakOnFirstError", 
"pollOnError ERROR_HANDLER, DISCARD, RETRY and RECONNECT", "a poll that fails 
whi [...]
   }
 }
diff --git 
a/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/streaming/KafkaRecordStreamingProcessor.java
 
b/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/streaming/KafkaRecordStreamingProcessor.java
index ab2aa6afa5ea..2ff958e6c5e5 100644
--- 
a/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/streaming/KafkaRecordStreamingProcessor.java
+++ 
b/components/camel-kafka/src/main/java/org/apache/camel/component/kafka/consumer/support/streaming/KafkaRecordStreamingProcessor.java
@@ -61,7 +61,10 @@ import org.apache.camel.spi.ExceptionHandler;
 import org.apache.camel.support.EventNotifierSupport;
 import org.apache.camel.support.LanguageSupport;
 import org.apache.kafka.clients.consumer.ConsumerRecord;
+import org.apache.kafka.common.KafkaException;
 import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.errors.InterruptException;
+import org.apache.kafka.common.errors.WakeupException;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -727,12 +730,25 @@ final class KafkaRecordStreamingProcessor extends 
KafkaRecordProcessor {
             // we should just do a commit (vs the original forceCommit)
             // when route uses NOOP Commit Manager it will rely
             // on the route implementation to explicitly commit offset
-            // when route uses Synch/Asynch Commit Manager it will
-            // ALWAYS commit the offset for the failing record
-            // and will ALWAYS retry it
-            commitManager.commit(topicPartition);
+            // when route uses Synch/Asynch Commit Manager it commits the 
offset of the
+            // records before the failing record. That commit can fail or time 
out: the
+            // consumer still breaks out and polls again from the last 
committed offset
+            try {
+                commitManager.commit(topicPartition);
+            } catch (WakeupException | InterruptException e) {
+                throw e;
+            } catch (KafkaException e) {
+                // the commit failed or timed out (a commitSync timeout may 
still have been applied): break out anyway,
+                // so the new Kafka consumer polls again from the committed 
offset. Leaving the batch through the
+                // exception would let the same consumer go on after the 
records of this batch that were not routed
+                exceptionHandler.handleException(
+                        "Error committing the offset of the records before the 
failed record on partition "
+                                                 + topicPartition
+                                                 + ". The consumer polls again 
from the last committed offset.",
+                        e);
+            }
 
-            // continue to next partition
+            // break out: the consumer reconnects and polls again from the 
committed offset
             return true;
         } else {
             // will handle/log the exception and then continue to next
diff --git 
a/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/KafkaConsumerBreakOnFirstErrorCommitTimeoutTest.java
 
b/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/KafkaConsumerBreakOnFirstErrorCommitTimeoutTest.java
new file mode 100644
index 000000000000..85b9f5a1cad8
--- /dev/null
+++ 
b/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/KafkaConsumerBreakOnFirstErrorCommitTimeoutTest.java
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.kafka;
+
+import java.time.Duration;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Properties;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.locks.LockSupport;
+
+import org.apache.camel.BindToRegistry;
+import org.apache.camel.Exchange;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.kafka.consumer.KafkaManualCommit;
+import org.apache.camel.spi.ExceptionHandler;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.apache.camel.test.junit6.TestExecutionConfiguration;
+import org.apache.kafka.clients.consumer.Consumer;
+import org.apache.kafka.clients.consumer.ConsumerRecord;
+import org.apache.kafka.clients.consumer.ConsumerRecords;
+import org.apache.kafka.clients.consumer.MockConsumer;
+import org.apache.kafka.clients.consumer.OffsetAndMetadata;
+import org.apache.kafka.clients.consumer.OffsetCommitCallback;
+import org.apache.kafka.clients.producer.Producer;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.errors.TimeoutException;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.EnumSource;
+
+import static org.awaitility.Awaitility.await;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * With breakOnFirstError and synchronous manual commits, the consumer commits 
the records before the failed one and
+ * reconnects, so the failed record and the records after it in the same poll 
are polled again. The commitSync of that
+ * break can fail: it throws a TimeoutException when the timeout expires 
first, and the commit may or may not have been
+ * applied (KafkaConsumer#commitSync(Map, Duration)). The consumer must still 
poll the failed record again, and must not
+ * go on after the records of the batch that it did not route. A {@link 
MockConsumer} on a log and committed offsets
+ * replaces the broker: each Kafka consumer of the fetcher is a new instance 
that starts from the committed offset.
+ */
+class KafkaConsumerBreakOnFirstErrorCommitTimeoutTest extends CamelTestSupport 
{
+
+    private static final String TOPIC = "break-commit-timeout";
+    private static final TopicPartition PARTITION = new TopicPartition(TOPIC, 
0);
+
+    // the records of the partition (the broker): f is stored after c failed
+    private final List<String> log = new CopyOnWriteArrayList<>(List.of("a", 
"b", "c", "d", "e"));
+    // the committed offsets of the consumer group (the broker)
+    private final Map<TopicPartition, OffsetAndMetadata> committed = new 
ConcurrentHashMap<>();
+    private final List<String> routed = new CopyOnWriteArrayList<>();
+    private final AtomicInteger consumersCreated = new AtomicInteger();
+    private final AtomicInteger commitTimeouts = new AtomicInteger();
+    private final AtomicBoolean failC = new AtomicBoolean(true);
+    private final AtomicBoolean timeoutNextCommit = new AtomicBoolean();
+
+    // the exception handler of the consumer
+    private final List<Throwable> handled = new CopyOnWriteArrayList<>();
+    @BindToRegistry("commitErrors")
+    private final ExceptionHandler exceptionHandler = new ExceptionHandler() {
+        @Override
+        public void handleException(Throwable exception) {
+            handled.add(exception);
+        }
+
+        @Override
+        public void handleException(String message, Throwable exception) {
+            handled.add(exception);
+        }
+
+        @Override
+        public void handleException(String message, Exchange exchange, 
Throwable exception) {
+            handled.add(exception);
+        }
+    };
+
+    @BindToRegistry("logFactory")
+    private final KafkaClientFactory factory = new KafkaClientFactory() {
+        @Override
+        public Producer getProducer(Properties kafkaProps) {
+            throw new UnsupportedOperationException("The test has no 
producer");
+        }
+
+        @Override
+        public Consumer getConsumer(Properties kafkaProps) {
+            consumersCreated.incrementAndGet();
+            LogMockConsumer consumer = new LogMockConsumer();
+            // the group coordinator assigns the partition in the first poll
+            consumer.schedulePollTask(() -> 
consumer.rebalance(List.of(PARTITION)));
+            return consumer;
+        }
+
+        @Override
+        public String getBrokers(KafkaConfiguration configuration) {
+            return "localhost:9092";
+        }
+    };
+
+    @Override
+    public void configureTest(TestExecutionConfiguration 
testExecutionConfiguration) {
+        testExecutionConfiguration.withUseRouteBuilder(false);
+    }
+
+    // ERROR_HANDLER (the default), DISCARD and RETRY go on polling with the 
same Kafka consumer after the exception;
+    // RECONNECT is a control: it reconnects after any exception of the poll 
loop
+    @ParameterizedTest
+    @EnumSource(value = PollOnError.class, names = { "ERROR_HANDLER", 
"DISCARD", "RETRY", "RECONNECT" })
+    void 
testFailedRecordIsPolledAgainWhenTheCommitOfTheBreakTimesOut(PollOnError 
pollOnError) throws Exception {
+        startRoute(pollOnError);
+
+        // f is the last record: once it is committed, the consumer has gone 
past every record of the log
+        await().atMost(20, TimeUnit.SECONDS).untilAsserted(
+                () -> assertEquals(6L, committedOffset(), "The consumer must 
commit every record of the log"));
+
+        assertEquals(1, commitTimeouts.get(), "The commitSync of the break 
must have timed out");
+        assertEquals(List.of("a", "b", "c", "d", "e", "f"), routed,
+                "The failed record and the records after it must be polled 
again, not skipped");
+        assertTrue(consumersCreated.get() >= 2, "The break must create a new 
Kafka consumer");
+    }
+
+    @Test
+    void testFailedCommitOfTheBreakIsReportedToTheExceptionHandler() throws 
Exception {
+        // RECONNECT polls the failed record again also without the change: 
this test is about the report only
+        startRoute(PollOnError.RECONNECT);
+
+        await().atMost(20, TimeUnit.SECONDS).untilAsserted(
+                () -> assertEquals(6L, committedOffset(), "The consumer must 
commit every record of the log"));
+
+        assertEquals(1, commitTimeouts.get(), "The commitSync of the break 
must have timed out");
+        
assertTrue(handled.stream().anyMatch(TimeoutException.class::isInstance),
+                "The failed commit must be reported to the exception handler 
of the consumer, got " + handled);
+    }
+
+    private void startRoute(PollOnError pollOnError) throws Exception {
+        context.addRoutes(new RouteBuilder() {
+            @Override
+            public void configure() {
+                from("kafka:" + TOPIC + 
"?brokers=localhost:9092&kafkaClientFactory=#logFactory"
+                     + 
"&groupId=break-commit-timeout&pollTimeoutMs=10&autoOffsetReset=earliest"
+                     + 
"&autoCommitEnable=false&breakOnFirstError=true&pollOnError=" + pollOnError
+                     + "&exceptionHandler=#commitErrors"
+                     + 
"&allowManualCommit=true&kafkaManualCommitFactory=#class:"
+                     + 
"org.apache.camel.component.kafka.consumer.DefaultKafkaManualCommitFactory")
+                        .routeId("kafka")
+                        
.process(KafkaConsumerBreakOnFirstErrorCommitTimeoutTest.this::process)
+                        .process(exchange -> exchange.getMessage()
+                                .getHeader(KafkaConstants.MANUAL_COMMIT, 
KafkaManualCommit.class).commit());
+            }
+        });
+        context.start();
+    }
+
+    private void process(Exchange exchange) {
+        String body = exchange.getMessage().getBody(String.class);
+        if ("c".equals(body) && failC.compareAndSet(true, false)) {
+            // the commit of the break times out, and a new record is stored 
after the failed one
+            timeoutNextCommit.set(true);
+            log.add("f");
+            throw new IllegalStateException("Simulated failure of record " + 
body);
+        }
+        routed.add(body);
+    }
+
+    private long committedOffset() {
+        OffsetAndMetadata offset = committed.get(PARTITION);
+        return offset != null ? offset.offset() : -1;
+    }
+
+    /**
+     * A Kafka consumer of the group: it polls the records of {@link #log} 
from its position, which a new consumer takes
+     * from the committed offset of the group, or else from the beginning 
(autoOffsetReset=earliest).
+     */
+    private final class LogMockConsumer extends MockConsumer<Object, Object> {
+        private final Set<TopicPartition> positioned = new HashSet<>();
+
+        LogMockConsumer() {
+            super("earliest");
+            updateBeginningOffsets(Map.of(PARTITION, 0L));
+        }
+
+        @Override
+        public synchronized ConsumerRecords<Object, Object> poll(Duration 
timeout) {
+            updateEndOffsets(Map.of(PARTITION, (long) log.size()));
+            // runs the poll tasks (the assignment), and sets the position of 
a newly assigned partition
+            super.poll(timeout);
+            for (TopicPartition partition : assignment()) {
+                OffsetAndMetadata offset = committed.get(partition);
+                if (positioned.add(partition) && offset != null) {
+                    // a newly assigned partition starts from the committed 
offset of the group
+                    super.seek(partition, offset.offset());
+                }
+            }
+            Map<TopicPartition, List<ConsumerRecord<Object, Object>>> records 
= new HashMap<>();
+            Map<TopicPartition, OffsetAndMetadata> nextOffsets = new 
HashMap<>();
+            int end = log.size();
+            for (TopicPartition partition : assignment()) {
+                List<ConsumerRecord<Object, Object>> fetched = new 
ArrayList<>();
+                for (long offset = position(partition); offset < end; 
offset++) {
+                    fetched.add(new ConsumerRecord<>(TOPIC, 0, offset, null, 
log.get((int) offset)));
+                }
+                if (!fetched.isEmpty()) {
+                    records.put(partition, fetched);
+                    nextOffsets.put(partition, new OffsetAndMetadata(end));
+                    super.seek(partition, end);
+                }
+            }
+            if (records.isEmpty()) {
+                // a real consumer waits up to the poll timeout when there is 
nothing to fetch
+                LockSupport.parkNanos(TimeUnit.MILLISECONDS.toNanos(10));
+            }
+            return new ConsumerRecords<>(records, nextOffsets);
+        }
+
+        @Override
+        public void commitSync(Map<TopicPartition, OffsetAndMetadata> offsets, 
Duration timeout) {
+            if (timeoutNextCommit.compareAndSet(true, false)) {
+                // as KafkaConsumer when the timeout expires before the commit 
completed (here it was not applied)
+                commitTimeouts.incrementAndGet();
+                throw new TimeoutException(
+                        "Timeout of " + timeout.toMillis() + "ms expired 
before successfully committing offsets "
+                                           + offsets);
+            }
+            // MockConsumer (kafka-clients 4.3.x): commitSync(Map, Duration) 
-> commitSync(Map) -> commitAsync(Map, callback),
+            // which is overridden here and records the offsets that 
committedOffset() reads
+            super.commitSync(offsets, timeout);
+        }
+
+        @Override
+        public synchronized void commitAsync(
+                Map<TopicPartition, OffsetAndMetadata> offsets, 
OffsetCommitCallback callback) {
+            super.commitAsync(offsets, callback);
+            committed.putAll(offsets);
+        }
+
+        @Override
+        public synchronized Map<TopicPartition, OffsetAndMetadata> 
committed(Set<TopicPartition> partitions) {
+            // as the Kafka consumer: a partition without a committed offset 
maps to null
+            Map<TopicPartition, OffsetAndMetadata> answer = new HashMap<>();
+            for (TopicPartition partition : partitions) {
+                answer.put(partition, committed.get(partition));
+            }
+            return answer;
+        }
+    }
+}
diff --git 
a/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/dst/KafkaWorld.java
 
b/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/dst/KafkaWorld.java
index 6a0bb45807c8..47c7ec288b91 100644
--- 
a/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/dst/KafkaWorld.java
+++ 
b/components/camel-kafka/src/test/java/org/apache/camel/component/kafka/dst/KafkaWorld.java
@@ -61,9 +61,9 @@ import org.apache.kafka.clients.producer.Producer;
  * failed batch is committed, see "Break on First Error in Batching Mode" in 
the component documentation); manual
  * commits from another thread; consumersCount above 1; seekTo, offset 
repositories and resume strategies.
  * <p>
- * Left out of the default set: breakOnFirstError with the synchronous manual 
commits (SyncCommitManager), where a
- * commitSync that times out in the break path is being looked at separately;
- * {@code -Ddst.kafka.syncBreakOnFirstError=true} puts the combination back.
+ * breakOnFirstError with the synchronous manual commits (SyncCommitManager) 
is explored too: this world found that a
+ * commitSync that failed or timed out in the break skipped the failed record 
and the rest of the poll (CAMEL-26967,
+ * fixed).
  * <p>
  * consumersCount is 1: a consumer thread loops for ever, and with the 
one-stack model a second looping thread nested
  * above the first would never let it resume. For the same reason a route stop 
that runs while the consumer thread waits
@@ -104,7 +104,7 @@ import org.apache.kafka.clients.producer.Producer;
                   "the committed offsets reach the end once the faults 
cleared",
                   "the route holds one Kafka consumer once the faults cleared",
                   "no Kafka consumer is left open after the route stopped" },
-          issues = { "CAMEL-25313", "CAMEL-25756" })
+          issues = { "CAMEL-25313", "CAMEL-25756", "CAMEL-26967" })
 final class KafkaWorld {
 
     static final String ROUTE = "kafka";
@@ -142,11 +142,9 @@ final class KafkaWorld {
         this.partitions = dst.pick(List.of(1, 2));
         this.kafka = new FakeKafka(dst, ledger, partitions);
         this.mode = 
dst.pick(Arrays.asList(System.getProperty("dst.kafka.modes", 
"AUTO,SYNC,ASYNC,BATCH").split(",")));
-        // see the class Javadoc: batching with breakOnFirstError is not 
explored, and the synchronous manual commits
-        // with breakOnFirstError are left out unless 
-Ddst.kafka.syncBreakOnFirstError=true
-        this.breakOnFirstError = !"BATCH".equals(mode)
-                && (!"SYNC".equals(mode) || 
Boolean.getBoolean("dst.kafka.syncBreakOnFirstError"))
-                && dst.chance(0.5);
+        // see the class Javadoc: batching with breakOnFirstError is not 
explored; the synchronous manual commits with
+        // breakOnFirstError are, since CAMEL-26967 fixed the commit that 
failed or timed out in the break
+        this.breakOnFirstError = !"BATCH".equals(mode) && dst.chance(0.5);
         this.commitEveryRecord = dst.chance(0.5);
         this.pollOnError = dst.pick(Arrays.asList(
                 System.getProperty("dst.kafka.pollOnError", 
"ERROR_HANDLER,DISCARD,RETRY,RECONNECT").split(",")));
diff --git a/components/camel-test/camel-test-dst/src/main/docs/test-dst.adoc 
b/components/camel-test/camel-test-dst/src/main/docs/test-dst.adoc
index d6ee0ac24ad9..a78c6d18b373 100644
--- a/components/camel-test/camel-test-dst/src/main/docs/test-dst.adoc
+++ b/components/camel-test/camel-test-dst/src/main/docs/test-dst.adoc
@@ -121,7 +121,7 @@ a|* commits by Kafka's auto-commit, by manual synchronous 
or asynchronous commit
 * route restart, also while the consumer polls
 * route suspend and resume, with a second request while the consumer handles 
the first
 * partitions revoked and assigned again by a rebalance while records are 
processed
-|CAMEL-25313, CAMEL-25756
+|CAMEL-25313, CAMEL-25756, CAMEL-26967
 
 |camel-kafka-share
 |`KafkaShareWorld` (`KafkaShareConsumerDstTest`)
diff --git 
a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc 
b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
index 7c4c92bbbf34..c773cd661d82 100644
--- a/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
+++ b/docs/user-manual/modules/ROOT/pages/camel-4x-upgrade-guide-4_24.adoc
@@ -8247,6 +8247,17 @@ resumed after the failed record. The new consumer leaves 
and rejoins the consume
 rebalance of the group, also when the partition has a committed offset. There 
is no option to keep the previous
 behaviour.
 
+=== camel-kafka - breakOnFirstError: a failed commit before the break goes to 
the exception handler
+
+With `breakOnFirstError=true`, the commit before the break on a failed record 
can fail or time out (for example a
+`TimeoutException` of `commitSync` with the `SyncCommitManager`). The consumer 
now reports that failure to its
+`ExceptionHandler` and still breaks out, polling again from the last committed 
offset.
+Before, the commit failure went to the `pollOnError` strategy. With 
`ERROR_HANDLER` (the default), `DISCARD` or
+`RETRY`, the failed record and the rest of the poll were skipped.
+The failure no longer reaches the error handler of the route (such as an 
`onException` that caught commit timeouts)
+unless `bridgeErrorHandler=true`. With `pollOnError=STOP`, the consumer now 
reconnects in this case instead of
+stopping.
+
 === camel-kafka - the Kafka idempotent and key-value repositories use the acks 
of the Kafka client
 
 `KafkaIdempotentRepository` and `KafkaKeyValueRepository` no longer set 
`acks=1` on their producer, so the default of

Reply via email to