[
https://issues.apache.org/jira/browse/CAMEL-25299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-25299.
---------------------------------
Fix Version/s: 4.23.0
Resolution: Fixed
Fixed on main via https://github.com/apache/camel/pull/27340
> Error handler - with asyncDelayedRedelivery,
> allowRedeliveryWhileStopping=false is ignored: a route stop waits for the
> redelivery delay and then redelivers
> -----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25299
> URL: https://issues.apache.org/jira/browse/CAMEL-25299
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Reporter: shashank
> Priority: Minor
> Fix For: 4.23.0
>
>
> The documentation of the dead letter channel says that with
> {{allowRedeliveryWhileStopping=false}} any new redelivery attempt is not
> allowed once the stop of the route (or Camel) has been triggered: a
> {{RejectedExecutionException}} is set and the exchange stops being routed
> (the dead letter channel still receives it).
> That holds for the synchronous redelivery: {{RedeliveryTask.sleep()}} waits
> in chunks of 1 second and checks {{preparingShutdown}} after each chunk, then
> marks the exchange rejected and exhausted, so it goes to the dead letter
> channel within a second
> ({{NotAllowRedeliveryWhileStoppingDeadLetterChannelTest}}).
> With {{asyncDelayedRedelivery}} the delay is a task on the error handler's
> scheduled thread pool:
> {code:java}
> executorService.schedule(() -> reactiveExecutor.schedule(this::redeliver),
> redeliveryDelay, TimeUnit.MILLISECONDS);
> {code}
> and {{redeliver()}} calls the output again without any check of the shutdown
> state (only {{run()}} checks {{isRunAllowed()}}, and the redelivery decision
> in {{doRun()}} was taken before the stop). So when the route is stopped while
> a redelivery waits for its delay:
> * the graceful stop waits for the inflight exchange for the whole remaining
> delay, or until the shutdown timeout (45 s by default) if the delay is
> longer, then stops the consumer forcibly;
> * when the delay ends, the message is redelivered anyway (also after the
> forced stop, into the stopped route), which is what
> {{allowRedeliveryWhileStopping=false}} is meant to prevent.
> CAMEL-3364 (2.11.0), which added {{allowRedeliveryWhileStopping}}, already
> named this case in its discussion: the error handler keeps the delayed
> exchanges as tasks in its thread pool, and on shutdown those have to be dealt
> with too. The implementation only made {{sleep()}} check the shutdown state;
> nothing in the ticket or the code says the scheduled task was meant to be
> exempt.
> h3. Reproduction
> Test {{NotAllowRedeliveryWhileStoppingAsyncDelayedTest}} (camel-core):
> {{deadLetterChannel("mock:dead").maximumRedeliveries(5).redeliveryDelay(1
> hour, capped to the 60 s
> maximumRedeliveryDelay).asyncDelayedRedelivery().allowRedeliveryWhileStopping(false)}},
> a route {{seda:start -> mock:foo -> throwException}}, and a scheduled thread
> pool for the error handler that counts down a latch when the redelivery is
> scheduled. After the first attempt failed and the redelivery is scheduled,
> the test calls {{stopRoute("foo")}} (shutdown timeout 5 s). Expected: the
> dead letter channel gets the exchange with
> {{RejectedExecutionException("Redelivery not allowed while stopping")}}, no
> redelivery, nothing inflight. A second test does the same with
> {{context.stop()}}. On main both stops run into the 5 s timeout and the dead
> letter channel has nothing:
> {noformat}
> NotAllowRedeliveryWhileStoppingAsyncDelayedTest.testStopRouteRejectsScheduledRedelivery
> mock://dead Received message count. Expected: <1> but was: <0>
> NotAllowRedeliveryWhileStoppingAsyncDelayedTest.testStopCamelContextRejectsScheduledRedelivery
> mock://dead Received message count. Expected: <1> but was: <0>
> {noformat}
> The defect was found with a TLA+ model of the redelivery task (delivery,
> {{run()}}/{{doRun()}}, synchronous {{sleep()}}, the scheduled task,
> {{redeliver()}}) and the graceful shutdown ({{prepareShutdown}}, waiting for
> inflight exchanges, timeout). The invariant "no redelivery starts after its
> delay ended while the error handler prepares to shut down
> (allowRedeliveryWhileStopping=false)" is violated with
> {{asyncDelayedRedelivery}} in 6 steps (deliver, fail, schedule,
> prepareShutdown, delay ends, redeliver); it holds for the synchronous
> redelivery, with and without a dead letter channel, and for the fixed task.
> h3. Proposed fix
> When the redelivery is not allowed while stopping, the scheduled task wakes
> up every second, like {{sleep()}}, and when the error handler is preparing to
> shut down it rejects the redelivery the same way
> ({{RejectedExecutionException("Redelivery not allowed while stopping")}},
> redelivery exhausted, back to {{run()}}, which moves the exchange to the dead
> letter channel or fails it). When the delay is over it redelivers as before.
> With {{allowRedeliveryWhileStopping=true}} (the default) nothing changes: one
> task for the whole delay. If the thread pool rejects the next wake-up, the
> redelivery is rejected the same way instead of losing the callback.
> With the fix the new tests pass (each in about 1.5 s), and {{*Redelivery*}},
> {{NotAllowRedelivery*}} and {{DeadLetterChannel*}} tests in camel-core pass
> (95 tests).
> Cost: one short task per second per waiting redelivery, only with
> {{allowRedeliveryWhileStopping=false}} (the same rate as the synchronous
> {{sleep()}} loop, but without a blocked thread). One difference to
> {{sleep()}}: {{sleep()}} does not check at all for a delay below 1 second,
> while the scheduled task checks once when such a delay ends, so a short
> redelivery whose delay ends while stopping is rejected too (which is what the
> option describes). The upgrade guide gets a note, as a stop now rejects these
> messages within a second instead of redelivering them.
> Not addressed (low): with {{allowRedeliveryWhileStopping=true}} (the
> default), when Camel itself is stopped and the shutdown times out, the error
> handler thread pool is shut down with the delayed task still in it, so the
> exchange is never completed; Camel is stopping and the await manager releases
> waiting callers. (With {{false}} the fix rejects the task within a second,
> before the timeout.)
> Affected: 4.14.x, 4.18.x and main (same code).
> Duplicate check (2026-10-03): JIRA text "allowRedeliveryWhileStopping"
> (CAMEL-6033, CAMEL-20744, CAMEL-3364, CAMEL-19896, CAMEL-12603),
> "asyncDelayedRedelivery" with stop/shutdown (none), "redelivery" with
> "stopping" since 2022 (11 issues, none about the asynchronous delay). GitHub
> pull requests "asyncDelayedRedelivery" (#1771, a doc note, 2017),
> "allowRedeliveryWhileStopping" (#23179, CAMEL-23494).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
