shashank created CAMEL-27090:
--------------------------------
Summary: camel-smpp - a consumer or producer stopped while it
waits to reconnect leaves the thread of the lost session waiting for ever
Key: CAMEL-27090
URL: https://issues.apache.org/jira/browse/CAMEL-27090
Project: Camel
Issue Type: Bug
Components: camel-smpp
Reporter: shashank
Assignee: shashank
h3. What happens
When the session to the SMSC is lost, jsmpp calls the session state listener of
{{SmppConsumer}} (and {{SmppProducer}}) on the reader thread of the lost
session. The listener closes the session and calls {{reconnect()}}, which runs
the reconnect task on that thread: {{BackgroundTask.run}} schedules the
attempts on the reconnect service and waits on the latch of the task. The task
is built {{withUnlimitedDuration()}}, so only an attempt that reconnects, or
one that sees the consumer stopping, releases the thread.
A stop while the reconnect waits for its next attempt (the SMSC is down; the
attempts come every 5 s by default) shuts the reconnect service down, which
cancels the scheduled attempts. No attempt runs any more: the thread of the
lost session waits for ever, and the task stays in the task registry. A task
built after the stop began is scheduled on the shut down service and dropped
(CallerRunsPolicy), with the same result. Each route or context stop during an
SMSC outage leaves one such thread.
h3. How it was found
Found with a TLA+ model of the listener thread, the reconnect service thread
and the stop, checked with TLC. On main the invariant "after the stop, no
thread waits on a reconnect latch that nothing can count down" is violated in
10 states: SMSC down, session lost, the listener thread waits for the first
attempt, stop, shutdown cancels the attempt, the thread waits for ever. A fix
that only cancels the task in doStop is also violated (the task is published
after the stop looked for it). The fix below holds (382 distinct states).
h3. How to reproduce
{{SmppStopDuringReconnectTest}}: the real consumer (and producer) bound to an
SMSC made with jsmpp's {{SMPPServerSessionListener}}; the SMSC closes the
session and stops listening; once a thread waits in {{reconnect()}}, the route
is stopped: the thread is still there 10 seconds later.
h3. Fix
{{reconnect()}} keeps the task it runs and runs it only if the service is not
stopping; {{doStop}} cancels that task after the reconnect service is shut
down, which releases the waiting thread (as camel-pgevent and camel-master do
with their tasks). {{SmppUtils.newReconnectTask}} returns the
{{BackgroundTask}} so that it can be cancelled.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)