Henrik created CAMEL-24315:
------------------------------
Summary: camel-google-pubsub: consumer stop misses subscribers
still starting, thread parks forever in awaitTerminated
Key: CAMEL-24315
URL: https://issues.apache.org/jira/browse/CAMEL-24315
Project: Camel
Issue Type: Bug
Components: camel-google-pubsub
Affects Versions: 4.21.0, 4.20.0, 4.19.0, 4.18.3, 4.18.0
Reporter: Henrik
Attachments:
0001-CAMEL-24315-camel-google-pubsub-stop-subscribers-tha.patch
Since CAMEL-22898 (4.18.0), GooglePubsubConsumer adds a subscriber to the
subscribers list only after startAsync().awaitRunning() returns. doStop() and
deferShutdown() only stop subscribers already in that list, so stopping the
consumer while a subscriber is still starting misses it. Nothing ever calls
stopAsync() on it, and the wrapper thread parks forever in awaitTerminated(),
an uninterruptible Guava wait.
The consumer executor then never terminates: context shutdown waits 2 x
shutdownAwaitTermination (default 10s) per stuck pool and the thread leaks even
after forced shutdown. Test suites that stop many CamelContexts (e.g. Spring
Boot integration tests) pay up to 20s per leaked consumer per context; we saw a
CI build go from 5 to 25 minutes from this alone.
The synchronous pull path has the same window: the pull future is added to
pendingSynchronousPullResponses only after futureCall() returns, so a stop
landing in between misses it and get() blocks until the pull deadline instead
of being cancelled.
Fix: after registering the subscriber or pull future, re-check the consumer
state and stop/cancel it if the consumer is no longer allowed to run. stop()
sets the volatile service status to STOPPING before doStop() runs, so either
doStop() sees the subscriber in the list or the re-check sees the stopped
status. Unit tests reproduce both races deterministically and fail without the
fix.
Workaround on affected versions: lower shutdownAwaitTermination to bound the
shutdown delay; the thread still leaks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
