shashank created CAMEL-25003:
--------------------------------
Summary: camel-core - ServicePool hands out a stopped
non-singleton producer after LRU eviction, and stops producers in use
Key: CAMEL-25003
URL: https://issues.apache.org/jira/browse/CAMEL-25003
Project: Camel
Issue Type: Bug
Components: camel-core
Reporter: shashank
For endpoints whose producer is not a singleton ({{isSingletonProducer() ==
false}}: camel-ftp/sftp/ftps, camel-smb, camel-ssh, and the polling consumers
used by pollEnrich), {{ServicePool}} keeps one {{MultiplePool}} per endpoint (a
queue of idle producers). It also keeps an LRU cache, of size {{cacheSize}}, of
every producer it handed out.
When the LRU evicts such a producer, {{onEvict}} only adds it to the pool's
{{evicts}} list. The next {{acquire}} or {{release}} on that pool runs
{{cleanupEvicts}}, which stops the producer. But cleanupEvicts does not take
the producer out of the pool queue:
# If the producer was idle, the {{acquire}} that polls the queue right after
{{cleanupEvicts}} hands it out, stopped.
# If the producer was in use by another exchange, it is stopped under that
exchange. When that exchange releases it, {{release}} offers it back to the
queue, and it is handed out again, stopped.
Any workload that uses more producer instances than {{cacheSize}} evicts
producers that are idle in a queue or in use.
Reproduced against the real classes, using a test component whose producers are
non-singleton and which records use after stop. With
{{from("direct:in").toD("pooled:${header.to}", 2)}}, 4 threads and 2000
exchanges to two endpoints:
{noformat}
created=222 stopped=222 usedAfterStop=1664 stopWhileInUse=1
{noformat}
1664 of 2000 exchanges were sent with a stopped producer.
Handing out a stopped producer is a regression from CAMEL-20829 (4.7), which
removed the {{queue.remove(evict)}} from {{cleanupEvicts}}.
Found with a TLA+ model of ServicePool (pooled producers, LRU eviction,
concurrent acquire/release). The invariants "never used after stop", "never
stopped in use" and "queue holds only started producers" are violated.
Proposed fix, in {{ServicePool.MultiplePool}}:
* {{evict}} takes an idle producer out of the queue before it is stopped, and
leaves a producer in use alone.
* {{release}} stops a producer that was evicted while in use, instead of
offering it back to the queue.
* {{onEvict}} no longer stops a producer whose pool is gone. At that point the
producer can only be in use, and {{release}} stops it, as it does since
CAMEL-24248.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
