shashank created CAMEL-27336:
--------------------------------

             Summary: camel-core - an evicted singleton producer (and the 
endpoint of a toD) is stopped while it is in use, and a stale eviction entry of 
ServicePool stops the wrong producer
                 Key: CAMEL-27336
                 URL: https://issues.apache.org/jira/browse/CAMEL-27336
             Project: Camel
          Issue Type: Bug
          Components: camel-core
            Reporter: shashank
            Assignee: shashank


h3. What happens

{{ServicePool.SinglePool}} (the pool of a singleton producer or polling 
consumer, used by every {{DefaultProducerCache}}: toD, recipientList, 
routingSlip, enrich, ProducerTemplate) keeps no record of who uses its 
producer. When the LRU cache of the producer cache evicts it:

# {{onEvict}} stops the pool at once when the pool map is larger than the 
capacity (the usual case once the cache is full), or marks it in 
{{singlePoolEvicted}} so that the next acquire or release of any single pool 
stops it. Neither checks whether the producer is in use, so an in-flight send 
runs on a stopped producer. For a dynamic endpoint (toD) the cleanup then also 
finds the endpoint unused, and stops it and removes it from the registry during 
the send. With the default LRU cache a hit does not refresh the LRU order, so 
the evicted producer is the one created first, often the busiest one. 
{{MultiplePool}} protects its producers in use since CAMEL-25003.
# {{onEvict}} both marks the pool and stops it, so the mark stays behind. When 
the next {{acquire}} of the same endpoint has already created a new pool, the 
stale mark's cleanup removes the new pool from the map 
({{pool.remove(endpoint)}}, not {{remove(endpoint, this)}}), stops the 
endpoint, and the new pool hands out a producer of the stopped endpoint. The 
next acquire creates a second producer of the singleton endpoint, and a later 
eviction of the first one stops the current producer instead 
({{SinglePool.evict(s)}} ignores {{s}}) and leaks the evicted one until the 
CamelContext stops.
# {{SinglePool.acquire}} reads the volatile producer field twice (check, then 
return): an eviction in between makes it return null, and 
{{DefaultProducerCache}} fails with "No producer, this processor has not been 
started".

{{DefaultProducerCache}}'s {{lastUsedProducer}} fast path also hands out a 
producer without the pool, so it can return a producer that an eviction stopped.

CAMEL-25066 fixed the endpoint stop when another producer cache uses the 
endpoint; these are different gaps in the same eviction code.

h3. How to reproduce

{{DefaultProducerCache}} with cache size 1: a worker sends to {{my:a}} and 
waits inside the producer's {{process()}}, another worker sends to {{my:b}}. 
The producer of {{my:a}} and the endpoint are stopped while the first send is 
still in {{process()}}. With {{ProducerServicePool}} (capacity 1): 
{{acquire(a)}}, {{release(a)}}, {{acquire(b)}} (held), {{acquire(a)}} returns a 
producer of a stopped endpoint, and a second {{acquire(a)}} a different 
producer.

Found with a TLA+ model of {{ServicePool}}.

h3. Fix

{{SinglePool}} counts the users of its producer (acquire, the fast path and 
release). An eviction takes the pool out of the pool map at once (so the 
endpoint gets a new pool) and the producer is stopped, with the endpoint if 
nothing else uses it, by the eviction when the producer is idle or by its last 
release. The evicted pools in use are kept by producer, so a release finds 
them, and a stale eviction of a producer that is no longer the pool's producer 
is ignored. {{acquire}} returns a local copy, and retries with a new pool when 
the pool was evicted meanwhile.

_Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to