Krzysztof Jamróz created CAMEL-17544:
----------------------------------------
Summary: ServicePool.doStop still hangs during shutdown
Key: CAMEL-17544
URL: https://issues.apache.org/jira/browse/CAMEL-17544
Project: Camel
Issue Type: Bug
Affects Versions: 3.14.0
Reporter: Krzysztof Jamróz
Attachments: ServicePoolShutdownTest.java
{{ServicePool.doStop}} still hangs during shutdown with optimized fix of
CAMEL-17536. {{LinkedHashMap}} in cache is corrupted not only because of race
condtion between {{acquire}} and {{doStop}} but also between concurrent
invocations of {{acquire}} during parallel routing of exchanges (see attached
{{{}repeatTestRecipientList{}}}).
The root cause is that {{LinkedHashMap}} used by {{SimpleLRUCache}} is *not*
thread-safe but access is not synchronized. Even reads may modify it as it has
LRU policy. And access to {{{}LinkedHashMap{}}}/\{{ SimpleLRUCache }} is
possible during routing (so concurrently) not only during start/stop.
Uses of {{SimpleLRUCache}} in other places in Camel may also exhibit race
conditions. It is hard to demonstrate them reliably but I have another example
({{{}repeatEndpointsTest{}}}). This one usually crashes due to {{OutOfMemory}}
when converting corrupted (looping) {{SimpleLRUCache}} in {{EndpointRegistry}}
to array, but I got also other exceptions.
{noformat}
java.lang.OutOfMemoryError: Java heap space
at java.base/java.util.Arrays.copyOf(Arrays.java:3480)
at
java.base/java.util.AbstractCollection.finishToArray(AbstractCollection.java:227)
at
java.base/java.util.AbstractCollection.toArray(AbstractCollection.java:148)
at java.base/java.util.ArrayList.<init>(ArrayList.java:181)
at
org.apache.camel.impl.engine.AbstractCamelContext.shutdownServices(AbstractCamelContext.java:3582)
at
org.apache.camel.impl.engine.AbstractCamelContext.shutdownServices(AbstractCamelContext.java:3576)
at
org.apache.camel.impl.engine.AbstractCamelContext.doStop(AbstractCamelContext.java:3414)
at org.apache.camel.support.service.BaseService.stop(BaseService.java:160)
at
org.apache.camel.impl.engine.AbstractCamelContext.stop(AbstractCamelContext.java:2658)
at
org.apache.camel.component.http.ServicePoolShutdownTest.testEndpoints(ServicePoolShutdownTest.java:129)
at
org.apache.camel.component.http.ServicePoolShutdownTest.repeatEndpointsTest(ServicePoolShutdownTest.java:135){noformat}
Caffeine, which used to be default cache implementation until CAMEL-16093 is
thread-safe. Enabling caffeine-lrucache might be a workaround.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
