Gerda Ilger created CAMEL-18472:
-----------------------------------
Summary: deadlock when concurrently adding and removing routes
Key: CAMEL-18472
URL: https://issues.apache.org/jira/browse/CAMEL-18472
Project: Camel
Issue Type: Improvement
Components: came-core
Affects Versions: 3.7.7, 3.19.0
Reporter: Gerda Ilger
As discussed on [email protected], i am opening a Jira issue +
We ran into the following issue when dynamically adding and removing ftp
routes, whose consumers sometimes take a longer time to start up:
Deadlock when concurrently adding and removing routes
* {{Default/Abstract)CamelContext.addRoutes}} ->
{{DefaultCamelContext.addRouteDefinitions}} locks {{DefaultModel}} first, and
is then blocked trying to lock {{DefaultCamelContext}}
* {{(Default/Abstract)CamelContext.removeRoute}} locks {{DefaultCamelContext}}
first, and is then blocked trying to lock {{DefaultModel}}
{*}Threaddump{*}:
{{Found one Java-level deadlock:}}
{{=============================}}
{{"pool-2-thread-1":}}
{{waiting to lock monitor 0x00007fd22000b0c0 (object 0x000000068c800170, a
org.apache.camel.impl.DefaultModel),}}
{{{}which is held by "pool-2-thread-3"{}}}{{{}"pool-2-thread-3":{}}}
{{waiting to lock monitor 0x00007fd21404a130 (object 0x000000068c002a20, a
org.apache.camel.impl.DefaultCamelContext),}}
{{{}which is held by "pool-2-thread-2"{}}}{{{}"pool-2-thread-2":{}}}
{{waiting to lock monitor 0x00007fd22000b0c0 (object 0x000000068c800170, a
org.apache.camel.impl.DefaultModel),}}
{{{}which is held by "pool-2-thread-3"{}}}{{{}Java stack information for the
threads listed above:{}}}
{{===================================================}}
{{"pool-2-thread-1":}}
{{at
org.apache.camel.impl.DefaultModel.addRouteDefinitions(DefaultModel.java:163)}}
- {{{}waiting to lock <0x000000068c800170> (a
org.apache.camel.impl.DefaultModel){}}}{{{}at
org.apache.camel.impl.DefaultCamelContext.addRouteDefinitions(DefaultCamelContext.java:351){}}}{{{}at
org.apache.camel.builder.RouteBuilder.populateRoutes(RouteBuilder.java:723){}}}{{{}at
org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:569){}}}{{{}at
org.apache.camel.impl.engine.AbstractCamelContext.addRoutes(AbstractCamelContext.java:1186){}}}{{{}at
org.apache.camel.impl.DynamicallyConcurrentlyAddRoutesTest.lambda$1(DynamicallyConcurrentlyAddRoutesTest.java:64){}}}{{{}at
org.apache.camel.impl.DynamicallyConcurrentlyAddRoutesTest$$Lambda$487/0x00000008002c52d0.call(Unknown
Source){}}}{{{}at
java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264){}}}{{{}at
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1136){}}}{{{}at
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635){}}}{{{}at
java.lang.Thread.run([email protected]/Thread.java:833){}}}{{{}"pool-2-thread-3":{}}}{{{}at
org.apache.camel.impl.engine.AbstractCamelContext.removeRoute(AbstractCamelContext.java:1400){}}}
- {{{}waiting to lock <0x000000068c002a20> (a
org.apache.camel.impl.DefaultCamelContext){}}}{{{}at
org.apache.camel.impl.DefaultModel.removeRouteDefinition(DefaultModel.java:212){}}}
- {{{}locked <0x000000068c800170> (a
org.apache.camel.impl.DefaultModel){}}}{{{}at
org.apache.camel.impl.DefaultModel.removeRouteDefinitions(DefaultModel.java:201){}}}
- {{{}locked <0x000000068c800170> (a
org.apache.camel.impl.DefaultModel){}}}{{{}at
org.apache.camel.impl.DefaultModel.addRouteDefinitions(DefaultModel.java:179){}}}
- {{{}locked <0x000000068c800170> (a
org.apache.camel.impl.DefaultModel){}}}{{{}at
org.apache.camel.impl.DefaultCamelContext.addRouteDefinitions(DefaultCamelContext.java:351){}}}{{{}at
org.apache.camel.builder.RouteBuilder.populateRoutes(RouteBuilder.java:723){}}}{{{}at
org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:569){}}}{{{}at
org.apache.camel.impl.engine.AbstractCamelContext.addRoutes(AbstractCamelContext.java:1186){}}}{{{}at
org.apache.camel.impl.DynamicallyConcurrentlyAddRoutesTest.lambda$1(DynamicallyConcurrentlyAddRoutesTest.java:64){}}}{{{}at
org.apache.camel.impl.DynamicallyConcurrentlyAddRoutesTest$$Lambda$487/0x00000008002c52d0.call(Unknown
Source){}}}{{{}at
java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264){}}}{{{}at
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1136){}}}{{{}at
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635){}}}{{{}at
java.lang.Thread.run([email protected]/Thread.java:833){}}}{{{}"pool-2-thread-2":{}}}{{{}at
org.apache.camel.impl.DefaultModel.getRouteDefinition(DefaultModel.java:237){}}}
- {{{}waiting to lock <0x000000068c800170> (a
org.apache.camel.impl.DefaultModel){}}}{{{}at
org.apache.camel.impl.DefaultCamelContext.getRouteDefinition(DefaultCamelContext.java:343){}}}{{{}at
org.apache.camel.impl.DefaultCamelContext.removeRoute(DefaultCamelContext.java:959){}}}
- {{{}locked <0x000000068c002a20> (a
org.apache.camel.impl.DefaultCamelContext){}}}{{{}at
org.apache.camel.impl.engine.AbstractCamelContext.removeRoute(AbstractCamelContext.java:1400){}}}
- {{{}locked <0x000000068c002a20> (a
org.apache.camel.impl.DefaultCamelContext){}}}{{{}at
org.apache.camel.impl.DynamicallyConcurrentlyAddRoutesTest.lambda$1(DynamicallyConcurrentlyAddRoutesTest.java:77){}}}{{{}at
org.apache.camel.impl.DynamicallyConcurrentlyAddRoutesTest$$Lambda$487/0x00000008002c52d0.call(Unknown
Source){}}}{{{}at
java.util.concurrent.FutureTask.run([email protected]/FutureTask.java:264){}}}{{{}at
java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1136){}}}{{{}at
java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635){}}}{{{}at
java.lang.Thread.run([email protected]/Thread.java:833){}}}
{{Found 1 deadlock.}}
My JUnit test executed successfully when I changed {{DefaultCamelContext}} to
obtain the lock on {{model}} before locking {{camelContext}} in
{{{}removeRoute{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
