Randheer Chauhan created CAMEL-25093:
----------------------------------------
Summary: Thread-Safety: Concurrent updateRoutes() Against a Single
CamelContext Causes ConcurrentModificationException
Key: CAMEL-25093
URL: https://issues.apache.org/jira/browse/CAMEL-25093
Project: Camel
Issue Type: Improvement
Components: camel-core, camel-xml-io
Affects Versions: 4.14.2
Reporter: Randheer Chauhan
h2. Background
* We use {{{}camel-spring-boot{}}}, which provides a single {{CamelContext}}
within the Spring Boot application.
* During startup, necessary Camel components and beans are registered into
this single {{{}CamelContext{}}}.
* After the application is running, dynamically-loaded content is deployed
into it. Each deployment unit contains a Camel XML DSL file with one or more
route definitions and bean definitions used by those routes.
* To register these routes and beans into the running {{{}CamelContext{}}},
the following {{camel-support}} API is used:
{code:java}
PluginHelper.getRoutesLoader(camelContext).updateRoutes(resource) {code}
* The number of deployment units can range from a few hundred to several
thousand per application instance.
* Once all units are deployed and their routes and beans are registered, the
application is marked as ready.
----
h2. Problem Statement
* When the above API is invoked *sequentially* for each XML DSL
{{{}Resource{}}}, it works correctly and registers all routes and beans as
expected.
* When the API is invoked *concurrently from multiple threads* — each thread
loading a different XML DSL resource into the same {{CamelContext}} — it fails
with {{ConcurrentModificationException}} originating from multiple
Camel-internal paths.
* The stacktraces below capture the distinct failure patterns observed. All
application-specific frames have been removed; only {{org.apache.camel}} and
{{java.base}} frames are retained.
----
h2. Steps to Reproduce
# Start a Spring Boot application with {{camel-spring-boot}} (single
{{{}CamelContext{}}}).
# After context startup, prepare multiple independent XML DSL resource files —
each containing its own route definitions and bean definitions.
# Submit all resources for loading concurrently via a thread pool, calling
{{PluginHelper.getRoutesLoader(camelContext).updateRoutes(resource)}} from each
thread.
# Observe {{ConcurrentModificationException}} failures from the Camel-internal
paths listed below.
The failures do not occur when the same resources are loaded sequentially (one
{{updateRoutes}} call at a time on the same thread).
----
h2. Observed Failures
h3. Failure 1 — {{ArrayList}} iterator race in {{XmlRoutesBuilderLoader}}
The anonymous {{configureCamel}} callback in {{XmlRoutesBuilderLoader}}
iterates a shared {{ArrayList}} at line 195 while another concurrent thread
structurally modifies it. The iterator's fail-fast {{checkForComodification}}
detects the change and throws.
*Camel classes implicated:*
* {{XmlRoutesBuilderLoader}} — anonymous configure callback iterates a shared
{{ArrayList}} (line 195)
* {{RouteBuilder.checkInitialized}} — reads shared state (line 908)
* {{DefaultRoutesLoader.updateRoutes}} — invoked concurrently from multiple
threads (line 302)
java.util.ConcurrentModificationException
at java.util.ArrayList$Itr.checkForComodification
at java.util.ArrayList$Itr.next
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader$1.configureCamel(XmlRoutesBuilderLoader.java:195)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader$1.configure(XmlRoutesBuilderLoader.java:129)
at
org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:908)
at
org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:832)
at
org.apache.camel.builder.RouteBuilder.updateRoutesToCamelContext(RouteBuilder.java:786)
at
org.apache.camel.impl.engine.DefaultRoutesLoader.updateRoutes(DefaultRoutesLoader.java:302)
at org.apache.camel.spi.RoutesLoader.updateRoutes(RoutesLoader.java:111)
... [caller: route activation on parallel deployment thread]
at java.util.concurrent.CompletableFuture$AsyncSupply.run
at java.util.concurrent.ThreadPoolExecutor.runWorker
at java.lang.Thread.run
{{}}
----
h3. Failure 2 — {{ArrayList.removeIf}} race in {{DefaultModel.addCustomBean}}
Multiple concurrent threads each call
{{XmlRoutesBuilderLoader.registerBeanDefinition}} → {{bindBean}} →
{{DefaultCamelContext.addCustomBean}} → {{{}DefaultModel.addCustomBean{}}}.
Inside {{{}addCustomBean{}}}, {{ArrayList.removeIf}} is called on a shared,
unsynchronized list while another thread is simultaneously modifying that same
list.
*Camel classes implicated:*
* {{DefaultModel.addCustomBean}} (line 855) — invokes {{ArrayList.removeIf}}
on an unsynchronized shared list
* {{DefaultCamelContext.addCustomBean}} (line 554) — delegates directly to
{{DefaultModel}}
* {{XmlRoutesBuilderLoader.bindBean}} (line 435) — calls {{addCustomBean}}
without any external lock
* {{XmlRoutesBuilderLoader.registerBeanDefinition}} (line 415) — entry point
from each parallel thread
java.lang.RuntimeException: Error creating bean: <beanName> of type: <beanType>
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader.registerBeanDefinition(XmlRoutesBuilderLoader.java:423)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader$1.configureCamel(XmlRoutesBuilderLoader.java:197)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader$1.configure(XmlRoutesBuilderLoader.java:129)
at
org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:908)
at
org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:832)
at
org.apache.camel.builder.RouteBuilder.updateRoutesToCamelContext(RouteBuilder.java:786)
at
org.apache.camel.impl.engine.DefaultRoutesLoader.updateRoutes(DefaultRoutesLoader.java:302)
at org.apache.camel.spi.RoutesLoader.updateRoutes(RoutesLoader.java:111)
... [caller: route activation on parallel deployment thread]
at java.util.concurrent.CompletableFuture$AsyncSupply.run
at java.util.concurrent.ThreadPoolExecutor.runWorker
at java.lang.Thread.run
Caused by: java.util.ConcurrentModificationException
at java.util.ArrayList.removeIf
at java.util.ArrayList.removeIf
at org.apache.camel.impl.DefaultModel.addCustomBean(DefaultModel.java:855)
at
org.apache.camel.impl.DefaultCamelContext.addCustomBean(DefaultCamelContext.java:554)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader.bindBean(XmlRoutesBuilderLoader.java:435)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader.registerBeanDefinition(XmlRoutesBuilderLoader.java:415)
{{}}
----
h3. Failure 3 — {{HashMap.computeIfAbsent}} race in {{SimpleRegistry.bind}}
Two concurrent threads call {{SimpleRegistry.bind}} simultaneously.
{{SimpleRegistry}} is backed by a plain {{{}HashMap{}}}. Concurrent
{{HashMap.computeIfAbsent}} calls from two threads cause a structural
modification that the second thread detects as a concurrent modification.
*Camel classes implicated:*
* {{SimpleRegistry.bind}} (line 106) — uses {{HashMap.computeIfAbsent}} with
no synchronization
* {{DefaultRegistry.bind}} (line 202) — delegates to {{SimpleRegistry}}
* {{XmlRoutesBuilderLoader.bindBean}} (line 431) — called concurrently per
resource from each parallel thread
{{}}
java.lang.RuntimeException: Error creating bean: <beanName> of type: <beanType>
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader.registerBeanDefinition(XmlRoutesBuilderLoader.java:423)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader$1.configureCamel(XmlRoutesBuilderLoader.java:197)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader$1.configure(XmlRoutesBuilderLoader.java:129)
at
org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:908)
at
org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:832)
at
org.apache.camel.builder.RouteBuilder.updateRoutesToCamelContext(RouteBuilder.java:786)
at
org.apache.camel.impl.engine.DefaultRoutesLoader.updateRoutes(DefaultRoutesLoader.java:302)
at org.apache.camel.spi.RoutesLoader.updateRoutes(RoutesLoader.java:111)
... [caller: route activation on parallel deployment thread]
at java.util.concurrent.CompletableFuture$AsyncSupply.run
at java.util.concurrent.ThreadPoolExecutor.runWorker
at java.lang.Thread.run
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap.computeIfAbsent
at org.apache.camel.support.SimpleRegistry.bind(SimpleRegistry.java:106)
at org.apache.camel.support.DefaultRegistry.bind(DefaultRegistry.java:202)
at org.apache.camel.spi.Registry.bind(Registry.java:57)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader.bindBean(XmlRoutesBuilderLoader.java:431)
at
org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader.registerBeanDefinition(XmlRoutesBuilderLoader.java:415)
----
h2. Camel Classes Observed in Failures
The following classes appear directly in the failure stacktraces. This is not
an exhaustive list — a full audit of the route-loading and bean-registration
path may reveal additional classes with similar thread-safety gaps.
||Camel Class||Unsafe Operation||Pattern||
|{{org.apache.camel.dsl.xml.io.XmlRoutesBuilderLoader}}|{{{}registerBeanDefinition{}}},
{{{}bindBean{}}}, anonymous {{configureCamel}} — race entry point|1, 2, 3|
|{{org.apache.camel.impl.DefaultModel}}|{{addCustomBean}} —
{{ArrayList.removeIf}} on shared unsynchronized list|2|
|{{org.apache.camel.impl.DefaultCamelContext}}|{{addCustomBean}} — delegates to
{{DefaultModel}} without locking|2|
|{{org.apache.camel.support.SimpleRegistry}}|{{bind}} — plain
{{HashMap.computeIfAbsent}} with no synchronization|3|
|{{org.apache.camel.support.DefaultRegistry}}|{{bind}} — delegates to
{{SimpleRegistry}}|3|
|{{org.apache.camel.builder.RouteBuilder}}|{{{}checkInitialized{}}},
{{{}configureRoutes{}}}, {{updateRoutesToCamelContext}} — shared state read
under concurrent modification|1|
|{{org.apache.camel.impl.engine.DefaultRoutesLoader}}|{{updateRoutes}} —
invoked concurrently per resource, shares {{RouteBuilder}} state|1, 2, 3|
|{{org.apache.camel.spi.RoutesLoader}}|{{updateRoutes}} — {{RoutesLoader}} SPI
entry point for all patterns|1, 2, 3|
----
h2. Expected Solution
With a single {{CamelContext}} hosted in a Spring Boot application, the
internal Camel APIs, SPIs, and classes involved in route and bean registration
should be enhanced to support concurrent invocations of
{{RoutesLoader.updateRoutes}} from multiple threads — each loading a different
XML DSL resource. Based on the observed failures, the changes needed include at
minimum:
* *{{DefaultModel}} / {{DefaultCamelContext}}* — synchronize {{addCustomBean}}
or replace the underlying {{ArrayList}} with a thread-safe equivalent so
concurrent {{removeIf}} calls do not race.
* *{{SimpleRegistry}}* — replace the backing {{HashMap}} with
{{ConcurrentHashMap}} so concurrent {{bind}} calls from multiple threads are
safe.
* *{{XmlRoutesBuilderLoader}}* — ensure each {{updateRoutes}} invocation works
with an independent, thread-locally-scoped builder instance; no mutable state
should be shared across concurrent calls for different resources.
* *{{RoutesLoader}} SPI* — document the thread-safety contract of
{{updateRoutes}} so that callers have a clear expectation.
A broader audit of the full route-loading and bean-registration call path is
likely needed to surface any additional unsynchronized shared state not
captured in these stacktraces.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
