[
https://issues.apache.org/jira/browse/CAMEL-23975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
NITESH KUMAR updated CAMEL-23975:
---------------------------------
Summary: java.util.concurrent.RejectedExecutionException: CamelContext is
stopped : at runTime (Hazelcast MapLoader Calls Route with producer Template))
(was: java.util.concurrent.RejectedExecutionException: CamelContext is stopped
: at runTime )
> java.util.concurrent.RejectedExecutionException: CamelContext is stopped :
> at runTime (Hazelcast MapLoader Calls Route with producer Template))
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-23975
> URL: https://issues.apache.org/jira/browse/CAMEL-23975
> Project: Camel
> Issue Type: Bug
> Environment: Prod
> Reporter: NITESH KUMAR
> Priority: Major
>
> Hi Team,
> I am facing an issue with an Apache Camel application running on Spring Boot
> in a clustered environment and would appreciate your guidance.
> *🔧* *Setup Details*
> * *Framework:* Spring Boot(3.5x) with Apache Camel (4.10.9)
> * *Deployment:* 5 EC2 instances running the same service
> * *Caching:* Embedded Hazelcast cluster (5.5.0)
> * *Usage:* Hazelcast distributed IMap with a custom MapLoader implementation
> *⚠️* *Issue Description*
> During application startup, specifically while Hazelcast performs {*}initial
> cache population{*}, we intermittently encounter the following exception:
> java.util.concurrent.RejectedExecutionException: CamelContext is stopped
>
> But My IMaps are LAZY and executes at the time of message processing
> indicates routes & camel Context are up
> Note : This Exception We Also gets intermittently when cache reloads after
> eviction.
> *🔍* *Stack Trace (Relevant Section)*
> at
> com.hazelcast.map.impl.MapStoreWrapper.lambda$load$7(MapStoreWrapper.java:176)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.callWithClassLoader(NamespaceUtil.java:200)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.callWithOwnClassLoader(NamespaceUtil.java:152)
> at
> com.hazelcast.map.impl.MapStoreWrapper.load(MapStoreWrapper.java:176)
> at
> com.hazelcast.map.impl.mapstore.writethrough.WriteThroughStore.load(WriteThroughStore.java:88)
> at
> com.hazelcast.map.impl.mapstore.writethrough.WriteThroughStore.load(WriteThroughStore.java:31)
> at
> com.hazelcast.map.impl.recordstore.DefaultRecordStore.loadValueWithExpiry(DefaultRecordStore.java:494)
> at
> com.hazelcast.map.impl.operation.steps.GetOpSteps$2.runStep(GetOpSteps.java:58)
> at
> com.hazelcast.map.impl.operation.steps.GetOpSteps$2.runStep(GetOpSteps.java:48)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepSupplier.runStepWithState(StepSupplier.java:203)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepSupplier$1.run(StepSupplier.java:146)
> at
> com.hazelcast.internal.namespace.impl.NoOpUserCodeNamespaceService.runWithNamespace(NoOpUserCodeNamespaceService.java:82)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.runWithNamespace(NamespaceUtil.java:106)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepRunner.runDirect(StepRunner.java:234)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepRunner.run(StepRunner.java:176)
> at
> com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:220)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> at
> com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
> at
> com.hazelcast.internal.util.executor.PoolExecutorThreadFactory$ManagedThread.executeRun(PoolExecutorThreadFactory.java:74)
> at
> com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111)
> Caused by: java.util.concurrent.RejectedExecutionException: CamelContext is
> stopped
> at
> org.apache.camel.support.cache.DefaultProducerCache.send(DefaultProducerCache.java:155)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:175)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:171)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.requestBody(DefaultProducerTemplate.java:413)
> ... 25 common frames omitted
> *🔍* *Key Implementation Detail*
> We are using a custom Hazelcast MapLoader, where:
> * loadAllKeys() is triggered during first message processing as maps are
> lazy. (indirectly camel is fully UP)
> * Inside loadAllKeys(), we invoke a Camel route using ProducerTemplate
> *Example:*
> Java
> @Override
> public Iterable<K> loadAllKeys() {
> producerTemplate.requestBody("direct:loadKeysRoute", payload);
> return keys;
> }
>
> *🔍* *Observations*
> * Issue occurs primarily during *initial cache loading*
> * Sometimes observed during steady-state runtime
> * Occurs across multiple EC2 nodes concurrently
> * No explicit shutdown or stop is triggered from application code
> * CamelContext stops automatically after the exception
> * Suspected causes:
> * Interaction between Hazelcast-managed threads and Camel execution
>
> *📉* *Impact*
> * Message processing is interrupted
>
> *✅* *Request*
> Kindly help with:
> * Root cause analysis
> * Recommended configuration or architectural changes
> * Preventive measures to avoid CamelContext shutdown in such scenarios
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
