[
https://issues.apache.org/jira/browse/CAMEL-23975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094978#comment-18094978
]
NITESH KUMAR commented on CAMEL-23975:
--------------------------------------
Hi [~davsclaus] , What are the way to Overcome this (anyway to make Hz Coupled
with camel lifecycle). My Understanding is if Application is running means
camel context should be in started state.
Please Help with this.
> java.util.concurrent.RejectedExecutionException: CamelContext is stopped :
> at runTime (Hazelcast MapLoader Calls Route with producer Template))
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-23975
> URL: https://issues.apache.org/jira/browse/CAMEL-23975
> Project: Camel
> Issue Type: Bug
> Affects Versions: 4.10.9
> Environment: Prod
> Reporter: NITESH KUMAR
> Priority: Major
>
> Hi Team,
> I am facing an issue with an Apache Camel application running on Spring Boot
> in a clustered environment and would appreciate your guidance.
> *🔧* *Setup Details*
> * *Framework:* Spring Boot(3.5x) with Apache Camel (4.10.9)
> * *Deployment:* 5 EC2 instances running the same service
> * *Caching:* Embedded Hazelcast cluster (5.5.0)
> * *Usage:* Hazelcast distributed IMap with a custom MapLoader implementation
> *⚠️* *Issue Description*
> During application startup, specifically while Hazelcast performs {*}initial
> cache population{*}, we intermittently encounter the following exception:
> java.util.concurrent.RejectedExecutionException: CamelContext is stopped
>
> But My IMaps are LAZY and executes at the time of message processing
> indicates routes & camel Context are up
> Note : This Exception We Also gets intermittently when cache reloads after
> eviction.
> *🔍* *Stack Trace (Relevant Section)*
> at
> com.hazelcast.map.impl.MapStoreWrapper.lambda$load$7(MapStoreWrapper.java:176)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.callWithClassLoader(NamespaceUtil.java:200)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.callWithOwnClassLoader(NamespaceUtil.java:152)
> at
> com.hazelcast.map.impl.MapStoreWrapper.load(MapStoreWrapper.java:176)
> at
> com.hazelcast.map.impl.mapstore.writethrough.WriteThroughStore.load(WriteThroughStore.java:88)
> at
> com.hazelcast.map.impl.mapstore.writethrough.WriteThroughStore.load(WriteThroughStore.java:31)
> at
> com.hazelcast.map.impl.recordstore.DefaultRecordStore.loadValueWithExpiry(DefaultRecordStore.java:494)
> at
> com.hazelcast.map.impl.operation.steps.GetOpSteps$2.runStep(GetOpSteps.java:58)
> at
> com.hazelcast.map.impl.operation.steps.GetOpSteps$2.runStep(GetOpSteps.java:48)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepSupplier.runStepWithState(StepSupplier.java:203)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepSupplier$1.run(StepSupplier.java:146)
> at
> com.hazelcast.internal.namespace.impl.NoOpUserCodeNamespaceService.runWithNamespace(NoOpUserCodeNamespaceService.java:82)
> at
> com.hazelcast.internal.namespace.NamespaceUtil.runWithNamespace(NamespaceUtil.java:106)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepRunner.runDirect(StepRunner.java:234)
> at
> com.hazelcast.map.impl.operation.steps.engine.StepRunner.run(StepRunner.java:176)
> at
> com.hazelcast.internal.util.executor.CachedExecutorServiceDelegate$Worker.run(CachedExecutorServiceDelegate.java:220)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> at
> com.hazelcast.internal.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76)
> at
> com.hazelcast.internal.util.executor.PoolExecutorThreadFactory$ManagedThread.executeRun(PoolExecutorThreadFactory.java:74)
> at
> com.hazelcast.internal.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:111)
> Caused by: java.util.concurrent.RejectedExecutionException: CamelContext is
> stopped
> at
> org.apache.camel.support.cache.DefaultProducerCache.send(DefaultProducerCache.java:155)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:175)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.send(DefaultProducerTemplate.java:171)
> at
> org.apache.camel.impl.engine.DefaultProducerTemplate.requestBody(DefaultProducerTemplate.java:413)
> ... 25 common frames omitted
> *🔍* *Key Implementation Detail*
> We are using a custom Hazelcast MapLoader, where:
> * loadAllKeys() is triggered during first message processing as maps are
> lazy. (indirectly camel is fully UP)
> * Inside loadAllKeys(), we invoke a Camel route using ProducerTemplate
> *Example:*
> Java
> @Override
> public Iterable<K> loadAllKeys() {
> producerTemplate.requestBody("direct:loadKeysRoute", payload);
> return keys;
> }
>
> *🔍* *Observations*
> * Issue occurs primarily during *initial cache loading*
> * Sometimes observed during steady-state runtime
> * Occurs across multiple EC2 nodes concurrently
> * No explicit shutdown or stop is triggered from application code
> * CamelContext stops automatically after the exception
> * Suspected causes:
> * Interaction between Hazelcast-managed threads and Camel execution
>
> *📉* *Impact*
> * Message processing is interrupted
>
> *✅* *Request*
> Kindly help with:
> * Root cause analysis
> * Recommended configuration or architectural changes
> * Preventive measures to avoid CamelContext shutdown in such scenarios
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
