[
https://issues.apache.org/jira/browse/CAMEL-16696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jawad Ahmad resolved CAMEL-16696.
---------------------------------
Resolution: Invalid
> Enrich and Aggregate Causing OutOfMemory
> ----------------------------------------
>
> Key: CAMEL-16696
> URL: https://issues.apache.org/jira/browse/CAMEL-16696
> Project: Camel
> Issue Type: Bug
> Components: came-core
> Affects Versions: 3.10.0
> Reporter: Jawad Ahmad
> Priority: Major
>
> I have a route that receives the bulk amount of data and then processes data
> individually from 3rd Party API using Enrich Pattern. After Enriching I am
> then aggregating the responses into a batch of 100 and then sending them to
> our system. This causes the camel to allocate memory like crazy, stuck into
> the loop and eventually going out of memory. I tried with Sync routes as well
> but it gets failed.
> Here is the route configuration :
> {code:java}
> from("direct:sync-likes-from-sf-reviews")
> .id("direct:sync-likes-from-sf-reviews")
> .process(this::convertReviewPayload)
> .split(body())
> .to("direct:enrich-and-fetch-likes")
> .end();
> from("direct:enrich-and-fetch-likes")
> .id("direct-enrich-and-fetch-likes")
> .enrich("direct:fetch-like-by-sf-review", new
> ReviewLikesAggregation(), true)
> .aggregate(constant(true), new
> GroupedBodyAggregationStrategy())
> .completionSize(100)
> .to("direct:upsert-collected-likes")
> .end();
> from("direct:upsert-collected-likes")
> .id("direct:upsert-collected-likes")
> .multicast()
>
> .to("direct:upsert-consumers-batch").to("direct:upsert-likes-batch")
> .end()
> .end();
> from("direct:upsert-consumers-batch?synchronous=false")
> .to("direct:upsert-consumers-batch")
> .process(this.trustpilotLikeConsumersMapper)
> .process(this::setupConsumerHeaders)
> .to("direct:upsert-collections")
> .end();
> from("direct:upsert-likes-batch?synchronous=false")
> .to("direct:upsert-likes-batch")
> .process(this.trustPilotLikesMapper)
> .process(this::setupLikesHeaders)
> .to("direct:upsert-collections")
> .end();
> {code}
> Logs Loop :
> 2021-06-08 12:41:18.568 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.568 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
> 2021-06-08 12:41:18.569 DEBUG 11896 --- [ent@65b17373-56]
> o.apache.camel.processor.SendProcessor : >>>>
> direct://upsert-consumers-batch Exchange[]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
