[
https://issues.apache.org/jira/browse/CAMEL-8221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen updated CAMEL-8221:
-------------------------------
Fix Version/s: Future
> No backpressure in aws-sqs Consumer
> -----------------------------------
>
> Key: CAMEL-8221
> URL: https://issues.apache.org/jira/browse/CAMEL-8221
> Project: Camel
> Issue Type: Improvement
> Components: camel-aws
> Affects Versions: 2.14.1
> Reporter: Dan Brown
> Fix For: Future
>
>
> We're using a camel Consumer with an aws-sqs endpoint and running into issues
> with large queues and slow consumers. The consumer jvm downloads messages
> very fast from sqs, onto the jvm heap, even though the camel receive method
> processes each message very slowly (and Ack's when done, with autoAck =
> false). The result of this is that the consumer jvm continually fills its
> heap with an unbounded queue of incoming messages and eventually throws OOME.
> To avoid this failure, we're looking for a way to enable backpressure when
> using aws-sqs—e.g. an on-heap bounded queue between the component fetching
> from sqs and the user-defined actor processing the messages—but I don't see
> anything relevant in the config:
> - http://camel.apache.org/aws-sqs.html
> Looking at the code, I see that SqsConsumer subtypes
> ScheduledBatchPollingConsumer, which subtypes ScheduledPollConsumer. To get
> backpressure, should it use something like EventDrivenPollingConsumer
> somewhere, which uses a blocking queue to avoid the heap blowup above?
> -
> https://github.com/apache/camel/blob/camel-2.14.1/components/camel-aws/src/main/java/org/apache/camel/component/aws/sqs/SqsConsumer.java#L52-L57
> -
> https://github.com/apache/camel/blob/camel-2.14.1/camel-core/src/main/java/org/apache/camel/impl/ScheduledBatchPollingConsumer.java#L31-L34
> -
> https://github.com/apache/camel/blob/camel-2.14.1/camel-core/src/main/java/org/apache/camel/impl/ScheduledPollConsumer.java#L40-L43
> -
> https://github.com/apache/camel/blob/camel-2.14.1/camel-core/src/main/java/org/apache/camel/impl/EventDrivenPollingConsumer.java#L35-L42
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
