Kai Levy created CAMEL-16790:
--------------------------------
Summary: camel-pulsar is prone to uneven message distribution with
large backlog
Key: CAMEL-16790
URL: https://issues.apache.org/jira/browse/CAMEL-16790
Project: Camel
Issue Type: Bug
Components: camel-pulsar
Environment: camel 3.5.0, pulsar 2.7.1, unix
Reporter: Kai Levy
I have filed a bug in the pulsar project about uneven distribution of messages
using the Java client: [https://github.com/apache/pulsar/issues/11008] - there
are some reproduction instructions in this ticket. This means that for large
backlogs, only the instances that were connected when the messages were
produced will be able to process the messages.
The issue occurs when you use a consumer with `messageListener` and set
`receiverQueueSize > 0`. It appears that the camel-pulsar component uses that
message listener:
[https://github.com/apache/camel/blob/main/components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/utils/consumers/CommonCreationStrategyImpl.java#L54]
We observed the issue in our camel application using camel 3.5.0, but I believe
it occurs on more versions. If the issue is not fixed in the java client, a
convenient workaround could be to use a threadpool calling `consumer.receive()`
in a loop instead of `messageListener`.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
