[
https://issues.apache.org/jira/browse/CAMEL-25843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18130585#comment-18130585
]
Claus Ibsen commented on CAMEL-25843:
-------------------------------------
Merged to main via https://github.com/apache/camel/pull/28264 (commit
8e939dc95ed1).
> camel-hazelcast - a reliable topic consumer that falls behind the ringbuffer
> is removed by Hazelcast and receives nothing again
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25843
> URL: https://issues.apache.org/jira/browse/CAMEL-25843
> Project: Camel
> Issue Type: Bug
> Components: camel-hazelcast
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> With {{reliable=true}} the hazelcast-topic consumer
> ({{HazelcastTopicConsumer}}, main 20cdb309f7d0) listens to a reliable topic
> with a plain {{MessageListener}} ({{CamelMessageListener}}). A reliable topic
> keeps its messages in a ringbuffer of limited capacity (10000 by default,
> optionally with a time to live). When a listener falls behind by more than
> that, the messages it has not read yet are overwritten. Hazelcast then
> removes a listener that is not loss tolerant, for good:
> {{MessageRunner.accept}} (Hazelcast 5.4.0) calls {{cancel()}} when the read
> result shows lost messages and {{isLossTolerable}} is false, and a plain
> {{MessageListener}} is wrapped in {{ReliableMessageListenerAdapter}}, whose
> {{isLossTolerant()}} returns false.
> So one burst that the route cannot keep up with (the route runs on
> Hazelcast's listener thread; a slow processor, a downstream system that slows
> down) ends the consumer: the route stays started, receives nothing again
> until it is restarted, and Camel logs nothing (Hazelcast logs one warning
> through its own logging). The overwritten messages are lost either way; the
> issue is that every message after them is lost too.
> *Fix*: the reliable topic consumer registers
> {{CamelReliableMessageListener}}, a {{ReliableMessageListener}} that is loss
> tolerant: it logs a WARN with the number of messages lost ("The consumer of
> the reliable topic news lost 21 messages: they were overwritten in the
> ringbuffer before they were received") and goes on with the oldest message
> still in the ringbuffer. It starts with the messages published from then on
> ({{retrieveInitialSequence}} -1), as the plain listener did. The plain topic
> is unchanged.
> *Test*: new {{HazelcastReliableTopicConsumerLossTest}} (an embedded member,
> ringbuffer capacity 10): the route is busy with a first message while 30 more
> are published, then one more message is published. On main it is never
> received: {{mock://news Received message count. Expected: <1> but was: <0>}}.
> With the fix it is, and the WARN reports 21 lost messages. camel-hazelcast
> unit tests with the fix: 243 run, 0 failures, 1 error
> ({{HazelcastUserConfigSerializationFilterWarningTest.warnsForUserClientConfigWithoutFilter}},
> a client that cannot reach a cluster in the test environment; it errors the
> same way without the change).
> *How it was found*: variant analysis of CAMEL-25815 (pattern: the server side
> ends a consumer's subscription and the consumer never subscribes again), then
> the JUnit test.
> *Affected*: main, camel-4.18.x and camel-4.14.x (the same plain listener,
> Hazelcast 5.4.0; read, not run).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
