Have you changed the backlog quota policy or enabled the message TTL? Pulsar will not remove any cursors or skip messages by default.
Penghui On Mar 11, 2022, 1:25 PM +0800, Tecno Brain <cerebrotecnolog...@gmail.com>, wrote: > Hello, > I have an application using Pulsar just as JMS (we get single messages, > acknowledge them when we are done processing it) > The entire system, composed of several types of apps, uses about 40 > different topics. > > Yesterday, an application that subscribes to about 20 queues, suddenly was > inundated with thousands of messages from two of the queues but I could not > track those messages to an application producing them. We found that the > messages were duplicates. > So it seems that the cursor to these two topics was lost, and messages from > 3 hours earlier were consumed again. I found the following paragraph : > > "Each subscription stores a cursor. The cursor is the current offset in the > log. Subscriptions store their cursor in BookKeeper in ledgers. This makes > cursor tracking scalable just like topics." > > (https://jack-vanlightly.com/blog/2018/10/2/understanding-how-apache-pulsar-works) > > My guess is that the cursor was lost. > How could I verify that this was the case? I can't find anything relevant in > the logs. > > The only message I found occurring around the same time is > > New ensemble: [pulsar-bookie-2.pulsar-bookie.pulsar.svc.cluster.local:3181, > pulsar-bookie-0.pulsar-bookie.pulsar.svc.cluster.local:3181, > pulsar-bookie-3.pulsar-bookie.pulsar.svc.cluster.local:3181] is not adhering > to Placement Policy > > Any pointers are appreciated. > > >
