dao-jun commented on PR #22452: URL: https://github.com/apache/pulsar/pull/22452#issuecomment-2046921479
> > I just don't understand, it seems ReplicatorSubscriptionController only snapshots `ledger.lastPosition`, why do we update `lastDataMessagePublishedTimestamp` after `maxReadPosition` moved? > > We may need to have a discussion about which time point should we update `lastDataMessagePublishedTimestamp`. There are two options: > > * update `lastDataMessagePublishedTimestamp` whenever a transactional/non-transactional message is persisted. > * update `lastDataMessagePublishedTimestamp` when the max position move forward. > > The difference of these two options is that the frequency of the former is greater than the latter. With greater frequency, we have a more precice position for ReplicatorSubscription. But because the consumer can't consume the messages after the max read position, and the number of snapshot is limited(default 10), there are high risk that when a consumer ack a position belonging to a committed transaction, the corresponding snapshots are already evicted(the frequency of snapshot is very fast, a transaction last for more than 10s will meet such case in default configuration), thus the `ReplicatorSubscription` feature can't work at all. Looking for your reply. @lhotari @codelipenghui @liangyepianzhou @dao-jun @Technoboy- +1 with option 1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
