>The problem is that if the ledger is deleted, the next time a client produces a message, a new ledger needs to be opened. This is an operation that may take some time, disrupting latency.
Hi Enrico, I think there may be some misunderstandings. If the state of the managerLedger is `ClosedLedger`, the current ledger is closed and cannot be written to. So when a client sends new messages, it must open a new ledger. This will cause latency regardless of whether the current ledger is deleted, since the current ledger is already closed. In the discussion of the PR[1], we found that there are different behaviors in Pulsar after ledger rollover. 1. If the rollover is triggered in the process of adding an entry, a new ledger will be created depending on whether there are pending write operations. 2. In other cases, a new ledger will be created directly regardless of whether there are pending write operations. In the latest implementation of the PR, for the first situation, a new ledger will be created directly even if there are no pending write operations. Of course, this will increase the overhead of the bookkeeper. But for normal production and consumption scenarios, there should be no situation where the new ledger is not written to for a long time. Even for inactive topics, users can handle this situation according to the inactive_topic_policies. [1] - https://github.com/apache/pulsar/pull/22034 On Tue, Feb 6, 2024 at 9:55 PM Enrico Olivelli <eolive...@gmail.com> wrote: > Xiangying > > Il giorno mar 6 feb 2024 alle ore 13:01 Xiangying Meng > <xiangy...@apache.org> ha scritto: > > > > Dear Community, > > > > I hope this message finds you well. I am writing to discuss a > modification > > to the behavior of deleting the current ledger. As you may know, in > Pulsar, > > the current ledger cannot be deleted because it may still be written to. > > However, there is an exception. When the current ledger is rolled over, > but > > no new messages are written, the current ledger does not change. In this > > case, the current ledger will not be written to, but it is also not > deleted. > > I understand the problem and I was surprised about it the first time I saw > it. > The problem is that if you delete the ledger the next time a client > produces a message the broken > must open a new ledger and this is an operation that may take some > time, disrupting latency. > > It is a trade-off, I know, in production you usually don't need to > release the disk space. > Maybe you have a use case in which you write a lot to a topic, then > you stop writing ? > > Maybe you could "unload" the topic, and that will force a ledger > rollover (with an impact on latency) > > Enrico > > > > > This can be confusing for users, especially when they configure > > `managedLedgerMaxLedgerRolloverTimeMinutes` and `retentionTimeInMinutes`. > > They expect the current ledger to roll over and then be deleted after > > `managedLedgerMaxLedgerRolloverTimeMinutes` and `retentionTimeInMinutes`. > > However, in reality, while the current ledger does rollover, it is not > > deleted. > > > > The purpose of this discussion is to consider deleting the current ledger > > when it is rolled over. The specific implementation can be found at > > https://github.com/apache/pulsar/pull/22034. > > > > Looking forward to a productive discussion. > > > > Best Regards, > > > > xiangying >
