Nicklee007 opened a new issue, #15473:
URL: https://github.com/apache/pulsar/issues/15473

   **Describe the bug**
   In some case, expired ledger data cannot be cleanup when the topic has not 
new message and ledger consumed completely, because the slowestReaderPosition 
which set to the new ledger will be cover to the old ledger by some cursor 
operator which like flink reset the Durable Cursor and the auto  flushed dirty 
mark-delete position.
   
   Also, the last expired ledger can not be clean if the topic has not active 
consumer and expired the message by MessageExpiryMonitor or the cursor 
markDeletePosition are in some old ledger  when stop produce message.
   
   <img width="1218" alt="image" 
src="https://user-images.githubusercontent.com/84127069/167107771-5390825d-45e1-40d9-b58f-c73776097281.png";>
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. For quickly reproduce we change the config 
managedLedgerMaxLedgerRolloverTimeMinutes and 
managedLedgerMinLedgerRolloverTimeMinutes to few  minutes.
   2. start broker, create topic and  set-message-ttl and set-retention to 4 *  
managedLedgerMaxLedgerRolloverTimeMinutes;
   3. create Subscription and produce some message longer than set-message-ttl;
   4.  stop producer and wait the message expired by MessageExpiryMonitor;
   5. the are two ledger. The has data ledger's all message are expired but can 
not be cleanup.
   
   **Expected behavior**
   All expired ledger data should be cleanup. There are many  expired ledger's 
data can not be clean , If we have many not new message's topic. we also need 
migrate the ledger data when we offline one bookie. 
   
   
   **Additional context**
   The bug can be reproduce before the pulsar version 2.10.1, the current 
master pr [#14672](https://github.com/apache/pulsar/pull/14672) change the 
behavior of rollCurrentLedgerIfFull when roll full ledger could not create new 
ledger, I think should resume the behavior to create new ledger.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to