Technoboy- opened a new pull request #14703: URL: https://github.com/apache/pulsar/pull/14703
Cherry-pick https://github.com/apache/pulsar/pull/14664 ### Motivation pre-require : User config `managedLedgerMaxLedgerRolloverTimeMinutes > 0`. Then, if ManagedLedger creates a ledger in the below case : https://github.com/apache/pulsar/blob/7998c44b0b85e8ae1af5fec64d8f873032877a2f/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L778-L785 The ManagedLedger state now is `CreatingLedger `. At this moment, rollover is triggered, it will set the state to `ClosingLedger `(line-1674) https://github.com/apache/pulsar/blob/7998c44b0b85e8ae1af5fec64d8f873032877a2f/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L1671-L1692 And invoke `createLedgerAfterClosed -> isNeededCreateNewLedgerAfterCloseLedger` is passing, because `isNeededCreateNewLedgerAfterCloseLedger` checks `CreatingLedger ` and `LedgerOpened ` state, but the current state is `ClosingLedger`: https://github.com/apache/pulsar/blob/7998c44b0b85e8ae1af5fec64d8f873032877a2f/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L1648-L1667 So it will also create a new ledger. And result in the message being lost. It's the same result with https://github.com/apache/pulsar/issues/12221 #### Detail log: ``` 2022-03-04T01:41:49.908333235+02:00 2022-03-03T23:41:49,907 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/task_topic-partition-0] Closing ledger 1846355 for being full 2022-03-04T01:41:49.921447018+02:00 2022-03-03T23:41:49,920 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/task_topic-partition-0] Creating a new ledger 2022-03-04T01:41:49.921482783+02:00 2022-03-03T23:41:49,921 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/itomdipostload/persistent/task_topic-partition-0] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 105, 116, 111, 109, 100, 105, 112, 111, 115, 116, 108, 111, 97, 100, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 100, 105, 95, 112, 111, 115, 116, 108, 111, 97, 100, 95, 116, 97, 115, 107, 95, 116, 111, 112, 105, 99, 45, 112, 97, 114, 116, 105, 116, 105, 111, 110, 45, 48], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 2022-03-04T01:41:49.926896931+02:00 2022-03-03T23:41:49,926 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/task_topic-partition-0] Start checking if current ledger is full 2022-03-04T01:41:49.926944096+02:00 2022-03-03T23:41:49,926 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/task_topic-partition-0] Creating a new ledger 2022-03-04T01:41:49.927043799+02:00 2022-03-03T23:41:49,926 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/task_topic-partition-0] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 105, 116, 111, 109, 100, 105, 112, 111, 115, 116, 108, 111, 97, 100, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 100, 105, 95, 112, 111, 115, 116, 108, 111, 97, 100, 95, 116, 97, 115, 107, 95, 116, 111, 112, 105, 99, 45, 112, 97, 114, 116, 105, 116, 105, 111, 110, 45, 48], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 2022-03-04T01:41:49.936780564+02:00 2022-03-03T23:41:49,936 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/task_topic-partition-0] Created new ledger 1846376 2022-03-04T01:41:49.944880616+02:00 2022-03-03T23:41:49,944 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/task_topic-partition-0] Created new ledger 1846377 ``` ### Modification - Rollover only with ledger `LedgerOpened` state. - Modify log to help troubleshoot problem. ### Documentation - [x] `no-need-doc` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
