djsweet opened a new issue, #4900: URL: https://github.com/apache/bookkeeper/issues/4900
**BUG REPORT** ***Describe the bug*** If a client creates more than `openFileLimit` ledgers, all taking less storage space than `skipListSizeLimit`, reading from some of the earlier created ledgers results in a spurious ENOLEDGER error if SortedLedgerStorage is in use. ***To Reproduce*** Assuming that the default values of `openFileLimit` (20,000) and `skipListSizeLimit` (64 MB) are in place, 1. Create a ledger with exactly one entry with a one-byte payload 2. Immediately close the new ledger 3. Repeat steps 1 and 2 over 20,000 times 4. Attempt to read from the very first created ledger This attempt will fail with an ENOLEDGER error code. ***Expected behavior*** Reading from the very first ledger should result in its entry being returned. ***Additional context*** The issue actually stems from the LAC being read in [the call to `readLastAddConfirmed` in `ReadEntryProcessorV3`](https://github.com/apache/bookkeeper/blob/773a988ddf8b888a8869c6ae33f696935eba5cdc/bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/ReadEntryProcessorV3.java#L182). Currently, `SortedLedgerStorage`'s `getLastAddConfirmed` delegates [entirely to `InterleavedLedgerStorage`'s `getLastAddConfirmed`](https://github.com/apache/bookkeeper/blob/773a988ddf8b888a8869c6ae33f696935eba5cdc/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SortedLedgerStorage.java#L243), while skipping the internal `memTable`. `InterleavedLedgerStorage` additionally [delegates `getLastAddConfirmed` to the `LedgerCache`](https://github.com/apache/bookkeeper/blob/773a988ddf8b888a8869c6ae33f696935eba5cdc/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java#L361), which [delegates `lastAddConfirmed` to the `IndexPersistenceMgr`](https://github .com/apache/bookkeeper/blob/773a988ddf8b888a8869c6ae33f696935eba5cdc/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java#L80). Until the `memTable` is flushed, the LAC is [temporarily stored in a `FileInfo`](https://github.com/apache/bookkeeper/blob/773a988ddf8b888a8869c6ae33f696935eba5cdc/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java#L115) inside one of `writeFileInfoCache` or `readFileInfoCache` [in the `IndexPersistenceMgr`](https://github.com/apache/bookkeeper/blob/773a988ddf8b888a8869c6ae33f696935eba5cdc/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/IndexPersistenceMgr.java#L357). If the `FileInfo` is rotated out of one of the caches, it normally gets loaded back in. However, in the case of ledgers not yet flushed into `InterleavedLedgerStorage`, the file does not yet exist, so the `lastAddConfirmed` gets lost until either the `memTable` is flushed or the Bookie is restarted. This results in a `Bookie.NoLedgerException` while trying to read the LAC, and thus the spurious `ENOLEDGER`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
