lhotari opened a new pull request, #17273: URL: https://github.com/apache/pulsar/pull/17273
Fixes #16054 ### Motivation The broker cache eviction of entries read by active cursors has been broken in Pulsar since 2.8.2 version. This broke in the PR #12045 changes. It was a change to optimize the broker cache behavior where there was a high over head of the eviction task (issue reported as #9958 and also reported on Pulsar Slack by multiple users during that time). PR #12045 change was partially mitigated by a new feature "cacheEvictionByMarkDeletedPosition" added by PR #14985. However that introduces an issue that entries could get cached for too long and unnecessarily. The main motivation of this PR is to fix and restore broker cache evicition of entries based on the earliest read position of active cursor and also handle this in an efficient and performant way so that the original intention of PR #12045 will be covered, the performance issue reported as #9958. ### Modifications - add a new ManagedCursorContainer instance to ManagedLedgerImpl that keeps a sorted list of cursors ordered by the read position - make changes to ManagedLedgerImpl and ManagedCursorImpl to notify the ledger instance when the cursor's read position gets updated. This makes it possible to react to the changes and sort the cursors only when there has been changes to the data that is used for sorting. This reactive approach has been used for the mark delete position and this exists. It fixes the performance issue in the most efficient way. - There were some hard to understand tests added in #14985. Those tests have been replaced by simpler tests that will prevent future regressions. - Some tests contained invalid assertions perhaps caused by earlier bugs in the eviction by read position. This has been fixed. - More tests have been added to cover possible corner cases -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
