lhotari commented on code in PR #25873: URL: https://github.com/apache/pulsar/pull/25873#discussion_r3420628129
########## pip/pip-480.md: ########## @@ -0,0 +1,153 @@ +# PIP-480: Add cursorless readEntries API to ManagedLedger + +# Background knowledge + +Most `ManagedLedger` reads go through `ManagedCursor`. +A cursor is durable state: read position, mark-delete position, and individual deleted entries. +That is the right model for Pulsar subscriptions because the broker owns subscription progress and uses cursor state for +acknowledgement, backlog accounting, and ledger retention. + +Protocol handlers such as KoP do not always want that state. Kafka offsets are maintained outside Pulsar's +`ManagedCursor` abstraction. If KoP opens a cursor only to fetch entries, it creates extra managed-ledger metadata and +then has to keep the cursor lifecycle aligned with Kafka offsets. + +`ManagedLedger` already exposes `asyncReadEntry(Position, ...)` for reading a single entry by position, but it does not +provide a batch read primitive for this use case. Callers have to repeat the ledger traversal logic themselves: validate +the start position, cross ledger boundaries, skip empty ledgers, and stop at the current last confirmed entry. + +# Motivation + +Provide a cursorless read path for downstream projects like KoP. Review Comment: For Pulsar 5.0, it would be fine to break ManagedLedger in this aspect since there aren't many external implementors. If there's objections about breaking it, a default random read implementation could delegate to existing asyncReadEntry. For the Pulsar ManagedLedgerImpl, the implementation would be moved and there could be a delegation from asyncReadEntry to "random read"'s asyncReadEntry. This solution would be possible for backporting. I'd prefer skipping this for Pulsar 5.0 so that we could clean up the abstraction for this part. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
