lhotari opened a new pull request, #23231:
URL: https://github.com/apache/pulsar/pull/23231
Fixes #23200
### Motivation
This PR introduces enhancements to the Key_Shared subscription dispatching
logic in the Pulsar broker. The changes aim to optimize message dispatching,
improve the handling of slow consumers, and address several identified issues
related to message replay and message "look ahead" when consumers are blocked
in delivery due to key ordering restrictions.
This PR adds limits to the Key_Shared Subscription look-ahead feature, which
was introduced in PR #7105. Additionally, there are several improvements to the
previous logic.
Since there wasn't any named concept for the feature added in PR #7105, it
has been named "key shared look ahead" in this PR. This references the feature
added in PR #7105, where the dispatcher will skip replaying messages and read
more messages from the backlog. The changes in PR #7105 didn't add a limit for
reading more messages. The comments in the PR mention that it will be limited
by the unacked message limits. However, this isn't the case since the read
messages aren't considered unacked messages until they have been dispatched to
consumers.
### Modifications
1. **Configuration Properties:**
- Added new configuration properties to control the limits of Key_Shared
subscriptions look ahead in dispatching:
- `keySharedLookAheadNumberOfReplayMessagesThresholdPerConsumer`
- `keySharedLookAheadNumberOfReplayMessagesThresholdPerSubscription`
- `keySharedLookAheadEnabledWhenRecentlyJoinedConsumersExist`
2. **Broker Configuration:**
- Updated `broker.conf` to include descriptions and default values for
the new configuration properties.
3. **Key_Shared Subscription Logic:**
- Implement the key shared look-ahead limits that are configured with
the above configuration properties.
- Enhance the dispatching to ensure efficient dispatching:
- Instead of filtering the list of entries pulled from the message
redelivery controller, refactor the solution to take a function that can filter
the entries until there's a sufficient amount of messages for dispatching.
- This refactoring eliminates the previous method
`filterOutEntriesWillBeDiscarded` while preserving the previous logic of
filtering.
- Remove the unnecessary thread-local fields `localGroupedEntries` and
the unused `localGroupedPositions`.
- It is unnecessary to eliminate the creation of short-lived objects
in modern JVMs. It just clutters the code without any performance benefit. "No
garbage" style is relevant mainly for long-lived objects stored in caches or
large array allocations.
- Replace `getMessagesToReplayNow(1)` with `getFirstPositionInReplay()`.
- This check will no longer have the side effect of pulling messages
from delayed delivery when it's called.
- Add a new method `canReplayMessages` to make the look-ahead logic more
explicit. The previous solution of returning an empty set from the
`getMessagesToReplayNow` method made it hard to understand how the look-ahead
behavior is implemented.
- Rename the `isDispatcherStuckOnReplays` field to
`skipNextReplayToTriggerLookAhead` to make the meaning of the field explicit
and self-descriptive.
- Rename the `keyNumbers` variable to
`remainingConsumersToFinishSending` so that the meaning of the variable is
explicit and self-descriptive.
- Rename the `groupedEntries` variable to
`entriesByConsumerForDispatching` so that the meaning of the variable is
explicit and self-descriptive.
- Rename the `entriesWithSameKey` variable to `entriesForConsumer` so
that the variable is self-descriptive.
- Calculate permits for batch messages and take them into account for
initial filtering:
- In replay.
- In sending messages.
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [x] `doc-required` <!-- Your PR changes impact docs and you will update
later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
