poorbarcode opened a new pull request, #24696:
URL: https://github.com/apache/pulsar/pull/24696
### Motivation
**Background**
- The oldest behaviour of offload settings: Pulsar only supports setting
Broker-level offload policies and namespace-level thresholds. and handle
namespace-level thresholds by the following fields:
- `{@link Policies#offload_deletion_lag_ms}`
- `{@link Policies#offload_threshold}`
- `{@link Policies#offload_threshold_in_seconds}`
- After https://github.com/apache/pulsar/pull/6183, Pulsar supports
namespace-level policies, which were supported by `{@link
Policies#offload_policies}`. And the thresholds were moved to the following
fields:
- `{@link Policies#offload_policies} -> {@link
OffloadPoliciesImpl#getManagedLedgerOffloadDeletionLagInMillis}`
- ` {@link Policies#offload_policies} -> {@link
OffloadPoliciesImpl#getManagedLedgerOffloadThresholdInBytes}`
- `{@link Policies#offload_policies} -> {@link
OffloadPoliciesImpl#getManagedLedgerOffloadThresholdInSeconds}`
- https://github.com/apache/pulsar/pull/6422 made setting offload
thresholds(`{@link Policies#offload_deletion_lag_ms}`, `{@link
Policies#offload_threshold}`, `{@link Policies#offload_threshold_in_seconds}`)
affect namespace policies(`{@link Policies#offload_policies} `)
- But neither mechanisms do not work well together
**The issue we encountered**, Thanks @horizonzy for investigating the issue
```
> pulsar-admin topicPolicies get-offload-policies <topic>
null
> more conf/broker.conf | grep -i "managedLedgerOffload"
managedLedgerOffloadDeletionLagMs=63072000000
managedLedgerOffloadAutoTriggerSizeThresholdBytes=-1
managedLedgerOffloadThresholdInSeconds=-1
> pulsar-admin namespaces get-offload-threshold <namespace>
offloadThresholdInBytes: 0
offloadThresholdInSeconds: -1
> pulsar-admin topicPolicies get-offload-policies <topic> -ap
{
...
"managedLedgerOffloadThresholdInSeconds" : -1,
"managedLedgerOffloadThresholdInBytes" : -1,
}
```
- How to reproduce the issue
- create namespace-level offload policies
- set namespace-level offload policies thresholds
- remove namespace-level offload policies
- get topic-level offload policies with `-ap`
### Modifications
To make the offload policies compatible with the old policies, use the old
policies' threshold(`{@link Policies#offload_deletion_lag_ms}`, `{@link
Policies#offload_threshold}`, `{@link Policies#offload_threshold_in_seconds}`)
if the new policies(`{@link Policies#offload_policies} `) are not set. Once the
new fields are set, the old fields will be removed.
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update
later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
