J-HowHuang opened a new issue, #16057:
URL: https://github.com/apache/pinot/issues/16057
### Background
In the Tenant Rebalance API, the parameters `parallelWhitelist` and
`parallelBlacklist` have been used to categorize tables into the parallel or
sequential queue for rebalance operations. Specifically, tables listed in
`parallelWhitelist` would be rebalanced concurrently (according to the
specified degree of parallelism), whereas tables not in the whitelist (or
explicitly listed in `parallelBlacklist`) would be rebalanced sequentially
(degree of parallelism = 1).
### Motivation for Change
The introduction of `allowTables` and `blockTables` simplifies the
configuration: these new parameters are used solely to include or exclude
tables from the rebalance run. An important behavioral change is that all
tables included for rebalance are now processed with the same degree of
parallelism. There is no longer a mechanism to handle some tables in parallel
and others sequentially within the same rebalance operation.
### Key Differences
- **Old API (`parallelWhitelist` / `parallelBlacklist`):**
- Allowed users to specify certain tables to be rebalanced in parallel
(with a configurable degree), while others could be processed sequentially
(degree = 1).
- Provided fine-grained control over which tables ran in parallel vs.
sequentially.
- **New API (`allowTables` / `blockTables`):**
- Used to include or exclude tables from the rebalance operation.
- All included tables are rebalanced with the same degree of parallelism
as specified for the operation.
- No longer possible to mix parallel and sequential handling of tables in
a single run.
### Support for Old Usage
To ensure a smooth transition, the new API maintains backward compatibility
with the legacy parameters. When the `parallelWhitelist` or `parallelBlacklist`
parameters are provided (i.e., non-empty), the system will continue to use the
original logic—categorizing tables into parallel and sequential queues as
before. This means that existing clients or scripts relying on the old behavior
will continue to work seamlessly.
If both `parallelWhitelist` and `parallelBlacklist` are not set (i.e., both
are empty, which is now the default), the API will use the new logic based on
`allowTables` and `blockTables`, and all included tables will be rebalanced
with the same degree of parallelism. This dual support guarantees backward
compatibility while encouraging migration to the new, more streamlined
configuration.
Going forward, users are highly encouraged to migrate to `allowTables` and
`blockTables`, as future improvements and documentation will focus on the new
parameters and behavior.
### Deprecation Plan
- The use of `parallelWhitelist` and `parallelBlacklist` is being deprecated
in favor of `allowTables` and `blockTables`.
- Documentation and usage examples will be updated accordingly.
- Existing configurations should be reviewed and updated to migrate to the
new parameters, with attention to the changed parallelism behavior.
### References
See PR #15891 for implementation details and API examples:
https://github.com/apache/pinot/pull/15891
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
