kmozaid opened a new pull request #8224:
URL: https://github.com/apache/pinot/pull/8224
## Description
This PR adds a new implementation of `PartitionFunction` which is used to
partition segments. The new partition function named `BoundedColumnValue` can
be used to partition segments on column values and still generating partitionId
of integer type.
Example Usage -
```
"segmentPartitionConfig": {
"columnPartitionMap": {
"subject": {
"functionName": "BoundedColumnValue",
"functionConfig": {
"columnValues": "Maths|English|Chemistry"
}
}
}
```
PartitionId is generated based on position in columnValues. PartitionId
would 1 for Maths, 2 for English and so on.
PartitionId 0 is reserved for any other subject which are not present in
given `columnValues`. The different column values can be specified using pipe
(`|`) separation. This additional partition `functionConfig` is persisted in
metadata.properties and segment metadata in zookeeper. Broker can also use
this function to prune segments.
## Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order:
Controller, Broker, Server, Minion)
* [ ] No
Does this PR fix a zero-downtime upgrade introduced earlier?
* [ ] No
Does this PR otherwise need attention when creating release notes? Things to
consider:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed
* [ ] Yes, Adds new configuration option and changes to public interfaces.
## Release Notes
Added new partition function called `BoundedColumnValue` to be able to
partition segments based on column value.
<!-- If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text.
-->
## Documentation
Yes, will create another PR in pinot-docs repo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
