raghavyadav01 opened a new pull request, #19673: URL: https://github.com/apache/pinot/pull/19673
`PartitionFunction.getPartition` returns a primitive `int`, so a function facing a value outside its column's domain has no way to say so — it has to return *some* id. Returning a real one lets a pruner drop a segment that does hold matching rows, so the only safe answer available today is to pick an id and arrange for every segment to also claim it. That works, but it costs pruning on every query whose value genuinely belongs to that partition: at N buckets, roughly 1/N of the key space stops pruning entirely. It also leaves each segment carrying more than one partition id, which disqualifies it from partition-aware placement and from `TablePartitionInfo`. This adds `UNKNOWN_PARTITION = -1` to the contract and teaches the readers to treat it as *"this value says nothing about which segments can match"*: - `SinglePartitionColumnSegmentPruner` and `MultiPartitionColumnsSegmentPruner` keep the segment - `ColumnValueSegmentPruner` (server side) keeps the segment - `AbstractColumnStatisticsCollector` does not record it as a partition of the segment - `MutableSegmentImpl` does not count an unplaceable row as a partition mismatch — the stream already decided which partition the row belongs to ### Compatibility Nothing returns `-1` today. Every `PartitionIdNormalizer` (`POSITIVE_MODULO`, `ABS`, `MASK`, `PRE_MODULO_ABS`) yields a value in `[0, numPartitions)`, so existing partition functions, existing segment metadata, and existing pruning behaviour are all unaffected. The sentinel is opt-in for implementations that choose to return it. ### Testing Two tests in `SegmentPrunerTest` covering both broker pruners, backed by a test-only `UnknownPartitionFunction`. Both fail without the change (verified by reverting the two pruners: 2 failures) and pass with it. `PartitionFunctionTest`, `PartitionFunctionFactoryTest` and `PartitionIdNormalizerTest` stay green (28 tests). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
