zclllyybb commented on issue #65606:
URL: https://github.com/apache/doris/issues/65606#issuecomment-4970191146
Breakwater-GitHub-Analysis-Slot: slot_87297ff7f6de
This content is generated by AI for reference only.
Initial analysis: this looks like a real FE partition-pruning correctness
bug, not a data-routing failure.
What I checked on the reported commit `61c910f2cb`:
- The table metadata keeps both the partition expression and the underlying
partition column. For this DDL, the partition expression is
`date_trunc(TIME_STAMP, 'month')`, while the underlying partition column is
`TIME_STAMP`.
- The insert/load path is expected to route by the stored partition
expression. `BindSink.LegacyExprTranslator#createPartitionExprList()` builds
the sink partition expression list from
`olapTable.getPartitionInfo().getPartitionExprs()`, so `2001-01-07` is routed
by `date_trunc('2001-01-07', 'month') = '2001-01-01'`.
- The static Nereids pruning path does not model that expression domain.
`PruneOlapScanPartition#getPartitionSlots()` builds pruning slots from
`partitionInfo.getPartitionColumns()`, so the pruning slot is the raw
`TIME_STAMP` slot.
- `PartitionPruner` then extracts `TIME_STAMP > '2001-01-06'` as a
raw-column range, while `SortedPartitionRanges` / `OneRangePartitionEvaluator`
evaluate it against `RangePartitionItem` bounds that are in the
partition-expression domain.
That makes the issue's example unsafe. Partition `p_2000` is `[2000-01-01,
2001-01-05)` in the `date_trunc(TIME_STAMP, 'month')` domain. It is wrong to
treat that as the possible raw `TIME_STAMP` range. The inverse image of this
expression range still includes raw dates such as `2001-01-07`, because their
truncated month is `2001-01-01`. Therefore pruning `p_2000` for `TIME_STAMP >
'2001-01-06'` can produce an empty result even though the row exists.
There is already a similar safety principle in runtime-filter partition
pruning: `RuntimeFilterPartitionPruneClassifier` rejects automatic partition
expressions containing function calls with the reason that the automatic
partition expression boundary is not modeled. Static partition pruning appears
to lack an equivalent guard or expression-domain conversion for this case.
Recommended next steps:
- Reproduce with `EXPLAIN` / profile and confirm that `p_2000` is pruned for
`select * from date_table where TIME_STAMP > '2001-01-06'`.
- Add a regression test using the issue DDL and query. The test should
assert the query returns `2001-01-07`, and preferably also check that the scan
does not prune away `p_2000`.
- Fix the correctness boundary before adding more aggressive pruning. A
conservative fix is to skip raw-column static range pruning when
`PartitionInfo.enableAutomaticPartition()` is true and the range partition
expression contains a function call, unless the pruning code can prove a safe
mapping between the query predicate domain and the partition-expression domain.
- If the intended product behavior is to allow manual partitions under `AUTO
PARTITION BY RANGE (date_trunc(...))`, the DDL path should also validate that
manually specified range boundaries align with the function granularity. In
this example, `p_2001` with `[2001-01-06, 2001-01-08)` is effectively
unreachable for a monthly `date_trunc` expression. However, DDL validation
alone is not enough unless the pruning code also avoids comparing raw-column
predicates directly with expression-domain bounds.
Useful missing confirmation from the reporter would be the `EXPLAIN` output,
selected partition count/name from the plan or profile, and whether the result
becomes correct with static partition pruning disabled or by explicitly
scanning `PARTITION(p_2000)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
