pingzh opened a new pull request, #7598:
URL: https://github.com/apache/datafusion-comet/pull/7598

   ## Which issue does this PR close?
   
   Part 1 of #5774; this PR does not close the issue. The three-PR plan is in 
[this issue 
comment](https://github.com/apache/datafusion-comet/issues/5774#issuecomment-6782151492).
   
   ## Rationale for this change
   
   A partial `MIN(x)` or `MAX(x)` can tighten a bound while consuming Parquet 
input and skip later row groups that cannot improve the result. DataFusion 
already updates that bound, but Comet needs to connect its producer to the 
native reader and scope the mutable state to one execution.
   
   ## What changes are included in this PR?
   
   Add the internal `MinMaxReaderFilterExec` with the existing Parquet reader 
attachment. It accepts one unfiltered, non-distinct DataFusion MIN/MAX over a 
direct signed-integer column, with no grouping, in Partial mode, and one native 
input partition. Standalone projections, filters, limits, non-Parquet inputs, 
and pushed scan predicates remain ineligible.
   
   Each execution constructs a fresh aggregate, accumulator-bound state, and 
predicate. Aggregate metric handles and attachment counters belong to the 
wrapper; reader pruning and byte metrics stay with the scan. Unsupported reader 
attachment retains ordinary aggregation. Runtime state is released on 
completion, error, or cancellation.
   
   This is the native building block only: no Spark configuration, 
serialization, or planner integration is included, and existing Spark execution 
is unchanged. Those changes, including handling identity casts in MIN/MAX 
arguments, belong to PR 2. The private module has a scoped dead-code allowance 
until that integration lands.
   
   ## How are these changes tested?
   
   Seven native tests cover:
   
   - Signed-integer MIN/MAX with empty/all-null input, duplicates, and negative 
values.
   - Filtering off/on for favorable and unfavorable layouts, with a remapped 
file column and assertions on row groups, scan rows, bytes, and metric 
ownership.
   - Fresh predicates and accumulator bounds across executions, resets, and 
child replacement; runtime-state release at completion and cancellation.
   - Unsupported aggregates/readers, reader-attachment fallback, and omitted 
Parquet null counts.
   
   `rustfmt --check` and `git diff --check` pass. Two independent source 
reviews completed. The tests have not been executed locally; execution is 
pending CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to