jackylee-ch opened a new pull request, #1418:
URL: https://github.com/apache/paimon-rust/pull/1418

   ### Purpose
   
   Linked issue: N/A (bug fix)
   
   A table that stores `incremental-between-timestamp`, or has it set for the 
session, reads the changes of that window. The partition `COUNT(*)` pushdown 
still answers from the latest snapshot's manifests, so `SELECT dt, COUNT(*) ... 
GROUP BY dt` returns the whole table's counts while `SELECT *` returns only the 
window's rows.
   
   ### Brief change log
   
   - `partition_count_pushdown::rewrite_aggregate` no longer rewrites an 
aggregate over a table that sets an incremental window (or sets an invalid one, 
so the error still surfaces on read).
   - `CoreOptions::incremental_timestamp_window` becomes `pub` so engine 
integrations can make this check.
   
   ### Tests
   
   - Added 
`test_count_of_an_incremental_window_counts_the_rows_the_window_reads` in 
`partition_count_pushdown.rs`: a partitioned table with two commits; with the 
window set by `SET` and by `ALTER TABLE ... SET TBLPROPERTIES`, the grouped 
count scans and matches the window's rows, and the pushdown is back after 
`RESET`. It fails on `main` and passes with this change.
   - `cargo test -p paimon-datafusion` passes; `cargo fmt --check` is clean.
   
   ### API and Format
   
   `CoreOptions::incremental_timestamp_window` is now public. No storage format 
change.
   
   ### Documentation
   
   No documentation change.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to