shehabgamin opened a new pull request, #27207: URL: https://github.com/apache/datafusion/pull/27207
- Closes #27206. ## Rationale for this change A physical plan protobuf round trip can change the results of window queries. The window aggregate `FILTER` is dropped. `IGNORE NULLS` on window functions is dropped. Window functions that the optimizer reversed are decoded as not reversed, so `lag` returns the following row. The ordering of an aggregate used as a window function is also dropped. ## What changes are included in this PR? The patch adds `filter`, `is_reversed` and `ordering_req` to `PhysicalWindowExprNode` and regenerates the proto code. The window encoder now writes the real `ignore_nulls`, `is_reversed`, aggregate ordering and filter, and the decoder rebuilds the window expression with them. `serialize_physical_window_expr` / `parse_physical_window_expr` now delegate to the same encoder and decoder in `datafusion-physical-plan` instead of keeping a second copy. ## What is the testing strategy for this PR? Three new round-trip tests in `datafusion/proto/tests/cases/plans/windows.rs` each fail without this change: - `roundtrip_window_filter`: plain and sliding window aggregates with `FILTER` - `roundtrip_window_udf_ignore_nulls_and_reversed`: `lag` with `IGNORE NULLS`, and a reversed `lag` - `roundtrip_window_aggregate_ordering`: an ordered `array_agg` window aggregate and its reversed form ## Are there any user-facing changes? Window expressions now survive physical plan serialization. API changes: - New public items: `AggregateWindowExpr`, `WindowUDFExpr::ignore_nulls()` / `is_reversed()`, and `encode_physical_window_expr` / `decode_physical_window_expr`. - Because the message is now recursive, prost boxes the generated `physical_expr_node::ExprType::WindowExpr` variant, so it holds `Box<PhysicalWindowExprNode>`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
