morrySnow opened a new pull request, #67886: URL: https://github.com/apache/doris/pull/67886
## Problem `UNION ALL` could publish output-column equality that does not hold for every row. Downstream rules may then remove a required window ordering key and change `RANK()` results. Two forms reproduce the problem: 1. Regular children project columns in a different order from their internal output. With child filters proving `a = b`, a union that projects `(c, a, b)` can incorrectly map that equality to `(c, a)` and remove `a` from `ORDER BY c, a`. 2. A regular child proves `a = b`, but constant rows such as `(1, 2)` and `(2, 1)` do not. The union can still claim the output columns are equal and remove `b` from `ORDER BY a, b`. ## Root cause Logical and physical unions independently mapped child equality sets through `child.getOutput()` positions. The authoritative union ordinal mapping is `regularChildrenOutputs`, which can have a different order. The derivation also considered only regular children and ignored every constant row carried by the union. ## Reproduction Create rows `(1, 1, 10)`, `(2, 2, 10)`, and `(1, 1, 20)`, duplicate a filtered `(c, a, b)` child with `UNION ALL`, and compute `RANK() OVER (ORDER BY c, a)`. The two rows with `(c, a) = (10, 2)` must have rank 3. Separately, union filtered table rows satisfying `a = b` with constant rows `(1, 2)` and `(2, 1)`, then compute `RANK() OVER (ORDER BY a, b)`. The four ordered pairs must receive ranks 1, 2, 3, and 4. ## Fix - Share one equal-set derivation between logical and physical unions. - Validate and use each regular child's explicit union-output mapping. - Intersect child equality classes by per-ordinal class signatures, avoiding pairwise quadratic candidate generation. - Refine candidates against every constant row after SQL comparison coercion and constant folding. - Accept only a folded `TRUE`; `NULL`, unsupported coercion, non-foldable expressions, and malformed mappings conservatively provide no equality proof. - Build each surviving equivalence class with linear star edges. ## Tests - `./run-fe-ut.sh --run org.apache.doris.nereids.properties.EqualSetTest` (13 tests passed) - `DISABLE_BUILD_UI=ON ./build.sh --fe` (passed; Checkstyle reported zero violations) - `./run-regression-test.sh --run -f regression-test/suites/nereids_rules_p0/union_equal_set/union_equal_set.groovy` (1 suite passed, 0 failed, 0 fatal) Unit coverage exercises both logical and planner-produced physical unions, reordered mappings, multiple constant rows, one violating row, constant folding, cross-type numeric equality, and `NULL`. Regression coverage checks both retained window order keys and exact results. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
