schenksj opened a new pull request, #4588:
URL: https://github.com/apache/datafusion-comet/pull/4588
Closes #4577.
Follow-up to #4531 (deep And/Or chains). Protobuf's recursion limit (100)
applies to
any deeply nested `BinaryExpr`, so a long left-deep chain of other
associative operators
overflows the same way when the serialized plan is re-parsed.
## What
Extend the rebalancing (`flattenAssociative` + a balanced O(log n)-depth
tree) to:
- **BitwiseAnd / BitwiseOr / BitwiseXor** — always integral and exactly
associative, so
they reuse the existing `createBalancedBinaryExpr` directly.
- **Add / Multiply** — gated via `isAssociativeAndRebalanceable` to integral
types in
LEGACY (wrapping, modular) eval mode, the only exactly-associative case.
Float isn't
associative (Spark's `ReorderAssociativeOperator` excludes it too);
ANSI/TRY make
integer-overflow position observable and grouping changes it; decimal
precision grows
per op. Those keep the existing left-deep serialization. `Add`/`Multiply`
emit a
`MathExpr` (eval_mode + return_type) rather than a `BinaryExpr`, so a new
`createBalancedMathExpr` builds the balanced tree with the chain's uniform
type and
eval mode at every inner node.
## Tests
Mirror #4531: project 200-deep chains and assert Comet runs them natively
with results
matching Spark (which also verifies the associativity guarantee).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
