spicysharkco commented on issue #26335: URL: https://github.com/apache/datafusion/issues/26335#issuecomment-6229974686
I started on this and ran into a planning question I'd like a maintainer view on before opening a PR. Spark's result type depends on the value of the scale argument, so `return_field_from_args` needs that value at planning time. `scalar_argument_for_field` (`datafusion/expr/src/expr_schema.rs`) only passes it through when the argument is a bare literal. A cast such as `2::int` or `CAST(2 AS INT)` arrives as `None`, and `scalar_arguments_exclude_expression_casts_and_preserve_invalid_values` pins that on purpose. Most decimal cases in `round.slt` write the scale as `0::int` / `2::int`, so they all take that path. What I tried locally: 1. Fall back to a wider type when the scale value is unknown. After constant folding the physical planner does see a literal, so the logical and physical types disagree. A join on the result, an INSERT followed by a SELECT, and a comparison all fail with schema or type mismatch errors. The core `round` in `datafusion-functions` already fails the same three queries today. 2. Return a planning error when the scale of a decimal input is not a literal. Clean, but stricter than Spark, which only requires the scale to be foldable (`CAST(2 AS INT)` works there), and about 20 existing `round.slt` rows would change from `2::int` to `2`. 3. Let `scalar_argument_for_field` fold a cast of a literal. That fixes Spark `round` and core `round` together, but it reverses the behaviour the test above pins and touches every function that reads scalar arguments. Which of these would you accept, or is there a better hook for an argument whose value is only known after simplification? I'd go with 3 if that change is welcome, otherwise 2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
