andygrove opened a new issue, #265:
URL: https://github.com/apache/datafusion-spark/issues/265
Part of #174 and apache/datafusion-comet#7300.
### What the function does
`positive(expr)` (also the unary `+expr` operator) returns `expr` unchanged.
It accepts numeric and interval types (`TypeCollection.NumericAndInterval`,
with `ImplicitCastInputTypes`, so a STRING argument is cast to DOUBLE). The
result type equals the (cast) input type; NULL in gives NULL out. Available
since 1.5.0.
In Spark 4.0+ `UnaryPositive` is `RuntimeReplaceable` with `replacement =
child`, so it disappears from optimized plans. In 3.4 and 3.5 it is a regular
expression that survives into physical plans.
### Examples
```sql
SELECT positive(1);
-- 1
SELECT positive('-1.11'), positive(-1.11);
-- -1.11 -1.11 (double, decimal(3,2))
SELECT +interval '-1 year 1 month';
-- -0-11 interval year to month
SELECT +interval '-1 day 1 hour -1 minute 1 second';
-- -0 23:00:59.000000000
SELECT +date '1999-01-01';
-- error DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE
```
### Edge cases to test
- NULL input of each type.
- Every integer type keeps its type and value, including `-2147483648` and
`-9223372036854775808` (no overflow, unlike `negative`).
- FLOAT / DOUBLE: NaN, -0.0 (sign preserved), +/-Infinity returned unchanged.
- DECIMAL keeps precision and scale (`positive(-1.11)` is `DECIMAL(3,2)`).
- STRING argument is cast to DOUBLE (`positive('-1.11')` is DOUBLE `-1.11`).
With ANSI on an invalid string fails in the cast (`CAST_INVALID_INPUT`); with
ANSI off it is NULL.
- Year-month and day-time intervals are returned unchanged
(`Interval(YearMonth)` and `Duration(Microsecond)` in this crate).
- DATE, TIMESTAMP, BOOLEAN, BINARY are rejected with
`DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE` (requiredType `("NUMERIC" or
"INTERVAL DAY TO SECOND" or "INTERVAL YEAR TO MONTH" or "INTERVAL")`).
- Scalar and array inputs.
### Differences across Spark versions
- 3.4 / 3.5: `UnaryPositive` is evaluated as an expression (identity
`nullSafeEval`).
- 4.0+: `RuntimeReplaceable`, replaced by its child during optimization.
Behaviour is the same.
### Implementation notes
- Spark:
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala`,
`UnaryPositive`.
- DataFusion core has no `positive` function. Its SQL planner accepts
`+expr` for numeric, interval and timestamp operands and returns the operand
unchanged (`datafusion/sql/src/expr/unary_op.rs`); Spark rejects timestamps,
and Spark day-time intervals arrive here as `Duration`, which that check may
not cover.
- Mirror `crates/spark/src/function/math/negative.rs` (same signature and
types) and return the input as-is. Implementing `ScalarUDFImpl::simplify` to
replace the call with its argument is also appropriate.
- Tests: the placeholder `crates/sqllogictest/test_files/math/positive.slt`
exists with a commented-out query; enable and extend it. Document with
`#[user_doc]`.
---
_This issue was generated by an LLM (Claude) from the Apache Spark source
code and tests. Examples come from Spark's `ExpressionDescription` examples,
test suites and `.sql.out` golden files, but please verify them against Spark
before relying on them._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]