andygrove opened a new issue, #3103:
URL: https://github.com/apache/datafusion-comet/issues/3103
## What is the problem the feature request solves?
> **Note:** This issue was generated with AI assistance. The specification
details have been extracted from Spark documentation and may need verification.
Comet does not currently support the Spark `try_make_interval` function,
causing queries using this function to fall back to Spark's JVM execution
instead of running natively on DataFusion.
The `TryMakeInterval` expression creates an interval from separate
components (years, months, weeks, days, hours, minutes, seconds). Unlike
`MakeInterval`, this expression returns `NULL` instead of throwing an exception
when an error occurs during interval creation. It is implemented as a
`RuntimeReplaceable` expression that delegates to `MakeInterval` with
`failOnError = false`.
Supporting this expression would allow more Spark workloads to benefit from
Comet's native acceleration.
## Describe the potential solution
### Spark Specification
**Syntax:**
```sql
try_make_interval([years [, months [, weeks [, days [, hours [, mins [,
secs]]]]]]])
```
**Arguments:**
| Argument | Type | Description |
|----------|------|-------------|
| years | Expression | Number of years (optional, defaults to 0) |
| months | Expression | Number of months (optional, defaults to 0) |
| weeks | Expression | Number of weeks (optional, defaults to 0) |
| days | Expression | Number of days (optional, defaults to 0) |
| hours | Expression | Number of hours (optional, defaults to 0) |
| mins | Expression | Number of minutes (optional, defaults to 0) |
| secs | Expression | Number of seconds as Decimal (optional, defaults to 0)
|
**Return Type:** Returns `INTERVAL` data type or `NULL` if an error occurs
during interval construction.
**Supported Data Types:**
- **Numeric types**: All numeric expressions are supported for input
parameters
- **Decimal**: The seconds parameter specifically uses Decimal type with
`MAX_LONG_DIGITS` precision and scale of 6
- **Integer literals**: Default values use integer literals (0) for most
parameters
**Edge Cases:**
- **Null handling**: Returns `NULL` if any input parameter is `NULL`
- **Invalid intervals**: Returns `NULL` instead of throwing exceptions for
invalid interval combinations
- **Overflow conditions**: Returns `NULL` when interval components exceed
valid ranges
- **Missing parameters**: Automatically fills missing parameters with
appropriate zero values (Literal(0) for most, Decimal(0) for seconds)
- **Empty constructor**: Intentionally not supported in try version as it
would never overflow
**Examples:**
```sql
-- Create interval with all components
SELECT try_make_interval(1, 2, 3, 4, 5, 6, 7.5);
-- Create interval with partial components
SELECT try_make_interval(2, 3);
-- Handle invalid input gracefully (returns NULL)
SELECT try_make_interval(9999999, 9999999, 9999999);
-- Compare with make_interval behavior
SELECT
try_make_interval(1, NULL, 3) as safe_result, -- Returns NULL
make_interval(1, NULL, 3) as unsafe_result; -- Throws exception
```
```scala
// DataFrame API usage
import org.apache.spark.sql.functions._
// Create interval column
df.select(try_make_interval(col("years"), col("months"), col("days")))
// With literal values
df.select(try_make_interval(lit(1), lit(6), lit(0), lit(15)))
// Handle potentially invalid data
df.select(try_make_interval(col("user_years"), col("user_months")))
.filter(col("interval_col").isNotNull)
```
### Implementation Approach
See the [Comet guide on adding new
expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html)
for detailed instructions.
1. **Scala Serde**: Add expression handler in
`spark/src/main/scala/org/apache/comet/serde/`
2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if
needed
4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has
built-in support first)
## Additional context
**Difficulty:** Medium
**Spark Expression Class:**
`org.apache.spark.sql.catalyst.expressions.TryMakeInterval`
**Related:**
- `MakeInterval` - The non-safe version that throws exceptions on errors
- `INTERVAL` literal syntax - Direct interval creation
- Date/time arithmetic functions for interval operations
---
*This issue was auto-generated from Spark reference documentation.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
