andygrove opened a new issue, #3121:
URL: https://github.com/apache/datafusion-comet/issues/3121
## What is the problem the feature request solves?
> **Note:** This issue was generated with AI assistance. The specification
details have been extracted from Spark documentation and may need verification.
Comet does not currently support the Spark `time_add_interval` function,
causing queries using this function to fall back to Spark's JVM execution
instead of running natively on DataFusion.
TimeAddInterval is a Spark Catalyst expression that adds a day-time interval
to a time value. It extends BinaryExpression and RuntimeReplaceable, meaning it
is replaced with a StaticInvoke call to DateTimeUtils.timeAddInterval during
expression optimization. The expression handles precision calculation to ensure
the result maintains appropriate time precision based on both the input time
and interval types.
Supporting this expression would allow more Spark workloads to benefit from
Comet's native acceleration.
## Describe the potential solution
### Spark Specification
**Syntax:**
```sql
time_expression + interval_expression
```
**Arguments:**
| Argument | Type | Description |
|----------|------|-------------|
| time | Expression | The base time value to which the interval will be
added |
| interval | Expression | The day-time interval to add to the time |
**Return Type:** Returns a `TimeType` with precision calculated as the
maximum of the input time precision and the interval precision. The interval
precision is determined by the interval's end field - if less than SECOND, uses
MIN_PRECISION, otherwise uses MICROS_PRECISION.
**Supported Data Types:**
- **time**: `AnyTimeType` - accepts time values with any precision
- **interval**: `DayTimeIntervalType` - accepts day-time intervals with any
start and end fields
**Edge Cases:**
- **Null handling**: The expression is null intolerant (nullIntolerant =
true), meaning null inputs produce null outputs with proper null propagation
- **Type validation**: Throws SparkException.internalError if unexpected
input types are encountered during replacement
- **Precision handling**: Automatically adjusts precision to accommodate
both time and interval precision requirements
- **Overflow behavior**: Relies on underlying DateTimeUtils.timeAddInterval
implementation for overflow handling
**Examples:**
```sql
-- Add 2 hours to a time value
SELECT TIME '10:30:00' + INTERVAL '2' HOUR;
-- Add days and hours to a time (days component wraps around)
SELECT TIME '14:15:30' + INTERVAL '1 5' DAY TO HOUR;
```
```scala
// Example DataFrame API usage
import org.apache.spark.sql.functions._
df.select(col("time_col") + expr("INTERVAL '30' MINUTE"))
// Using interval literal
df.select(col("time_col") + lit(Duration.ofHours(3)))
```
### Implementation Approach
See the [Comet guide on adding new
expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html)
for detailed instructions.
1. **Scala Serde**: Add expression handler in
`spark/src/main/scala/org/apache/comet/serde/`
2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if
needed
4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has
built-in support first)
## Additional context
**Difficulty:** Medium
**Spark Expression Class:**
`org.apache.spark.sql.catalyst.expressions.TimeAddInterval`
**Related:**
- TimeSubInterval - for subtracting intervals from time values
- TimestampAddInterval - for adding intervals to timestamp values
- DateTimeUtils.timeAddInterval - the underlying implementation method
- DayTimeIntervalType - for day-time interval data type details
---
*This issue was auto-generated from Spark reference documentation.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
