andygrove opened a new issue, #3102:
URL: https://github.com/apache/datafusion-comet/issues/3102
## What is the problem the feature request solves?
> **Note:** This issue was generated with AI assistance. The specification
details have been extracted from Spark documentation and may need verification.
Comet does not currently support the Spark `multiply_ym_interval` function,
causing queries using this function to fall back to Spark's JVM execution
instead of running natively on DataFusion.
The `MultiplyYMInterval` expression multiplies a year-month interval by a
numeric value, returning a new year-month interval. This expression supports
various numeric data types and handles precision conversions appropriately for
each type.
Supporting this expression would allow more Spark workloads to benefit from
Comet's native acceleration.
## Describe the potential solution
### Spark Specification
**Syntax:**
```sql
year_month_interval * numeric_value
```
**Arguments:**
| Argument | Type | Description |
|----------|------|-------------|
| `interval` | YearMonthIntervalType | The year-month interval to be
multiplied |
| `num` | NumericType | The numeric multiplier (byte, short, int, long,
float, double, or decimal) |
**Return Type:** `YearMonthIntervalType()` - Returns a year-month interval
representing the multiplication result.
**Supported Data Types:**
- **Interval**: YearMonthIntervalType only
- **Numeric**: All numeric types including:
- Integer types: ByteType, ShortType, IntegerType, LongType
- Floating-point types: FloatType, DoubleType
- DecimalType (arbitrary precision)
**Edge Cases:**
- **Null handling**: Returns null if either operand is null (nullIntolerant
= true)
- **Overflow behavior**:
- Integer multiplication uses `Math.multiplyExact()` throwing
ArithmeticException on overflow
- Long results converted using `Math.toIntExact()` throwing exception if
out of int range
- Decimal results use `intValueExact()` throwing exception if fractional
part exists after rounding
- **Floating-point precision**: Uses HALF_UP rounding mode for consistent
behavior
- **Zero multiplication**: Results in zero-month interval (valid)
**Examples:**
```sql
-- Multiply interval by integer
SELECT INTERVAL '2-6' YEAR TO MONTH * 3; -- Results in '7-6' (7 years 6
months)
-- Multiply interval by decimal
SELECT INTERVAL '1-0' YEAR TO MONTH * 2.5; -- Results in '2-6' (2 years 6
months)
```
```scala
// DataFrame API usage
import org.apache.spark.sql.functions._
df.select(col("year_month_interval") * lit(2))
df.select(col("year_month_interval") * col("multiplier"))
```
### Implementation Approach
See the [Comet guide on adding new
expressions](https://datafusion.apache.org/comet/contributor-guide/adding_a_new_expression.html)
for detailed instructions.
1. **Scala Serde**: Add expression handler in
`spark/src/main/scala/org/apache/comet/serde/`
2. **Register**: Add to appropriate map in `QueryPlanSerde.scala`
3. **Protobuf**: Add message type in `native/proto/src/proto/expr.proto` if
needed
4. **Rust**: Implement in `native/spark-expr/src/` (check if DataFusion has
built-in support first)
## Additional context
**Difficulty:** Medium
**Spark Expression Class:**
`org.apache.spark.sql.catalyst.expressions.MultiplyYMInterval`
**Related:**
- `MultiplyDTInterval` - For day-time interval multiplication
- `DivideYMInterval` - For year-month interval division
- `AddYMInterval` - For year-month interval addition
- `SubtractYMInterval` - For year-month interval subtraction
---
*This issue was auto-generated from Spark reference documentation.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
