0lai0 commented on code in PR #5193:
URL: https://github.com/apache/datafusion-comet/pull/5193#discussion_r3699467811
##########
native/spark-expr/src/math_funcs/internal/make_decimal.rs:
##########
@@ -45,14 +47,41 @@ pub fn spark_make_decimal(
ColumnarValue::Array(a) => match a.data_type() {
DataType::Int64 => {
let arr = a.as_primitive::<Int64Type>();
- let mut result = Decimal128Builder::new();
- for v in arr.into_iter() {
- result.append_option(long_to_decimal(v, precision, scale,
fail_on_error)?)
- }
let result_type = DataType::Decimal128(precision, scale);
+ // The Int64 is already the unscaled Decimal128 value, so we
only reinterpret
+ // the bits (an Arrow Int64->Decimal cast would rescale the
value). Both arity
+ // helpers reuse the input null buffer and only invoke the
closure on valid rows.
+ let result: Decimal128Array = if fail_on_error {
+ // ANSI mode: overflow is a hard error. `try_unary`
surfaces the closure's
+ // ArrowError; unwrap the ExternalError back to
DataFusionError::External so
+ // the ANSI error variant (not a generic ArrowError) is
preserved.
+ try_unary::<Int64Type, _, Decimal128Type>(arr, |v| {
+ let v = v as i128;
+ validate_decimal_precision(v, precision, scale)
+ .map(|()| v)
+ .map_err(|_| {
+
ArrowError::ExternalError(Box::new(decimal_overflow_error(
+ v, precision, scale,
+ )))
+ })
+ })
+ .map_err(|e| match e {
+ ArrowError::ExternalError(inner) =>
DataFusionError::External(inner),
+ other => DataFusionError::from(other),
+ })?
+ } else {
+ // Non-ANSI: overflow becomes null. `unary_opt` applies
the closure only to
+ // valid rows and marks a row null wherever the closure
returns None.
+ arr.unary_opt::<_, Decimal128Type>(|v| {
Review Comment:
The scan rewrite removed the duplication: the array path is now just one
`unary` widen + one `find` predicate, so it no longer inlines the validate
logic at all.
That leaves a single call site (the scalar `long_to_decimal`), where
extracting a shared helper would only add indirection. So I'd prefer to leave
it as is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
