mattsu2020 opened a new pull request, #65: URL: https://github.com/apache/datafusion-spark/pull/65
## Which issue does this PR close? Closes #33. Ports apache/datafusion#25955 to this repository following the datafusion-spark split, as requested in https://github.com/apache/datafusion/pull/25955#issuecomment-6803127841. The implementation and regression tests are unchanged apart from their repository paths. ## Rationale for this change With ANSI mode disabled, Spark `negative()` on decimal columns can fail with a schema type mismatch or return incorrectly scaled values when cast to `DOUBLE`. ## What changes are included in this PR? Preserve the input decimal data type after both checked and wrapping array negation, retaining the original precision and scale. ## What is the testing strategy for this PR? Added SQL regression tests in `crates/sqllogictest/test_files/math/negative.slt` using table columns to exercise the array path. They cover Decimal32, Decimal64, Decimal128, and Decimal256 with non-default precision, positive and negative values, zero, NULL, positive/zero/negative scales, and casts to `DOUBLE`, with ANSI mode both disabled and enabled. The regression test fails with the pre-fix implementation (a Decimal32 precision mismatch) and passes with the fix restored. Validation in this repository: - `cargo fmt --all -- --check --config edition=2024,max_width=90` - `cargo clippy --all-targets --all-features -- -D warnings`: no warnings - `cargo test -p datafusion-spark`: 324 unit tests and 3 doc tests passed; 1 doc test ignored - `cargo test -p datafusion-spark-sqllogictest`: all 245 test files passed - `RUST_BACKTRACE=1 cargo test --workspace --all-features --lib --tests --bins`: 326 unit tests and all 245 SQL test files passed Formatting uses the same settings as DataFusion. The repository currently lacks `rustfmt.toml`, so the default formatting CI check has an existing failure on `main`; #61 adds the missing configuration. No formatting configuration or unrelated source changes are included here. No existing local benchmark exercises Spark `negative()`. ## Are there any user-facing changes? Spark `negative()` now returns correctly scaled decimal column values and preserves their declared type when ANSI mode is disabled. No public API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
