andygrove opened a new pull request, #7651:
URL: https://github.com/apache/datafusion-comet/pull/7651

   ## Which issue does this PR close?
   
   Backport of #7215 to `branch-1.2`. Closes #7209 on this branch.
   
   ## Rationale for this change
   
   The native string-to-timestamp cast returns NULL (or `CAST_INVALID_INPUT` 
under ANSI) for several zone suffixes that Spark accepts through `ZoneId.of(id, 
ZoneId.SHORT_IDS)`: `SHORT_IDS` aliases such as `PST`, region ids without a `/` 
such as `CET` and `Zulu`, a region id with no space before it, and offsets with 
a seconds field. `branch-1.2` has the same parser. See #7215 and #7209 for the 
full analysis.
   
   ## What changes are included in this PR?
   
   Cherry-pick of 567dea8e31 (`git cherry-pick -x`). The Rust change in 
`native/spark-expr/src/conversion_funcs/string.rs`, its unit tests and the 
`cast_string_to_timestamp_zone_ids.sql` file test are identical to the source 
commit apart from line numbers and blob hashes.
   
   One adaptation: the source commit updates the "String to Timestamp" section 
of 
`docs/source/user-guide/latest/compatibility/expressions/_category_template/cast.md`.
 On a release branch the per-version pages (`spark-3.4`, `spark-3.5`, 
`spark-4.0`, `spark-4.1`, `spark-4.2`) hold a frozen copy of that section, so I 
applied the same replacement to each `cast.md`. The section was identical in 
all five before the change.
   
   Other release branches: `branch-1.2` is the newest release branch, so there 
is no newer one to port to.
   
   ## How are these changes tested?
   
   On this branch, with JDK 17 and the default Spark profile only: the 49 
`conversion_funcs::string` tests in `datafusion-comet-spark-expr` pass, `cargo 
check --all-targets` and `cargo fmt --check` are clean, prettier is clean on 
the five `cast.md` files, and `CometSqlFileTestSuite` for 
`cast_string_to_timestamp_zone_ids.sql` passes in both session time zone 
configurations against a native library built from this branch. With 
`string.rs` restored to `branch-1.2`'s version, both runs fail with Spark/Comet 
answer mismatches, so the branch has the bug and the test catches it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to