adriangb commented on issue #11039: URL: https://github.com/apache/arrow-rs/issues/11039#issuecomment-5623281692
Re-scoping this: it is a **blocker**, not a sibling follow-up to #11037. I validated https://github.com/apache/arrow-rs/pull/11038 against Apache DataFusion, expecting it to close https://github.com/apache/datafusion/issues/25084. It does not, because of this issue. Verified on DataFusion 55.0.0: ```sql -- goes through the cast kernel: FIXED by #11038 SELECT '2024-03-10 02:30:00'::timestamp AT TIME ZONE 'America/New_York'; -- Cast error: Cannot cast timezone to different timezone -- goes through string_to_datetime: NOT fixed by #11038 SELECT '2024-03-10 02:30:00' AT TIME ZONE 'America/New_York'; -- Parser error: Error parsing timestamp from '2024-03-10 02:30:00': error computing timezone offset SET datafusion.execution.time_zone = 'America/New_York'; SELECT '2024-03-10 02:30:00'::timestamptz; -- Parser error: Error parsing timestamp from '2024-03-10 02:30:00': error computing timezone offset ``` Those are the same query to a user. The third form is the most natural spelling of it, and it is entirely on the parser path. The practical consequence of landing #11038 without this is a self-inconsistent engine: inserting an explicit `::timestamp` in the middle of an expression would make a query start working. That is worse to explain than the current uniform failure. Other `string_to_datetime` entry points reachable from ordinary SQL, all failing the same way today: ```sql SELECT to_timestamp('2024-03-10 02:30:00'); SELECT arrow_cast('2024-03-10 02:30:00', 'Timestamp(Nanosecond, Some("America/New_York"))'); SELECT TIMESTAMPTZ '2024-11-03 01:30:00 America/New_York'; -- the parsed-Tz-suffix site SET datafusion.execution.time_zone = 'America/Sao_Paulo'; SELECT '2018-11-04'::timestamptz; -- date-only site: local midnight does not exist that day ``` That last one exercises the date-only call site, where the gap falls on midnight — worth a test case, since it is easy to miss when only datetime inputs are considered. The fix shape should match #11038's so the two paths cannot drift: ambiguous resolves to the later instant, a gap resolves by probing 24 hours earlier. I checked that assumption exhaustively while validating #11038 — across all 597 chrono-tz zones there is no pair of transitions closer than 167 hours, so the 24-hour probe is safe. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
