abhinav-phi opened a new pull request, #70724:
URL: https://github.com/apache/doris/pull/70724
### Background
BE datetime parsing resolves time zone names through `TimezoneUtils`, which
scans the system tzdata (`/usr/share/zoneinfo`, overridable via `TZDIR`) into a
name cache and keeps a separate cache of fixed-offset zones. ISO 8601 / RFC
3339 timestamps end in `Z` as the UTC designator, and today that designator
only resolves when the system tzdata scan finds a legacy top-level `Zulu` file.
### What problem does this PR solve?
Issue Number: close #69326
Related PR: None
PR Type: Bugfix
Problem Summary: On hosts whose tzdata ships no top-level `Zulu` file (for
example Ubuntu 24.04 ships only `Etc/Zulu`), `find_cctz_time_zone("Z")` and
`parse_tz_offset_string("Z")` both fail. A valid `2015-01-01T03:47:41Z` input
then converts to NULL, and a VARIANT path predefined as DATETIME filters the
NULL out, so the field silently disappears from the stored VARIANT value while
the load still reports success. The fix treats `z`/`zulu` as UTC aliases in
`normalize_timezone_name` and seeds the name cache with UTC for both names
unconditionally in `load_timezones_to_cache`, so the designator no longer
depends on any system tzfile. Most of the diff is the regression test.
### Details
- **Frequency:** every load or query that feeds a `Z`-suffixed timestamp
into BE datetime parsing on hosts without the legacy `Zulu` tzfile; the VARIANT
typed-path drop is the visible data-loss symptom.
- **Root cause:** `load_timezones_to_cache()` installed the `z` cache alias
only conditionally (`if (find("zulu")) emplace("z")`), and
`normalize_timezone_name()` did not accept `z` at all, so with no system `Zulu`
file both the cache path and the offset-parse path failed.
- **Why existing tests missed it:** the test environment's tzdata contains a
`Zulu` entry, so the conditional alias was always installed there; nothing
exercised the "no Zulu file" configuration.
- **Fix:** `normalize_timezone_name` now maps `z` to UTC alongside
`utc`/`etc/utc`/`zulu`, and `load_timezones_to_cache` unconditionally emplaces
`z` and `zulu` as `cctz::utc_time_zone()` instead of copying from a
system-provided `zulu` entry. `find_cctz_time_zone("Z")` now works on an empty
named cache too, because the normalized name falls through to the
`parse_tz_offset_string` UTC shortcut.
- **Other paths and branches:** `try_get_fixed_offset_seconds` benefits
automatically, since a `Z`-resolved zone is `cctz::utc_time_zone()` whose name
is `UTC` and reports offset 0. No other call sites changed.
### Testing
New unit test `TimezoneUtilsTest.UtcZDesignatorResolvesToUtc` in
`be/test/util/timezone_utils_test.cpp`: it asserts
`parse_tz_offset_string("Z"/"z")` succeeds, then runs
`find_cctz_time_zone("Z"/"z"/"zulu")` both with an emptied cache and with the
preloaded cache, asserting the resolved zone is `UTC` in each case. On master
(before the fix) `parse_tz_offset_string("Z")` returns false and the uncached
`find_cctz_time_zone("Z")` fails, so the test fails before the fix and passes
after it. No local BE build is available in my environment (Windows host), so
the suite itself runs in CI.
### Notes for Reviewers
The conditional `zulu`-copy block in `load_timezones_to_cache` was replaced
rather than extended; `emplace` keeps any system-provided `zulu` entry intact
since UTC has the same value. CST special case untouched.
### Release note
Timestamps ending in `Z` now load and query correctly on hosts without the
legacy `Zulu` tzfile, and VARIANT DATETIME typed paths no longer silently drop
such fields.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]