matanper opened a new issue, #69041: URL: https://github.com/apache/doris/issues/69041
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.1.3 and 4.1.4 (cloud and local mode). master is the same: `be/src/format/arrow/arrow_stream_reader.cpp:52` sets the reader's zone with `TimezoneUtils::find_cctz_time_zone(TimezoneUtils::default_time_zone, _ctzz)`, and `:120` passes `_ctzz` to `read_column_from_arrow`. ### Description Stream Load with `format=arrow` ignores the load's `timezone` header when it converts Arrow timestamps into DATETIME. `ArrowStreamReader` resolves its time zone once, in its constructor, from `TimezoneUtils::default_time_zone` (Asia/Shanghai), and uses it for every column. Every timestamp is therefore stored as Asia/Shanghai wall-clock time (+8h for UTC data), whatever the header, the session time zone or the Arrow field's own `tz` says. The Parquet reader honours the header: it receives `RuntimeState::timezone_obj()` from the file scanner. ### How to Reproduce? 1. `CREATE TABLE t (k varchar(32), ts datetime(6)) DUPLICATE KEY(k) DISTRIBUTED BY HASH(k) BUCKETS 1 PROPERTIES ('replication_num'='1');` on a cluster whose global `time_zone` is UTC. 2. Stream Load one row with `ts` = 2026-10-13 16:37:00 UTC, headers `format: arrow` and `timezone: UTC`, body an Arrow IPC stream with schema `k: utf8, ts: timestamp[us, tz=UTC]` (a naive `timestamp[us]` behaves the same). 3. `SELECT CAST(ts AS STRING) FROM t` returns `2026-10-14 00:37:00`. The same row loaded as Parquet (`format: parquet`, `timezone: UTC`) is stored as `2026-10-13 16:37:00`. Sending the load through the FE (307 redirect) or directly to a BE gives the same result, and so does the `time_zone` header spelling. ### Solution In the `ArrowStreamReader` constructor, use the load's time zone when a `RuntimeState` is present, matching the Parquet path: ```cpp if (state != nullptr) { _ctzz = state->timezone_obj(); } else { TimezoneUtils::find_cctz_time_zone(TimezoneUtils::default_time_zone, _ctzz); } ``` With this change a `timestamp[us, tz=UTC]` value of 2026-10-13 16:37:00 reads back as `2026-10-13 16:37:00` under load time zone UTC, and as `2026-10-14 00:37:00` under Asia/Shanghai. We have a patch with a BE unit test (`ArrowStreamReaderStreamingTest`) and can open a PR. ### Anything Else? Existing users of `format=arrow` whose session time zone is not Asia/Shanghai have been storing shifted DATETIME values, so the fix changes stored values for them going forward. Loads that relied on the shift would need the header set to Asia/Shanghai to keep the old behaviour. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
