[
https://issues.apache.org/jira/browse/ORC-2213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gang Wu updated ORC-2213:
-------------------------
Description: In old days, ORC only supports *orc::TIMESTAMP* type. The
semantics of {*}orc::{*}{*}TIMESTAMP{*} is equivalent to *TIMESTAMP_NTZ* but
with a complex writer and reader timezone adjustment. So it is difficult to
support both *TIMESTAMP_NTZ* and *TIMESTAMP_LTZ* types using a single
{*}orc::{*}{*}TIMESTAMP{*}. In later versions,
{*}orc::{*}{*}TIMESTAMP_INSTANT{*} has been added to support *TIMESTAMP_LTZ*
type. However, users may not know the difference under the hood and still use
legacy systems to write both *TIMESTAMP_LTZ* and *TIMESTAMP_NTZ* semantics to
the old {*}orc::{*}{*}TIMESTAMP{*} type. When users use *orc::TIMESTAMP* as
*TIMESTAMP_LTZ* values, we should set reader timezone to the writer timezone to
avoid value conversion. However, users may not know the writer timezone in
advance and ORC files may have different writer timezone (considering files are
produced by teams in different time zones.) We need an approach to enforce
reader to use writer timezone via explicit configuration. (was: In old days,
ORC only supports *orc::TIMESTAMP* type. The semantics of
{*}orc::{*}{*}TIMESTAMP{*} is equivalent to *TIMESTAMP_NTZ* but with a complex
writer and reader timezone adjustment. So it is difficult to support both
*TIMESTAMP_NTZ* and *TIMESTAMP_LTZ* types using a single
{*}orc::{*}{*}TIMESTAMP{*}. In later versions,
{*}orc::{*}{*}TIMESTAMP_INSTANT{*} has been added to support *TIMESTAMP_LTZ*
type. However, users may not know the difference under the hood and still use
legacy systems to write both *TIMESTAMP_LTZ* and *TIMESTAMP_NTZ* semantics to
the old {*}orc::{*}{*}TIMESTAMP{*} type. When users use *orc::TIMESTAMP* as
*TIMESTAMP_LTZ* values, we should set reader timezone to the writer timezone to
avoid slight value conversion. However, users may not know the writer timezone
in advance and ORC files may have different writer timezone (considering files
are produced by teams in different time zones.) We need an approach to enforce
reader to use writer timezone via explicit configuration.)
> [C++] Allow reading TIMESTAMP type using writer timezone
> --------------------------------------------------------
>
> Key: ORC-2213
> URL: https://issues.apache.org/jira/browse/ORC-2213
> Project: ORC
> Issue Type: Improvement
> Components: C++
> Reporter: Gang Wu
> Assignee: Gang Wu
> Priority: Major
>
> In old days, ORC only supports *orc::TIMESTAMP* type. The semantics of
> {*}orc::{*}{*}TIMESTAMP{*} is equivalent to *TIMESTAMP_NTZ* but with a
> complex writer and reader timezone adjustment. So it is difficult to support
> both *TIMESTAMP_NTZ* and *TIMESTAMP_LTZ* types using a single
> {*}orc::{*}{*}TIMESTAMP{*}. In later versions,
> {*}orc::{*}{*}TIMESTAMP_INSTANT{*} has been added to support *TIMESTAMP_LTZ*
> type. However, users may not know the difference under the hood and still use
> legacy systems to write both *TIMESTAMP_LTZ* and *TIMESTAMP_NTZ* semantics to
> the old {*}orc::{*}{*}TIMESTAMP{*} type. When users use *orc::TIMESTAMP* as
> *TIMESTAMP_LTZ* values, we should set reader timezone to the writer timezone
> to avoid value conversion. However, users may not know the writer timezone in
> advance and ORC files may have different writer timezone (considering files
> are produced by teams in different time zones.) We need an approach to
> enforce reader to use writer timezone via explicit configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
