[
https://issues.apache.org/jira/browse/FLINK-17752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17134204#comment-17134204
]
Dawid Wysakowicz commented on FLINK-17752:
------------------------------------------
Hi,
First of all thank you [~jark] for the thorough investigation.
I was also thinking in a similar way as [~twalthr]. In my understanding it
should be the data that defines the DataType. We should not drop any
information in the {{CREATE TABLE}} statement. I do understand it might be not
the most flexible behaviour, but at this stage I really do believe this is
actually for the user's benefit that we require proper typing. Otherwise it is
really easy to get into a very hard to debug problems when working with
dates/times. If user wants to convert one type to another, she/he can use an
explicit cast, but at least we force the user to think of the semantics, which
in my opinion is in the users best interest.
Moreover I think we can benefit a lot if we try to mimic as much of the
{{java.time}} API as possible. Personally I find it rather well designed.
If we want to be really flexible in here, I'd rather have separate configurable
formats for all three types:
{{code}}
timestamp-format.without-time-zone
timestamp-format.with-local-time-zone
timestamp-format.with-time-zone
{{code}}
Additionally we can have a set of predefined configurations corresponding to
the different formats you mentioned: SQL, ISO-8601, RFC-3339.
SQL (e.g. timestamp-format.standard=SQL):
{code}
timestamp-format.without-time-zone=''yyyy-MM-dd HH:mm:ss.s{precision}"
timestamp-format.with-local-time-zone=''yyyy-MM-dd HH:mm:ss.s{precision}Z"
timestamp-format.with-time-zone=''yyyy-MM-dd HH:mm:ss.s{precision}+HH:MM:ss"
{code}
ISO-8601(e.g. timestamp-format.standard=ISO-8601):
{code}
timestamp-format.without-time-zone=''yyyy-MM-ddTHH:mm:ss.s{precision}"
timestamp-format.with-local-time-zone=''yyyy-MM-ddTHH:mm:ss.s{precision}Z"
timestamp-format.with-time-zone=''yyyy-MM-ddTHH:mm:ss.s{precision}+HH:MM:ss"
{code}
RFC-3339 (e.g. timestamp-format.standard=RFC-3339):
{code}
timestamp-format.without-time-zone=<not supported>
timestamp-format.with-local-time-zone=''yyyy-MM-ddTHH:mm:ss.s{precision}Z"
timestamp-format.with-time-zone=''yyyy-MM-ddTHH:mm:ss.s{precision}+HH:MM:ss"
{code}
This way users could define their custom formats, but again this would be
explicit and users would have to think about the consequences:
{code}
timestamp.format.standard=custom
timestamp-format.without-time-zone=<millis since epoch>
timestamp-format.with-local-time-zone=<millis since epoch>
timestamp-format.with-time-zone=''yyyy-MM-dd HH:mm:ss.s{precision}+HH:MM:ss"
{code}
This would be the same as creating a {{DateTimeFormatter}} and calling
{{LocalDateTime/Instant/OffsetDateTime.from(...)}} with that formatter. If the
user uses one of the standard configuration, she/he can be sure the data will
comply with the chosen standard, but still we would give a flexibility to read
also any other custom format. But we would force the user to make that choice
explicitly. We would not allow multiple different formats for a single type for
a single table at the same time. Moreover we would not apply any casting like
logic, but we would ensure that the data must comply with the expected format.
> Align the timestamp format with Flink SQL types in JSON format
> --------------------------------------------------------------
>
> Key: FLINK-17752
> URL: https://issues.apache.org/jira/browse/FLINK-17752
> Project: Flink
> Issue Type: Bug
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table
> SQL / Ecosystem
> Reporter: Jark Wu
> Assignee: Shengkai Fang
> Priority: Critical
> Fix For: 1.11.0
>
>
> Currently, we are using RFC3339_TIMESTAMP_FORMAT (which will add timezone at
> the end of string) to as the timestamp format in JSON. However, the string
> representation fo {{TIMESTAMP (WITHOUT TIME ZONE)}} shoudn't adding 'Z' at
> the end.
> Other discussions:
> [1]:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/TIME-TIMESTAMP-parse-in-Flink-TABLE-SQL-API-td33061.html
> [2]:
> http://apache-flink.147419.n8.nabble.com/json-timestamp-json-flink-sql-td1914.html
> [3]: http://apache-flink.147419.n8.nabble.com/FLINK-SQL-td2074.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
