[ 
https://issues.apache.org/jira/browse/FLINK-41591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramin Gharib reassigned FLINK-41591:
------------------------------------

    Assignee: Moritz Manner

> PARSE_XML keeps only microseconds of a dateTime in 1677-09-21 00:12:43
> ----------------------------------------------------------------------
>
>                 Key: FLINK-41591
>                 URL: https://issues.apache.org/jira/browse/FLINK-41591
>             Project: Flink
>          Issue Type: Bug
>          Components: Table SQL / Runtime
>            Reporter: Ramin Gharib
>            Assignee: Moritz Manner
>            Priority: Minor
>
> A {{dateTime}} with digits below a microsecond keeps its nanoseconds if it is 
> in the range of a nanosecond timestamp, from {{1677-09-21 
> 00:12:43.145224192}} to {{2262-04-11 23:47:16.854775807}} UTC. In the first 
> second of that range, PARSE_XML keeps only microseconds.
> {code:sql}
> SELECT x, CAST(PARSE_XML('<a 
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:type="dateTime">' 
> || x || '</a>')['a'] AS TIMESTAMP(9)) AS ts
> FROM (VALUES ('1677-09-21T00:12:43.145224192'), 
> ('1677-09-21T00:12:43.999999999'), ('1677-09-21T00:12:44.000000001')) AS t(x);
> {code}
> ||x||Expected||Actual||
> |1677-09-21T00:12:43.145224192|1677-09-21 00:12:43.145224192|1677-09-21 
> 00:12:43.145224000|
> |1677-09-21T00:12:43.999999999|1677-09-21 00:12:43.999999999|1677-09-21 
> 00:12:43.999999000|
> |1677-09-21T00:12:44.000000001|1677-09-21 00:12:44.000000001|1677-09-21 
> 00:12:44.000000001|
> The same in batch and streaming mode. The upper bound works: 
> {{2262-04-11T23:47:16.854775807}} keeps its nanoseconds.
> h3. Cause
> {{BinaryVariantUtil#nanosSinceEpoch}} returns 
> {{ChronoUnit.NANOS.between(Instant.EPOCH, instant)}}. For 00:12:43, the epoch 
> second is -9223372037, and {{Instant#until}} multiplies it by 10^9 before it 
> adds the nanoseconds of the second. The product overflows, although the sum 
> fits into a {{long}}. The method throws, and {{XmlToVariantParser}} falls 
> back to microseconds. From the code, {{BinaryVariantBuilder}} calls the same 
> method for an {{Instant}} or a {{LocalDateTime}} without catching it, so it 
> rejects the same second.
> h3. Proposal
> * Compute the count from the epoch second plus one and the nanoseconds minus 
> 10^9 when the epoch second is negative, so the intermediate product stays in 
> range.
> * Test the first and the last value of the range, and one nanosecond outside 
> each.
> * The documentation page "XML and VARIANT" now gives the range from 
> {{1677-09-21 00:12:44}}. Restore the full range there with the fix.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to