chunxiaozheng commented on code in PR #14398:
URL: https://github.com/apache/pinot/pull/14398#discussion_r1835281970
##########
pinot-common/src/main/java/org/apache/pinot/common/utils/DataSchema.java:
##########
@@ -490,12 +540,24 @@ public Object toInternal(Object value) {
return ((boolean) value) ? 1 : 0;
case TIMESTAMP:
return ((Timestamp) value).getTime();
+ case TIMESTAMP_NTZ:
+ return ((LocalDateTime)
value).atZone(ZoneId.of("UTC")).toInstant().toEpochMilli();
Review Comment:
> @gortiz I think that snippet isn't correct because code assumes offset is
constant (and actually 0 while local is likely not :) ) @chunxiaozheng Please
consider :
>
> ```java
> public static void main(String[] args) {
> LocalDateTime dt = LocalDateTime.of(2024, 11, 8, 12, 13, 14,
231000000);
>
> ZoneId zoneId = ZoneId.of("Europe/Warsaw");// zone should be cached
> ZoneRules rules = zoneId.getRules();
> ZoneOffset offset = rules.getOffset(dt);
>
> System.out.println("to epoch (UTC) " +
dt.toEpochSecond(ZoneOffset.UTC) * 1000);
> System.out.println("to epoch " + dt.toEpochSecond(offset) * 1000);
> System.out.println("to epoch plus nanos " + dt.toEpochSecond(offset) *
1000 + dt.getNano() / 1000000);
> }
> ```
Thanks! The implementation of TIMESTAMP type without time zone, such as
`TIMESTAMP` in Flink and `TIMESTAMP_NTZ` type in Spark, will parse according to
UTC time to ensure that the time queried in any time zone is consistent.
Therefore, for TIMESTAMP types without time zones, I believe it is need to use
a constant(UTC time zone) for parsing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
