claudevdm commented on code in PR #36960:
URL: https://github.com/apache/beam/pull/36960#discussion_r2582706978
##########
sdks/java/extensions/arrow/src/main/java/org/apache/beam/sdk/extensions/arrow/ArrowConversion.java:
##########
@@ -456,21 +460,32 @@ public Optional<Function<Object, Object>>
visit(ArrowType.Time type) {
@Override
public Optional<Function<Object, Object>> visit(ArrowType.Timestamp
type) {
+ // Arrow timestamp semantics:
+ // - With timezone: epoch is always UTC, timezone is display metadata
+ // - Without timezone: epoch is in an unknown timezone ("naive"
wall-clock time)
DateTimeZone tz;
try {
tz = DateTimeZone.forID(type.getTimezone());
} catch (Exception e) {
throw new IllegalArgumentException(
"Encountered unrecognized Timezone: " + type.getTimezone());
}
- switch (type.getUnit()) {
- case MICROSECOND:
- return Optional.of((epochMicros) -> new DateTime((long)
epochMicros / 1000, tz));
- case MILLISECOND:
- return Optional.of((epochMills) -> new DateTime((long) epochMills,
tz));
- default:
- throw new AssertionError("Encountered unrecognized TimeUnit: " +
type.getUnit());
+
+ if (type.getUnit() == TimeUnit.NANOSECOND) {
+ return Optional.of(epoch -> Instant.ofEpochSecond(0L, (long) epoch));
Review Comment:
Instant.ofEpochSecond handles reading this way (see tests), but I can break
it down into seconds/ nanoAdjustment for completeness
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
