robinyqiu commented on a change in pull request #12493:
URL: https://github.com/apache/beam/pull/12493#discussion_r472526820
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamTableFunctionScanRel.java
##########
@@ -232,7 +233,11 @@ private Schema getKeySchema(Schema inputSchema,
List<Integer> keys) {
upstream
.apply(
"assignEventTimestamp",
- WithTimestamps.<Row>of(row ->
row.getDateTime(windowFieldIndex).toInstant())
+ WithTimestamps.<Row>of(
+ row ->
+ Instant.ofEpochMilli(
Review comment:
> I think there'll be surprising loss of precision if this field is used
for windowing (say, with TimestampCombiner.EARLIEST) and then the timestamp is
inspected (is even available from SQL, right?) and has been truncated.
I see. From Java user can definitely inspect the timestamp, so that is a
problem. From pure SQL (e.g. from Dataflow SQL UI) I think for TUMBLE and HOP
this will be fine, because the window start/end timestamp will not have a
microsecond component if we only allow windowing up to millisecond. For SESSION
I think there will be a problem, because the window starts/ends at a timestamp
of an element.
I will make sure we don't silently truncate precision before I merge the PR.
One option is to use a PipelineOptions for user to choose truncate or fail over
timestamp with microsecond component during windowing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
