[
https://issues.apache.org/jira/browse/FLINK-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180077#comment-16180077
]
Xingcan Cui commented on FLINK-7548:
------------------------------------
Thanks for the answers, [~fhueske]. Some extra comments.
About the first question, we must admit that a rowtime field is dualistic. On
one hand, it represents the rowtime and should be taken as the {{Long}} type.
On the other hand, it is a common field that gets its own type ({{Timestamp}}
or {{Long}} and maybe more in the future?). We don't want to perform the extra
type judgement when using this field as the rowtime field and also don't want
to lose the original data type when using it as a common field (e.g., be passed
to a UDF which formats a timestamp). Of course, if all the types are internally
represented as {{Long}}, we just give the fields different time indicators so
that we could recover the original data type after the processing.
The record number bounded out-of-order generation strategy is something like we
don't emit a watermark {{w}} until a specified number of records whose
timestamps are greater than {{w}} have reached. Just an idea that hits me :P
About my last question, I actually refer to the
{{TimestampsAndPeriodicWatermarksOperator}}. Here, the "periodic" refers to
proctime. Considering the time systems for the rowtime and the proctime may not
be synchronized (i.e., they get different speeds), could we consider providing
a "rowtime periodic" assigner?
> Support watermark generation for TableSource
> --------------------------------------------
>
> Key: FLINK-7548
> URL: https://issues.apache.org/jira/browse/FLINK-7548
> Project: Flink
> Issue Type: Bug
> Components: Table API & SQL
> Reporter: Jark Wu
>
> As discussed in FLINK-7446, currently the TableSource only support to define
> rowtime field, but not support to extract watermarks from the rowtime field.
> We can provide a new interface called {{DefinedWatermark}}, which has two
> methods {{getRowtimeAttribute}} (can only be an existing field) and
> {{getWatermarkGenerator}}. The {{DefinedRowtimeAttribute}} will be marked
> deprecated.
> How to support periodic and punctuated watermarks and support some built-in
> strategies needs further discussion.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
