Hi everyone, Thanks all for the valuable suggestions and feedbacks so far. Before starting the vote, I would like to summarize the proposed DDL syntax in the mailing list.
## Rowtime Attribute (Watermark Syntax) CREATE TABLE table_name ( WATERMARK FOR <columnName> AS <watermark_strategy_expression> ) WITH ( ... ) It marks an existing field <columnName> as the rowtime attribute, and the watermark is generated by the expression <watermark_strategy_expression>. <watermark_strategy_expression> can be arbitrary expression which returns a nullable BIGINT or TIMESTAMP as the watermark value. For common cases, users can use the following expressions to define a strategy. 1. Bounded Out of Orderness, the strategy can be "rowtimeField - INTERVAL 'string' timeUnit". 2. Preserve Watermark From Source, the strategy can be "SYSTEM_WATERMARK()". ## Proctime Attribute CREATE TABLE table_name ( ... proc AS SYSTEM_PROCTIME() ) WITH ( ... ) It uses the computed column syntax to add an additional column with proctime attribute. Here SYSTEM_PROCTIME() is a built-in function. For more details and the implementations, please refer to the design doc: https://docs.google.com/document/d/1-SecocBqzUh7zY6HBYcfMlG_0z-JAcuZkCvsmN3LrOw/edit?ts=5d822dba Feel free to leave your further feedbacks! Thanks, Jark On Thu, 19 Sep 2019 at 11:23, Kurt Young <ykt...@gmail.com> wrote: > +1 to start vote process. > > Best, > Kurt > > > On Thu, Sep 19, 2019 at 10:54 AM Jark Wu <imj...@gmail.com> wrote: > > > Hi everyone, > > > > Thanks all for joining the discussion in the doc[1]. > > It seems that the discussion is converged and there is a consensus on the > > current FLIP document. > > If there is no objection, I would like to convert it into cwiki FLIP page > > and start voting process. > > > > For more details, please refer to the design doc (it is slightly changed > > since the initial proposal). > > > > Thanks, > > Jark > > > > [1]: > > > > > https://docs.google.com/document/d/1-SecocBqzUh7zY6HBYcfMlG_0z-JAcuZkCvsmN3LrOw/edit?ts=5d8258cd > > > > On Mon, 16 Sep 2019 at 16:12, Kurt Young <ykt...@gmail.com> wrote: > > > > > After some review and discussion in the google document, I think it's > > time > > > to > > > convert this design to a cwiki flip page and start voting process. > > > > > > Best, > > > Kurt > > > > > > > > > On Mon, Sep 9, 2019 at 7:46 PM Jark Wu <imj...@gmail.com> wrote: > > > > > > > Hi all, > > > > > > > > Thanks all for so much feedbacks received in the doc so far. > > > > I saw a general agreement on using computed column to support > proctime > > > > attribute and extract timestamps. > > > > So we will prepare a computed column FLIP and share in the dev ML > soon. > > > > > > > > Feel free to leave more comments! > > > > > > > > Best, > > > > Jark > > > > > > > > > > > > > > > > On Fri, 6 Sep 2019 at 13:50, Dian Fu <dian0511...@gmail.com> wrote: > > > > > > > > > Hi Jark, > > > > > > > > > > Thanks for bringing up this discussion and the detailed design doc. > > > This > > > > > is definitely a critical feature for streaming SQL jobs. I have > left > > a > > > > few > > > > > comments in the design doc. > > > > > > > > > > Thanks, > > > > > Dian > > > > > > > > > > > 在 2019年9月6日，上午11:48，Forward Xu <forwardxu...@gmail.com> 写道： > > > > > > > > > > > > Thanks Jark for this topic, This will be very useful. > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > ForwardXu > > > > > > > > > > > > Danny Chan <yuzhao....@gmail.com> 于2019年9月6日周五 上午11:26写道： > > > > > > > > > > > >> Thanks Jark for bring up this topic, this is definitely an > import > > > > > feature > > > > > >> for the SQL, especially the DDL users. > > > > > >> > > > > > >> I would spend some time to review this design doc, really > thanks. > > > > > >> > > > > > >> Best, > > > > > >> Danny Chan > > > > > >> 在 2019年9月6日 +0800 AM11:19，Jark Wu <imj...@gmail.com>，写道： > > > > > >>> Hi everyone, > > > > > >>> > > > > > >>> I would like to start discussion about how to support time > > > attribute > > > > in > > > > > >> SQL > > > > > >>> DDL. > > > > > >>> In Flink 1.9, we already introduced a basic SQL DDL to create a > > > > table. > > > > > >>> However, it doesn't support to define time attributes. This > makes > > > > users > > > > > >>> can't > > > > > >>> apply window operations on the tables created by DDL which is a > > bad > > > > > >>> experience. > > > > > >>> > > > > > >>> In FLIP-66, we propose a syntax for watermark to define rowtime > > > > > attribute > > > > > >>> and propose to use computed column syntax to define proctime > > > > attribute. > > > > > >>> But computed column is another big topic and should deserve a > > > > separate > > > > > >>> FLIP. > > > > > >>> If we have a consensus on the computed column approach, we will > > > start > > > > > >>> computed column FLIP soon. > > > > > >>> > > > > > >>> FLIP-66: > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > https://docs.google.com/document/d/1-SecocBqzUh7zY6HBYcfMlG_0z-JAcuZkCvsmN3LrOw/edit# > > > > > >>> > > > > > >>> Thanks for any feedback! > > > > > >>> > > > > > >>> Best, > > > > > >>> Jark > > > > > >> > > > > > > > > > > > > > > > > > > > >
