gt;>>>>> Kui Yuan
> >>>>>>>>>>>
> >>>>>>>>>>> Timo Walther 于2023年2月27日周一 18:05写道：
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi Kui Yuan,
> >>>>
ific it ervery time.
[1]
https://nightlies.apache.org/flink/flink-docs-master/zh/docs/connectors/table/filesystem/#sink-parallelism
Best regards,
Yuxia
Best regards,
Yuxia
- 原始邮件 -----
发件人: "kui yuan"
收件人: "dev"
抄送: "Jark Wu"
发送时间: 星期三, 2023年
t; > >>>>>
> > > > > > >>>>> I know that this makes the keys even longer, but given that
> > > those
> > > > > > >>>>> options are for power users this should be acceptable. It
> > also
> > > >
watermark-related features in SQL layer
> > with
> > > > > >>> dynamic
> > > > > >>>>>>> table options and 'OPTIONS' hint, just as everyone
> expects. I
> > > > will
> > > > > >>>>> modify
> > > &
t;
> > > > >>>>>
> > > > >>>
> > > >
> > >
> >
> https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/table/kafka/#sink-delivery-guarantee
> > > > >>>>> [3]
> > > > &g
gt; > >>>>>
> > > >>>>>
> > > >>>>> On 24.02.23 04:55, kui yuan wrote:
> > > >>>>>> Hi all,
> > > >>>>>>
> > > >>>>>> I have updated the fl
nnectors/kafka.html#sink-partitioner
> > > > >>>>> [2]
> > > > >>>>>
> > > > >>>>>
> > > > >>>
> > > >
> > >
> >
> https://nightlies.apache.org/flink
ub.com/apache/flink/blob/master/flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/plan/abilities/source/WatermarkPushDownSpec.java
> > > >>>>>
> > > >>>>>
> > > >>>>> On 24.02.23 04:55, kui yuan wrote:
> > >
t;>>>
> > >>>>>> ```
> > >>>>>> -- configure in table options
> > >>>>>> CREATE TABLE user_actions (
> > >>>>>> ...
> > >>>>>> user_action_time TIMESTAMP(3),
> > >>>>
gt;>>>> 'ON_PERIODIC') */
> >>>>>> ```
> >>>>>>
> >>>>>> Does everybody have any other questions?
> >>>>>>
> >>>>>> Best
> >>>>>> Kui Yuan
> >
ursday, February 23, 2023 17:14
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-296: Watermark options for table API &
SQL
Hi,
While I can understand that there's a desire to first focus on
solving
this
problem for SQL, I do wonder if we should ignore the Table API at
this
point. If we
c table options,
>> >> maybe
>> >> >> we are able to use them indirectly in the table API like this[1]:
>> >> >>
>> >> >> ```
>> >> >> // register a table named "Orders"
>> >> >>
> >> >>
> >> >> [1]
> >> >>
> >>
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/create/
> >> >>
> >> >> Best
> >> >> Kui Yuan
> >> >>
&g
I think I
>> could
>> >>> give some explanations on the original intention to introduce another
>> >>> WATERMARK_PARAMS. If we take a look at the current datastream API, the
>> >>> watermark strategy does not belong to any specific connector. A
;>>>
> >>>>
> >>>> On Thu, Feb 23, 2023 at 10:52 AM Jingsong Li
> >>>> wrote:
> >>>>
> >>>>> Thanks for your proposal.
> >>>>>
> >>>>> +1 to yuxia, consider wa
From: Martijn Visser
Sent: Thursday, February 23, 2023 17:14
To: dev@flink.apache.org
Subject: Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL
Hi,
While I can understand that there's a desire to first focus on solving
this
problem for SQL, I do wonder if we should ignore the
upport the dynamic table options in table API, which is deserved to create
>> another FLIP.
>>
>> Best
>> Yun Tang
>>
>> From: Martijn Visser
>> Sent: Thursday, February 23, 2023 17:14
>> To: dev@flink.apache.org
>
gt; > hints. So, user can define a default watermark strategy for the table,
> > and
> > > if user dosen't needed to changes it, they need to do nothing in their
> > > query instead of specific it ervery time.
> > > >
> > > > [1]
> > >
> >
to do nothing in their
> > query instead of specific it ervery time.
> > >
> > > [1]
> >
> https://nightlies.apache.org/flink/flink-docs-master/zh/docs/connectors/table/filesystem/#sink-parallelism
> > >
> > > Best regards,
> > > Yuxia
&g
o nothing in their
> > query instead of specific it ervery time.
> > >
> > > [1]
> >
> https://nightlies.apache.org/flink/flink-docs-master/zh/docs/connectors/table/filesystem/#sink-parallelism
> > >
> > > Best regards,
> > > Yuxia
> > >
&g
che.org/flink/flink-docs-master/zh/docs/connectors/table/filesystem/#sink-parallelism
> >
> > Best regards,
> > Yuxia
> >
> > Best regards,
> > Yuxia
> >
> > ----- 原始邮件 -----
> > 发件人: "kui yuan"
> > 收件人: "dev"
> >
fic it ervery time.
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-master/zh/docs/connectors/table/filesystem/#sink-parallelism
>
> Best regards,
> Yuxia
>
> Best regards,
> Yuxia
>
> - 原始邮件 -
> 发件人: "kui yuan"
> 收件人: "dev"
> 抄送:
/connectors/table/filesystem/#sink-parallelism
Best regards,
Yuxia
Best regards,
Yuxia
- 原始邮件 -
发件人: "kui yuan"
收件人: "dev"
抄送: "Jark Wu"
发送时间: 星期三, 2023年 2 月 22日 下午 10:08:11
主题: Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL
Hi all,
Than
Hi all,
Thanks for the lively discussion and I will respond to these questions one
by one. However, there are also some common questions and I will answer
together.
@郑 Thanks for your reply. The features mentioned in this flip are only for
those source connectors that implement the
Hi Kui,
Thanks for driving the discussion. It's quite useful to introduce Watermark
options. I have some questions:
What kind of hints is "WATERMARK_PARAMS"?
Currently, we have two kinds of hints in Flink: Dynamic Table Options &
Query Hints. As described in the Flip, "WATERMARK_PARAMS" is more
Hi Kui,
Thanks for bringing this to the discussion. Toward the FLIP, I have several
questions.
1. What's the behavior if there are multiple table sources, among which
some do not support `SupportsWatermarkPushDown`?
> So the features that this flip intends to support are only for those
>
Hi Yuan Kui,
Thanks for creating the FLIP. A couple of questions / remarks
1. While the FLIP talks about watermark options for Table API & SQL, I only
see proposed syntax for SQL, not for the Table API. What is your proposal
for the Table API?
2. A rejected alternative is adding watermark
Hi kui
Thanks for initializing this discussion. I have two questions
1. What will happen if we add watermark related options in `the connector
options`? Will the connector ignore these options or throw an exception?
How can we support this?
2. If one table is used by two operators with
Hi kui,
There is a scenario using watermark that cannot be implemented through SQL now：
We only support specify rowtime column in table DDL，if the rowtime field is
generated by join dimension table , it cannot be implemented
Can we consider implement through HINTS like :
Select * from t1 join
Sent: Wednesday, February 22, 2023 10:22
To: dev@flink.apache.org
Subject: [DISCUSS] FLIP-296: Watermark options for table API & SQL
Hi devs,
I'd like to start a discussion thread for FLIP-296[1]. This comes from an
offline discussion with @Yun Tang, and we hope to enrich table API & SQL to
Hi devs,
I'd like to start a discussion thread for FLIP-296[1]. This comes from an
offline discussion with @Yun Tang, and we hope to enrich table API & SQL to
support many watermark-related features which were only implemented at the
datastream API level.
Basically, we want to introduce
31 matches
Mail list logo
