Hi, all.
Considering what Martijn suggested, as a big break, we can directly remove the
old group window agg operator in 2.0
without rewriting it to the new window agg operator.
At the same time, every subtask in this FLIP to align the new and old window
agg operators is almost covered
by
Hi, Martijn.
Thank you for your reminder :)
My idea is that in the current 1.x version, we can automatically convert the
agg operator in the old grammar into the agg operator
in the new grammar. Huge changes will be introduced in version 2.0 that old
syntax will be directly deleted at the code
Hi Xuyang,
It's currently the holiday season in Europe so do expect some slow responses.
> The key reason the original FLIP is called "Deprecate the Legacy Group Window
> Aggregation" is that we also plan to automatically rewrite the group window
> agg corresponding the old syntax into the
Hi, Timo. Sorry to bother you. There's something I really need to hear your
thoughts on.
When I'm trying to split this flip, having reviewed this discussion and the
FLIP document again, I realized that there is still a key issue that hasn't
been clarified. The key reason the original FLIP
Hi Xuyang,
sorry I missed the ping. Sounds reasonable to me. One FLIP about
changelog semantics, the other about SQL semantics.
Regards,
Timo
On 19.12.23 02:39, Xuyang wrote:
Hi, Timo. Sorry for this noise.
What do you think about splitting the flip like this?
--
Best！
Xuyang
Hi, Timo. Sorry for this noise.
What do you think about splitting the flip like this?
--
Best！
Xuyang
At 2023-12-15 10:05:32, "Xuyang" wrote:
>Hi, Timo, thanks for your advice.
>
>
>I am considering splitting the existing flip into two while leaving the
>existing flip (or
Hi, Timo, thanks for your advice.
I am considering splitting the existing flip into two while leaving the
existing flip (or without).
One of them points to the completion of the operator about window tvf to
support CDC (there are several
small work items, such as window agg, window rank,
Hi Xuyang,
> I'm not spliting this flip is that all of these subtasks like session
window tvf and cdc support do not change the public interface and the
public syntax
Given the length of this mailing list discussion and number of involved
people I would strongly suggest to simplify the FLIP
Hi, Timo, Sergey and Lincoln Lee. Thanks for your feedback.
> In my opinion the FLIP touches too many
> topics at the same time and should be split into multiple FLIPs. We > should
> stay focused on what is needed for Flink 2.0.
The main goal and topic of this Flip is to align the abilities
Thanks Xuyang driving this work! It's great that everyone agrees with the
work itself in this flip[1]!
Regarding whether to split the flip or adjust the scope of this flip, I'd
like to share some thoughts:
1. About the title of this flip, what I want to say is that flip-145[2] had
marked the
thanks for summarising Timo
+1 for splitting it in different FLIPs
and agree about having "SESSION Window TVF Aggregation" under FLIP-145
Moreover the task is already there, so no need to move it from one FLIP to
another
>And actually Sergey Nuyanzin wanted to work
>in this if I remember
Hi Xuyang,
thanks for proposing this FLIP. In my opinion the FLIP touches too many
topics at the same time and should be split into multiple FLIPs. We
should stay focused on what is needed for Flink 2.0.
Let me summarizing the topics:
1) SESSION Window TVF Aggregation
This has been agreed
Hi Xuyang,
Thank you for the clarification. I now have a clear understanding of the
issue.
Actually the work that supports unaligned window in window agg operator is
> included in this flip. We will always try to implement
> features in window agg operator first and then consider falling back to
Hi, Jim.
Thanks for your explaination.
>Ah, I mean to ask if you can contribute the new SESSION Table support
>without needing FLIP-392 completely settled. I was trying to see if that
>is separate work which can be done or if there is some dependency on this
>FLIP.
The pr available about session
Hi Xuyang,
On Sun, Dec 10, 2023 at 10:41 PM Xuyang wrote:
> Hi, Jim.
> >As a clarification, since FLINK-24204 is finishing up work from
> >FLIP-145[1], do we need to discuss anything before you work out the
> details
> >of FLINK-24024 as a PR?
> Which issue do you mean? It seems that
Hi Xuyang,
Thanks for driving this discussion.
I briefly reviewed the first PoC code[1] you provided for implementing the
session window TVF, and I noticed a problem. In the `translateToExecNode`
method of `StreamPhysicalWindowAggregate`, the session window TVF is
translated into the legacy
Hi, Jim.
>As a clarification, since FLINK-24204 is finishing up work from
>FLIP-145[1], do we need to discuss anything before you work out the details
>of FLINK-24024 as a PR?
Which issue do you mean? It seems that FLINK-24204[1] is the issue with table
api type system.
> I've got a PR up [3]
Hi, Shengkai.
> I think we shouldn't remove the operator if we can not give a solution to
> help users upgrade their jobs. But I think we can delay the discussion until
> we need to remove the operator.
+1 for it.
--
Best！
Xuyang
在 2023-12-08 19:22:40，"Shengkai Fang"
Hi Xuyang,
As a clarification, since FLINK-24204 is finishing up work from
FLIP-145[1], do we need to discuss anything before you work out the details
of FLINK-24024 as a PR?
Relatedly, as that goes up for a PR, as part of FLINK-33421 [2], Bonnie and
I are working through migrating some of the
Hi, Xuyang. Thanks for your response.
I just thought an another way to solve this question instead of introducing
> a new configuration. When using legacy syntax like `GROUP BY TUMBLE(xxx),
> f0`, the rewritten sql can be GROUP BY f0, window_start,
> window_end(window_start and window_end is
Hi, Martijn, thanks for your share.
>On the topic of syntax for early/late fires, there is existing
>configuration for the legacy group windows:
>
>SET table.exec.emit.early-fire.enabled = true;
>SET table.exec.emit.early-fire.delay = 5s;
>SET table.exec.emit.late-fire.enabled = true;
>SET
Hi Xuyang,
Thanks a lot for starting this discussion.
At first, I was a bit confused because the FLIP talks about
deprecating the Legacy Group Window Aggregations, but they have
already been marked as deprecated in the documentation [1].
My understanding was that the big challenge was that we
Hi, Shengkai. Thanks to share your thought. Let me answer related questions。
> Could you give an example about the pass-through column. A session window may
> contain multiple rows, which value is selected by the windowoperator?
The table function make the entire inpyt row available in the
Hi, xuyang
Thanks for starting this FLIP discussion, currently there are two types of
window aggregation in Flink SQL, namely legacy group window aggregation and
window tvf aggregation, these two types of window aggregation are not fully
aligned in behavior, which will bring a lot of confusion to
Hi, Xuyang. Thanks for your great work. Big +1!
After reading the FLIP, I have some questions. Please read the content
below.
1. Support SESSION Window TVF Aggregation
a. Could you give an example about the pass-through column? A session
window may contain multiple rows, which value is
Hi, Feng and David.
Thank you very much to share your thoughts.
This flip does not include the official exposure of these experimental conf to
users. Thus there is not adetailed description of this part.
However, in view that some technical users may have added these experimental
conf in
The current situation (where we have both the legacy windows and the
TVF-based windows) is confusing for users, and I'd like to see us move
forward as rapidly as possible.
Since the early fire, late fire, and allowed lateness features were never
documented or exposed to users, I don't feel that
Hi xuyang,
Thank you for initiating this proposal.
I'm glad to see that TVF's functionality can be fully supported.
Regarding the early fire, late fire, and allow lateness features, how will
they be provided to users? The documentation doesn't seem to provide a
detailed description of this
Hi all.
I'd like to start a discussion of FLIP-392: Deprecate the Legacy Group Window
Aggregation.
Although the current Flink SQL Window Aggregation documentation[1] indicates
that the legacy Group Window Aggregation
syntax has been deprecated, the new Window TVF Aggregation syntax has not
29 matches
Mail list logo
