Hi everyone,
I started a vote for this FLIP [1], and the voting thread can be found at
[2]. If you have any questions, please don't hesitate to join this
discussion.
[1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240883951
[2]
Hi, devs,
Thanks for all the feedback.
Based on the discussion [1], we seem to have a consensus so far, so I would
like to start a vote on FLIP-292 [2], which begins on the following Monday
(Apr. 10th at 10:00 AM GMT).
If you have any questions or concerns, please don't hesitate to follow up
on
Hi, devs,
Thanks for your valuable feedback. I've changed the title of the FLIP to
"Enhance COMPILED PLAN to support operator-level state TTL configuration"
and added an explanation for StateMetadata. If you have any concerns,
please let me know.
Best regards,
Jane
On Mon, Apr 3, 2023 at 11:42
Hi Timo,
Thanks for your valuable feedback. Let me explain the design details.
> However, actually fine-grained state TTL should already be possible
today. I don't fully understand where your proposed StateMetadata is
located? Would this be a value of @ExecNodeMetadata, StreamExecNode, or
Hi Jane,
Thanks for your clarification. +1 for using this feature for advanced use
cases.
Best regards,
Jing
On Mon, Apr 3, 2023 at 4:43 PM Jane Chan wrote:
> Hi Yisha,
>
> Thanks for your detailed explanation. Here are my thoughts.
>
> > I'm interested in how to design a graphical interface
Hi Yisha,
Thanks for your detailed explanation. Here are my thoughts.
> I'm interested in how to design a graphical interface to help users to
maintain their custom fine-grained configuration between their job versions.
Regarding the graphical IDE, this was mentioned to address the concern of
Hi Jane,
thanks for proposing this FLIP. More state insights and fine-grained
state TTL are a frequently requested feature for Flink SQL. Eventually,
we need to address this.
I agree with the previous responses that doing this with a hint might
cause more confusion than it actually helps.
Hi Jane,
Thanks for driving this FLIP.
I think the compiled plan solution and the hint solution do not
conflict, the two can exist at the same time.
The compiled plan solution can address the need of advanced users and
the platform users
which all stateful operators' state TTL can be defined by
Hi Jane,
Thanks for your detailed response.
You mentioned that there are 10k+ SQL jobs in your production
> environment, but only ~100 jobs' migration involves plan editing. Is 10k+
> the number of total jobs, or the number of jobs that use stateful
> computation and need state migration?
>
10k
Hi guys
Thanks for your enlightening opinions which have benefited me greatly. And I
have some experiences about configuring something on graphical IDE to share
with you. Hopefully, these experiences can give you some useful information.
At the FFA 2021, we proposed a solution for state
Hi Yisha,
Thank you for the valuable feedback! I appreciate that you find this
proposal beneficial. I'd like to answer your questions and explain the
reason why we don't prefer to use SQL hints.
> 1. Hints may not only aim to inspire the planner. For example, we have
dynamic options for tables
Hi Jane,
Thanks for driving this, and your solution is absolutely creative. In my
company, there also exist some scenarios in which users want to config state
TTL at operator level, especially for window operators which regard TTL as
allow lateness.
To support this scenarios, we implemented
12 matches
Mail list logo
