Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-06 Thread Rui Fan
nk we run into this problem in practice and we can keep the > metric aligned with other time rate metrics like busyTimeMsPerSec etc. > > Cheers, > Gyula > > On Wed, Sep 6, 2023 at 4:55 AM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Gyula, > > > > +1

Re: [DISCUSS] FLIP-361: Improve GC Metrics

2023-09-05 Thread Rui Fan
Hi Gyula, +1 for this proposal. The current GC metric is really unfriendly. I have a concern with your proposed rate metric: the rate is perSecond instead of per minute. I'm unsure whether it's suitable for GC metric. There are two reasons why I suspect perSecond may not be well compatible with

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-09-06 Thread Rui Fan
t, is to have an alternative implementation in addition to > > Kubernetes. YARN or a standalone mode implementation were already > > mentioned. Ultimately, this is the reason we are doing the > > refactoring. Without a new implementation, it becomes harder to > > justify t

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-09-05 Thread Rui Fan
ess). > > > > -- > > Best, > Matt Wang > > > Replied Message > | From | Rui Fan<1996fan...@gmail.com> | > | Date | 08/21/2023 17:41 | > | To | | > | Cc | Maximilian Michels , > Gyula Fóra , > Matt Wang | > | Subject | Re: [DISCU

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-09-07 Thread Rui Fan
P and we can > definitely help review code changes and build on the new design. > > -Max > > > On Wed, Sep 6, 2023 at 11:26 AM Rui Fan <1996fan...@gmail.com> wrote: > > > > Hi Max, > > > > As the FLIP mentioned, we have the plan to add the > &g

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-10 Thread Rui Fan
Thanks Zhanghao driving this FLIP, adding the port in Web UI seems good to me. Hi Shammon and Zhanghao, I would like to clarify the difference between Public Interfaces in FLIP and @Public in code. As I understand, the `Public Interfaces in FLIP` means these changes will be used in user side,

[RESULT][VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

2023-09-15 Thread Rui Fan
(binding) - Dong Lin (binding) - Rui Fan(binding) - Ahmed Hamdy - ConradJam - Matt Wang - Ferenc Csaky - Zhanghao Chen - Feng Jin - Samrat Deb - Ron Liu There is no disapproving vote. [1] https://cwiki.apache.org/confluence/x/x4qzDw [2] https://lists.apache.org/thread/3wmhhqgkkg1l7ncxnzwqnjqyhqz545gl

Re: [VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

2023-09-15 Thread Rui Fan
+1(binding) Best, Rui On Thu, Sep 14, 2023 at 10:06 AM liu ron wrote: > +1(non-binding) > > Best, > Ron > > Dong Lin 于2023年9月14日周四 09:01写道: > > > Thank you Rui for the proposal. > > > > +1 (binding) > > > > On Wed, Sep 13, 2023 at 10:52 AM

[VOTE] FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler

2023-09-12 Thread Rui Fan
Hi all, Thanks for all the feedback about the FLIP-334: Decoupling autoscaler and kubernetes and support the Standalone Autoscaler[1]. This FLIP was discussed in [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours (until Sep 16th 11:00 UTC+8) unless there is an

Re: [VOTE] FLIP-361: Improve GC Metrics

2023-09-13 Thread Rui Fan
+1(binding) Best, Rui On Wed, Sep 13, 2023 at 9:16 PM Gyula Fóra wrote: > Hi All! > > Thanks for all the feedback on FLIP-361: Improve GC Metrics [1][2] > > I'd like to start a vote for it. The vote will be open for at least 72 > hours unless there is an objection or insufficient votes. > >

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-13 Thread Rui Fan
One minor comment: In general, the generic java profiler includes memory analysis, cpu, thread, deadlock, etc. The FLIP title is java profiler, but the FLIP just supports flamegraph at process level. So the `powerful java profiler` title may not be suitable. Would you mind updating the FLIP

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-16 Thread Rui Fan
to TM and task > to > > slot. > > 1. Slot to TM is guaranteed by SlotManager in ResourceManager > > 2. Task to slot is guaranteed by the slot pool in JM > > > > These two are completely independent, what are the benefits of unifying > > these two into one option? Also,

[DISCUSS] FLIP-364: Improve the restart-strategy

2023-10-16 Thread Rui Fan
Hi all, I would like to start a discussion on FLIP-364: Improve the restart-strategy[1] As we know, the restart-strategy is critical for flink jobs, it mainly has two functions: 1. When an exception occurs in the flink job, quickly restart the job so that the job can return to the running state.

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-16 Thread Rui Fan
mon configuration options. From my own experience, `cpu` > (requiring > > perf_events) is a bit more accurate than `itimer`, and if I recall, and > > samples once per thread. `wall` is very useful to debug blocks on I/O or > > locks. Getting per-thread information is nice to

Re: [VOTE] FLIP-375: Built-in cross-platform powerful java profiler

2023-10-17 Thread Rui Fan
+1(binding) Best, Rui On Tue, Oct 17, 2023 at 3:52 PM Yu Chen wrote: > Hi all, > > Thank you to everyone for the feedback and detailed comments on > FLIP-375[1]. > Based on the discussion thread [2], I think we are ready to take a vote to > contribute this to Flink. > I'd like to start a vote

Re: [ANNOUNCE] Apache Flink 1.18.0 released

2023-10-26 Thread Rui Fan
Thanks for the great work! Best, Rui On Fri, Oct 27, 2023 at 10:03 AM Paul Lam wrote: > Finally! Thanks to all! > > Best, > Paul Lam > > > 2023年10月27日 03:58,Alexander Fedulov 写道: > > > > Great work, thanks everyone! > > > > Best, > > Alexander > > > > On Thu, 26 Oct 2023 at 21:15, Martijn

Re: [DISCUSS] FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects

2023-11-01 Thread Rui Fan
Thanks Junrui for driving this proposal! ConfigOption is easy to use for flink users, easy to manage options for flink platform maintainers, and easy to maintain for flink developers and flink community. So big +1 for this proposal! Best, Rui On Thu, Nov 2, 2023 at 10:10 AM Junrui Lee wrote:

Re: Request to release flink 1.6.3

2023-11-03 Thread Rui Fan
e forward and make it. > > Best > Yun Tang > ____________ > From: Rui Fan <1996fan...@gmail.com> > Sent: Tuesday, October 31, 2023 23:06 > To: dev@flink.apache.org > Subject: Re: Request to release flink 1.6.3 > > Thanks Vikas for the ask! > > Hi

Re: Request to release flink 1.6.3

2023-11-04 Thread Rui Fan
] https://lists.apache.org/thread/szq23kr3rlkm80rw7k9n95js5vqpsnbv > > > On Fri, Nov 3, 2023 at 7:38 PM Rui Fan <1996fan...@gmail.com> wrote: > > > Thanks Yun and Yu, > > > > That would be great! Let's move forward! > > > > Best, > > Rui > > &g

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-07 Thread Rui Fan
Hi Piotr, Thanks for driving this proposal! The trace reporter is useful to check a lot of duration monitors inside of Flink. I have some questions about this proposal: 1. I see the trace just supports Span? Does it support trace events? I'm not sure whether tracing events is reasonable for

[ANNOUNCE] Apache Flink Kubernetes Operator 1.6.1 released

2023-10-30 Thread Rui Fan
://hub.docker.com/r/apache/flink-kubernetes-operator The full release notes are available in Jira: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12353784 We would like to thank all contributors of the Apache Flink community who made this release possible! Regards, Rui Fan

Re: [DISCUSS] Kubernetes Operator 1.7.0 release planning

2023-10-31 Thread Rui Fan
Thanks Gyula for driving this release! I'd like to check with you and community, could we postpone the code freeze by a week? I'm developing the FLINK-33099[1], and the prod code is done. I need some time to develop the tests. I hope this feature is included in 1.7.0 for two main reasons: 1. We

Re: Request to release flink 1.6.3

2023-10-31 Thread Rui Fan
helping it? Thanks~ Best, Rui On Tue, 31 Oct 2023 at 21:42, vikas patil wrote: > Hello Rui, > > Do we need more votes for this or are we good to go with the release of > 1.6.3 ? Please let me know. Thanks. > > -Vikas > > On Tue, Oct 24, 2023 at 9:27 AM Rui Fan <

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-26 Thread Rui Fan
nk-kubernetes-operator/139454270?tag=51eeae1 > > > And notice that it does not have a description . Is this correct? > > > > > > 1. I get this in the gpg verification . Is this ok? > > > > > > > > > gpg --verify flink-kubernetes-operator-1.6.

[RESULT] [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-26 Thread Rui Fan
I'm happy to announce that we have unanimously approved this release. There are 7 approving votes, 4 of which are binding: * Gyula Fora (binding) * Thomas Weise (binding) * Marton Balassi (binding) * Maximilian Mixhels (binding) * Mate Czagany (non-binding) * Samrat Deb (non-binding) * Rui Fan

[DISCUSS] Release Flink 1.16.3

2023-11-05 Thread Rui Fan
Hi all, I would like to discuss creating a new 1.16 patch release (1.16.3). The last 1.16 release is over five months old, and since then, 50 tickets have been closed [1], of which 10 are blocker/critical [2]. Some of them are quite important, such as FLINK-32296 [3], FLINK-32548 [4] and

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-25 Thread Rui Fan
ts.java:187) > > Thanks, > Thomas > > > On Sat, Oct 21, 2023 at 7:35 AM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Everyone, > > > > Please review and vote on the release candidate #1 for the version 1.6.1 > of > > Apache Flink Kuberne

Re: [DISCUSS] Creating Kubernetes Operator release 1.6.1

2023-10-19 Thread Rui Fan
Hi Gyula, Thank you for driving this discussion! This release seems good to me, I have a question: I see some bugfix commits have been merged into the 1.6-release branch. Does it already contain all recent bugfix commits? Also, you said in the `Kubernetes Operator 1.6.0 release planning`[1]: >

Re: [VOTE] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-17 Thread Rui Fan
A gentle reminder about the location naming. The naming of location is a little unclear, but I can't think of any other better naming. So I +1(binding) first. Ping @Jing Ge to help double check the name again. Sorry for mentioning naming in the VOTE thread, I didn't know this VOTE would be so

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-17 Thread Rui Fan
> > > > > Since the meaning of "host" is not aligned, it seems good for me to > remove > > > it in the future release. > > > > > > Best, > > > Weihua > > > > > > > > > On Mon, Sep 11, 2023 at 11:48 AM Chen Zhanghao < > zhanghao.

Re: [VOTE] FLIP-327: Support switching from batch to stream mode to improve throughput when processing backlog data

2023-09-22 Thread Rui Fan
+1(binding), thanks for driving this proposal. Best, Rui On Fri, Sep 22, 2023 at 5:16 PM Jing Ge wrote: > +1(binding) Thanks! > > Best regards, > Jing > > On Fri, Sep 22, 2023 at 8:08 AM Dong Lin wrote: > > > Hi all, > > > > We would like to start the vote for FLIP-327: Support switching from

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-18 Thread Rui Fan
ncept you think one might confuse it > with? > > Best, > Yangze Guo > > On Mon, Sep 18, 2023 at 12:07 PM Rui Fan <1996fan...@gmail.com> wrote: > > > > Hi Zhanghao, > > > > > Use a field named "location" (already used in > JobExcepti

Re: [VOTE] FLIP-314: Support Customized Job Lineage Listener

2023-09-28 Thread Rui Fan
+1(binding) Best, Rui On Thu, 28 Sep 2023 at 14:41, Chen Zhanghao wrote: > +1 (non-binding), thanks for driving this. > > Best, > Zhanghao Chen > > 发件人: Shammon FY > 发送时间: 2023年9月25日 13:28 > 收件人: dev > 主题: [VOTE] FLIP-314: Support Customized Job Lineage

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-01 Thread Rui Fan
the benefits of unifying > these two into one option? Also, do we want to share the same > option between SlotPool in JM and SlotManager in RM? This sounds a bit > strange. > > Best, > Shammon FY > > > > On Thu, Sep 28, 2023 at 12:08 PM Rui Fan <1996fan...@gmail.com> wr

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-01 Thread Rui Fan
forward to your feedback, thanks! Best, Rui On Sun, Oct 1, 2023 at 5:52 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Yangze, > > Thanks for your feedback! > > > 1. Is it possible for the SlotPool to get the slot allocation results > > from the SlotManager in advanc

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-09-27 Thread Rui Fan
Hi Zhu Zhu, Thanks for your feedback here! You are right, user needs to set 2 options: - cluster.evenly-spread-out-slots=true - slot.sharing-strategy=TASK_BALANCED_PREFERRED Update it to one option is useful at user side, so `taskmanager.load-balance.mode` sounds good to me. I want to check

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-18 Thread Rui Fan
> > > top level service that aligns with the top level endpoint. > > > > > > WDYT? > > > > > > Best regards, > > > Jing > > > > > > > > > On Mon, Sep 11, 2023 at 6:01 AM Weihua Hu > wrote: > > > > > >

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-10 Thread Rui Fan
{cluster.evenly-spread-out-slots: > true, slot.sharing-strategy: LOCAL_INPUT_PREFERRED} > {taskmanager.load-balance.mode: Tasks} = {cluster.evenly-spread-out-slots: > true, slot.sharing-strategy: TASK_BALANCED_PREFERRED} > > Thanks, > Zhu > > Rui Fan <1996fan...@gmai

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
Thanks Yun and Yu for driving this proposal! It's very useful for troubleshooting why the CPU usage is high. +1 Best, Rui On Mon, Oct 9, 2023 at 7:21 PM Zhanghao Chen wrote: > Hi Yun and Yu, > > Thanks for driving this. This would definitely help users identify > performance bottlenecks,

Re: [VOTE] FLIP-366: Support standard YAML for FLINK configuration

2023-10-12 Thread Rui Fan
+1(binding) Best, Rui Fan On Fri, Oct 13, 2023 at 10:12 AM Junrui Lee wrote: > Hi all, > > Thank you to everyone for the feedback on FLIP-366[1]: Support standard > YAML for FLINK configuration in the discussion thread [2]. > I would like to start a vote for it. The vo

Re: [DISCUSS] FLIP-374: Adding a separate configuration for specifying Java Options of the SQL Gateway

2023-10-07 Thread Rui Fan
Thanks to Yangze driving this proposal. `env.java.opts.xxx` is already supported for client, historyserver, jobmanager and taskmanager. And it's very useful for troubleshooting. So +1 for `env.java.opts.sql-gateway`. I have a minor question: doesn't the `env.java.opts.all` support sql-gateway?

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-07 Thread Rui Fan
Hi Shammon, IIUC, you want more flexibility in controlling the two-phase strategy, right? > I want this because we would like to add a new slot to TM strategy such as SLOTS_NUM in the future for OLAP to improve the performance for olap jobs, which will use TASKS strategy for task to slot. cc

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-07 Thread Rui Fan
> (taskmanager.numberOfTaskSlots=3): > TM1: 3 slot > TM2: 2 slot > TM3: 2 slot > > Best, > Yangze Guo > > On Sun, Oct 1, 2023 at 6:14 PM Rui Fan <1996fan...@gmail.com> wrote: > > > > Hi Shammon, > > > > Thanks for your feedback as we

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-07 Thread Rui Fan
ng another waiting mechanism. I > think it would be better to address this issue within the scope of > this FLIP. However, I don't have a strong opinion on it, it depends on > your bandwidth. > > > Best, > Yangze Guo > > On Sat, Oct 7, 2023 at 4:16 PM Rui Fan <1996fan...@gmail.

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-10-19 Thread Rui Fan
> > but of course allowing users to change is valuable. > > > > So, overall +1. > > > > Cheers, > > > > Konstantin > > > > Am Di., 17. Okt. 2023 um 07:11 Uhr schrieb Rui Fan <1996fan...@gmail.com > >: > > > > > Hi all, &

Re: [VOTE] FLIP-374: Adding a separate configuration for specifying Java Options of the SQL Gateway

2023-10-10 Thread Rui Fan
+1(binding) Best, Rui On Wed, Oct 11, 2023 at 10:07 AM Yangze Guo wrote: > Hi everyone, > > I'd like to start the vote of FLIP-374 [1]. This FLIP is discussed in > the thread [2]. > > The vote will be open for at least 72 hours. Unless there is an > objection, I'll try to close it by October

Re: [DISCUSS] Creating Kubernetes Operator release 1.6.1

2023-10-20 Thread Rui Fan
cus on critical fixes / regressions to make > the changes minimal given that the 1.7.0 is not so far anyway :) > > Cheers, > Gyula > > > On Thu, Oct 19, 2023 at 8:20 AM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Gyula, > > > > Thank you for driving

Re: [ANNOUNCE] The Flink Speed Center and benchmark daily run are back online

2023-10-20 Thread Rui Fan
Thanks for your effort! It's very useful when some new commits affect performance. Best, Rui On Fri, Oct 20, 2023 at 4:42 PM Yanfei Lei wrote: > Thanks for your hard work! > Looking forward to the daily monitoring being available again soon. > > Best, > Yanfei > > Yuan Mei 于2023年10月20日周五

[VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-21 Thread Rui Fan
Hi Everyone, Please review and vote on the release candidate #1 for the version 1.6.1 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of

Re: [VOTE] Release 1.18.0, release candidate #3

2023-10-21 Thread Rui Fan
+1(non-binding) - Downloaded artifacts from dist[1] - Verified SHA512 checksums - Verified GPG signatures - Build the source with java-1.8 and verified the licenses together - Verified web PR [1] https://dist.apache.org/repos/dist/dev/flink/flink-1.18.0-rc3/ Best, Rui On Fri, Oct 20, 2023 at

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-21 Thread Rui Fan
, 2023 at 7:33 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Everyone, > > Please review and vote on the release candidate #1 for the version 1.6.1 of > Apache Flink Kubernetes Operator, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the relea

Re: [VOTE] FLIP-370: Support Balanced Tasks Scheduling

2023-10-22 Thread Rui Fan
+1(binding) Thanks to Yuepeng and to everyone who participated in the discussion! Best, Rui On Mon, Oct 23, 2023 at 11:55 AM Roc Marshal wrote: > Hi all, > > Thanks for all the feedback on FLIP-370[1][2]. > I'd like to start a vote for FLIP-370. The vote will last for at least 72 > hours

Re: [ANNOUNCE] New Apache Flink Committer - Jane Chan

2023-10-15 Thread Rui Fan
Congratulations Jane! Best, Rui On Mon, Oct 16, 2023 at 10:15 AM yu zelin wrote: > Congratulations! > > Best, > Yu Zelin > > > 2023年10月16日 09:58,Jark Wu 写道: > > > > Hi, everyone > > > > On behalf of the PMC, I'm very happy to announce Jane Chan as a new Flink > > Committer. > > > > Jane

Re: [ANNOUNCE] New Apache Flink Committer - Ron Liu

2023-10-15 Thread Rui Fan
Congratulations Ron ! Best, Rui On Mon, Oct 16, 2023 at 10:05 AM Lijie Wang wrote: > Congratulations Ron ! > > Best, > Lijie > > Samrat Deb 于2023年10月16日周一 10:03写道: > > > Congratulations Ron Liu :) > > > > On Mon, 16 Oct 2023 at 7:29 AM, tison wrote: > > > > > Congrats! Glad to see more and

Re: [DISCUSS] FLIP-375: Built-in cross-platform powerful java profiler on taskmanagers

2023-10-09 Thread Rui Fan
Hi Jing, > 1. will it replace the current flame graph, i.e. the current flame graph will be deprecated and removed? I think the current flame graph cannot be removed. As a core contributor to the current flame graph, and I use it almost every week. I would like to clarify the difference between

Re: [DISCUSS] FLIP-370 : Support Balanced Tasks Scheduling

2023-10-09 Thread Rui Fan
low the SlotManager to see all resource >> requirements, then it can allocate the slot more evenly. As a side >> effect, it could also significantly reduce the number of RPC messages >> to the ResourceManager, which could become a single-point bottleneck >> in OLAP scenarios

Re: Request to release flink 1.6.3

2023-10-24 Thread Rui Fan
Hi Vikas, Thanks for your feedback! Do you mean flink 1.16.3 instead of 1.6.3? The 1.16.2 and 1.17.1 were released on 2023-05-25, it’s been 5 months. And the flink community has fixed many bugs in the past 5 months. Usually, there is a fix(minor) version every three or four months, so I propose

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-21 Thread Rui Fan
:55 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi Matt Wang, > > Thanks for your discussion here. > > > it is recommended to unify the descriptions of AutoScalerHandler > > and AutoScalerEventHandler in the FLIP > > Good catch, I have updated all AutoScale

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Hi Jing, Thanks for the effort and update! It means the PR of flink-1.19 can be merged to master branch, right? Best, Rui On Wed, Aug 23, 2023 at 9:29 PM Jing Ge wrote: > Hi devs, The release-1.18 branch has been forked out from the master > branch, with commit ID

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Thanks for the quick response! Best, Rui On Wed, Aug 23, 2023 at 9:46 PM Jing Ge wrote: > yes please > > On Wed, Aug 23, 2023 at 3:35 PM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Jing, > > > > Thanks for the effort and update! > > > > It me

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.0, release candidate #2

2023-08-14 Thread Rui Fan
Thanks Gyula for the release! +1 (non-binding) - Compiled and tested the source code via mvn verify - Verified the signatures - Downloaded the image - Deployed helm chart to test cluster - Ran example job Best, Rui On Mon, Aug 14, 2023 at 3:58 PM Gyula Fóra wrote: > +1 (binding) > >

[DISCUSS] FLIP-227: Support overdraft buffer

2022-04-26 Thread rui fan
Hi everyone, Unaligned Checkpoint (FLIP-76 [1]) is a major feature of Flink. It effectively solves the problem of checkpoint timeout or slow checkpoint when backpressure is severe. We found that UC(Unaligned Checkpoint) does not work well when the back pressure is severe and multiple output

Re: [VOTE] FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects

2023-11-10 Thread Rui Fan
+1(binding) Best, Rui On Fri, Nov 10, 2023 at 11:58 AM Junrui Lee wrote: > Hi everyone, > > Thank you to everyone for the feedback on FLIP-381: Deprecate configuration > getters/setters that return/set complex Java objects[1] which has been > discussed in this thread [2]. > > I would like to

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-10 Thread Rui Fan
I'll start voting next Monday if there isn't any other comment. Best, Rui On Thu, Oct 19, 2023 at 6:59 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Konstantin and Max, > > Thanks for your feedback! > > Sorry, I forgot to mention the default value of > `restart-strate

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-08 Thread Rui Fan
> > > > > This is done just for the sake of simplicity, and test out the > basic > > > > > functionality. But I think, > > > > > those currently missing features should be added at some point in > > > > > the future. > > > > >

Re: [DISCUSS] Release Flink 1.16.3

2023-11-08 Thread Rui Fan
Hi All, Thank you for your feedback! As there are no other concerns or objections, and currently I am not aware of any unresolved blockers. I will kick off the release process and start preparing for the RC1 version from today. Best, Rui On Wed, Nov 8, 2023 at 4:23 PM ConradJam wrote: > +1

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-08 Thread Rui Fan
> Hi Rui, > > > > > > Thanks for the proposal. > > > > > > +1 I don't have any major comments :) > > > > > > One nit. In `SystemOutRedirectToLog` in this code: > > > > > > System.arraycopy(buf,

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-25 Thread rui fan
ks for all the discussion about this FLIP again, I will open a vote today. Best wishes fanrui On Fri, May 6, 2022 at 4:57 PM rui fan <1996fan...@gmail.com> wrote: > Hi > > I created the FLINK-27530[1] as the parent ticket. And I > updated it to FLIP. > > [1] https://issues.

[VOTE] FLIP-227: Support overdraft buffer

2022-05-25 Thread rui fan
Hi, everyone, Thanks for your feedback for FLIP-227: Support overdraft buffer[1] on the discussion thread[2]. I'd like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1]

Re: [VOTE] FLIP-168: Speculative Execution for Batch Job

2022-05-26 Thread rui fan
Hi +1(non-binding), it’s very useful for batch job stability. Best wishes fanrui On Thu, May 26, 2022 at 15:56 Zhu Zhu wrote: > Hi everyone, > > Thanks for the feedback for FLIP-168: Blocklist Mechanism [1] on the > discussion thread [2]. > > I'd like to start a vote for it. The vote will

All flink-ci fail

2022-05-24 Thread rui fan
Hi masters, All compile_ci of flink-ci[1] failed today, seemingly for the same reason: 05:03:16,982 ERROR org.apache.flink.tools.ci.suffixcheck.ScalaSuffixChecker [] - Violations found: Scala-free module 'flink-hadoop-compatibility' is referenced with scala suffix in

Re: All flink-ci fail

2022-05-24 Thread rui fan
Hi Chesnay Schepler Thanks for your information. Best wishes fanrui On Tue, May 24, 2022 at 4:21 PM Chesnay Schepler wrote: > See https://issues.apache.org/jira/browse/FLINK-27751 > > On 24/05/2022 08:12, rui fan wrote: > > Hi masters, > > > > All compile_ci

[RESULT][VOTE] FLIP-227: Support overdraft buffer

2022-05-29 Thread rui fan
Hi everyone, FLIP-227[1] has been accepted. There are 5 approving votes, 3 of which are binding.[2] - Non-Binding: Anton Kalashnikov - Binding: Piotr Nowojski - Binding: Dawid Wysakowicz - Binding: David Morávek - Non-Binding: fanrui None against. Thanks again to everyone who is concerned

Re: [VOTE] FLIP-227: Support overdraft buffer

2022-05-29 Thread rui fan
> > > Thanks Fanrui for this FLIP. I think it will be useful thing for us. > > > > > > > > +1(non-binding) > > > > > > > > -- > > > > > > > > Best regards, > > > > Anton Kalashnikov > > > > &g

Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-04 Thread Rui Fan
Hi. Thanks Mang for this FLIP. I think it will be useful for users. +1(non-binding) Best wishes Rui Fan On Mon, Jul 4, 2022 at 3:01 PM Mang Zhang wrote: > Hi everyone, > > > > > Thanks for all the feedback so far. Based on the discussion [1], we seem > to have consen

RE: Re: [VOTE] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

2022-07-06 Thread Rui Fan
Thanks Mang for this FLIP. I think it will be useful for users. +1(non-binding) Best wishes Rui Fan On 2022/07/06 08:13:11 Jiangang Liu wrote: > +1 for the design > > Jark Wu 于2022年7月5日周二 16:04写道: > > > +1 (binding) > > > > Best, > > Jark > > >

Re: [ANNOUNCE] Apache Flink Kubernetes Operator 1.0.0 released

2022-06-05 Thread rui fan
Thanks Yang for driving the release, and thanks to all contributors for making this release happen! Best wishes Rui Fan On Sun, Jun 5, 2022 at 4:14 PM Yang Wang wrote: > The Apache Flink community is very happy to announce the release of Apache > Flink Kubernetes Operator 1.0.0. > &g

Re: [ANNOUNCE] New Apache Flink PMC Member - Jingsong Lee

2022-06-13 Thread Rui Fan
Congratulations, Jingsong! Best, Rui Fan On Mon, Jun 13, 2022 at 3:40 PM LuNing Wang wrote: > Congratulations, Jingsong! > > Best, > LuNing Wang > > Ingo Bürk 于2022年6月13日周一 15:36写道: > > > Congrats, Jingsong! > > > > On 13.06.22 08:58, Becket Qin wrote: &

Re: [VOTE] FLIP-228: Support Within between events in CEP Pattern

2022-06-13 Thread Rui Fan
Thanks for Nicholas driving this. +1(non-binding) Best, Rui Fan On Mon, Jun 13, 2022 at 3:05 PM md peng wrote: > +1(not-binding) > > Best, > Mingde Peng > > Martijn Visser 于2022年6月13日周一 14:48写道: > > > +1 (binding) > > > > Looking forward :) > > &

Re: [DISCUSS ] Make state.backend.incremental as true by default

2022-06-13 Thread Rui Fan
Strongly +1 Best, Rui Fan On Mon, Jun 13, 2022 at 7:35 PM Martijn Visser wrote: > > BTW, from my knowledge, nothing would happen for HashMapStateBackend, > which does not support incremental checkpoint yet, when enabling > incremental checkpoints. > > Thanks Yun, if no

Re: [VOTE] FLIP-238: Introduce FLIP-27-based Data Generator Source

2022-07-19 Thread Rui Fan
+1(non-binding) New Source can better support some features, such as Unaligned Checkpoint, Watermark alignment, etc. The data generator based on the new Source is very helpful for daily testing. Very much looking forward to using it. Best wishes Rui Fan On Wed, Jul 20, 2022 at 4:22 AM Martijn

Re: [VOTE] FLIP-247 Bulk fetch of table and column statistics for given partitions

2022-07-27 Thread Rui Fan
+1 non-binding Best Rui Fan On Tue, Jul 26, 2022 at 4:05 PM Jingsong Li wrote: > +1 > > Best, > Jingsong > > On Tue, Jul 26, 2022 at 10:11 AM godfrey he wrote: > > > > +1 > > > > Best, > > Godfrey > > > > Jark Wu 于2022年7月25日周一 17

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-06 Thread rui fan
t; > Can you create a parent ticket for the whole FLIP to group all of the > issues together? > > Also FLIP should be officially voted first. > > Best, > Piotrek > > pt., 6 maj 2022 o 09:08 rui fan <1996fan...@gmail.com> napisał(a): > > > Hi Anton, Piot

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-06 Thread rui fan
lly we should > enable/disable it in the constructors, but that might be tricky. > > > I prefer it to be between 5 and 10 > > I would vote for a smaller value because of FLINK-13203 > > Piotrek > > > > czw., 5 maj 2022 o 11:49 rui fan <1996fan...@gmail.com> napisał(a): > >

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-04-27 Thread rui fan
t; complicate it, especially if it will be possible to resolve the problem > automatically(as described above). > > > So is my understanding about network memory/buffers correct? > > -- > > Best regards, > Anton Kalashnikov > > 27.04.2022 07:46, rui fan пишет: > > Hi ev

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-04-29 Thread rui fan
r how to handle LegacySource since it actually > works as unlimited flatmap and it will always work in overdraft mode > which is not a target. So we still need to think about that. > > > 29.04.2022 11:11, rui fan пишет: > > Hi Anton Kalashnikov, > > > > I think you

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-04-29 Thread rui fan
or LegacySource in > our >internal version. It works well. > > I don't really understand how it is possible for general case considering > that each user has their own implementation of LegacySourceOperator > > -- 5. For the benchmark, do you have any suggestions? I submitted

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-04-29 Thread rui fan
isAvailable elsewhere. I don't think it is elegant, but it's safe. Please correct me if I'm wrong. Thanks fanrui On Fri, Apr 29, 2022 at 10:23 PM rui fan <1996fan...@gmail.com> wrote: > Hi, > > Thanks for your quick response. > > For question 1/2/3, we think they ar

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-03 Thread rui fan
t we build to support multiple interfaces means adding more > > complexity and more possibilities for bugs. Let's try to make our lives a > > little bit easier. > > > > Best regards, > > > > Martijn Visser > > https://twitter.com/MartijnVisser82 > > ht

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-02 Thread rui fan
(including legacy source > tasks). Would it be a big issue if we changed it to check if at least > "overdraft number of buffers are available", where "overdraft number" is > configurable, instead of the currently hardcoded value of "1"? > > Best, >

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-04 Thread rui fan
gt; > >> I want to emphasize that overdraft buffers are soft configuration which > >> means it takes as many buffers as the global buffers pool has > >> available(maybe zero) but less than this configured value. It is also > >> important to notice that perhaps, not man

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-05-05 Thread rui fan
gt; > When we finish this feature and after users use it, if users feedback > > this issue we can discuss again. > > +1 > > Piotrek > > [1] https://issues.apache.org/jira/browse/FLINK-13203 > [2] https://issues.apache.org/jira/browse/FLINK-14872 > > czw., 5 maj 2022

Re: [ANNOUNCE] New Flink PMC member: Yang Wang

2022-05-05 Thread rui fan
Congratulations Yang! Best fanrui On Thu, May 5, 2022 at 19:47 Martijn Visser wrote: > Congratulations Yang Wang! > > On Thu, 5 May 2022 at 13:23, Lijie Wang wrote: > > > Congratulations Yang! > > > > Best, > > Lijie > > > > Xintong Song 于2022年5月5日周四 19:18写道: > > > > > Hi all, > > > > > >

Re: [DISCUSS] FLIP-227: Support overdraft buffer

2022-04-29 Thread rui fan
fanrui On Thu, Apr 28, 2022 at 12:39 PM rui fan <1996fan...@gmail.com> wrote: > Hi Anton Kalashnikov, > > Thanks for your very clear reply, I think you are totally right. > > The 'maxBuffersNumber - buffersInUseNumber' can be used as the > overdraft buffer, it w

Re: [ANNOUNCE] New Apache Flink PMC Members - Godfrey He, Xingbo Huang

2022-11-23 Thread Rui Fan
Congratulations, well deserved! Rui Fan On Wed, Nov 23, 2022 at 9:53 PM Konstantin Knauf wrote: > Congrats to both. > > Am Mi., 23. Nov. 2022 um 10:45 Uhr schrieb yu zelin >: > > > Congratulations,Godfrey and Xingbo! > > > > Best, > > Yu Ze

Re: [VOTE] FLIP-271: Autoscaling

2022-11-24 Thread Rui Fan
+1 (non-binding) Best, Rui Fan On Thu, Nov 24, 2022 at 9:54 PM Yang Wang wrote: > +1 (binding) > > > Best, > Yang > > Őrhidi Mátyás 于2022年11月24日周四 12:04写道: > > > +1 (binding) > > > > On Wed, Nov 23, 2022 at 11:46 AM Gyula Fóra > wr

Re: [DISCUSS] FLIP-271: Autoscaling

2022-11-24 Thread Rui Fan
ra/browse/FLINK-29501 Best Rui Fan On Fri, Nov 25, 2022 at 4:54 AM Gyula Fóra wrote: > Hi John! > > Thank you for the excellent question. > > There are few reasons why we felt that the operator is the right place for > this component: > > - Ideally the autoscaler

Re: [DISCUSS] FLIP-271: Autoscaling

2022-11-24 Thread Rui Fan
Hi Gyula Thanks for the clarification! Best Rui Fan On Fri, Nov 25, 2022 at 1:50 PM Gyula Fóra wrote: > Rui, Prasanna: > > I am afraid that creating a completely independent autoscaler process that > works with any type of Flink clusters is out of scope right now due to the

Re: [SUMMARY] Flink 1.17 release sync 23rd of January, 2023

2023-01-26 Thread Rui Fan
[3] http://codespeed.dak8s.net:8000/timeline/#/?exe=1=checkpointSingleInput.UNALIGNED_1=on=on=off=2=200 Best Rui Fan On Thu, Jan 26, 2023 at 8:57 PM Martijn Visser wrote: > Hi Lijie, > > > In summary, I would prefer to extend the "revert regression commits" to > 31

Re: [VOTE] FLIP-285: Refactoring LeaderElection to make Flink support multi-component leader election out-of-the-box

2023-01-30 Thread Rui Fan
+1 non-binding Best Rui Fan On Thu, Jan 26, 2023 at 19:07 Chesnay Schepler wrote: > +1 > > On 25/01/2023 10:33, Matthias Pohl wrote: > > Hi everyone, > > After the discussion thread [1] on FLIP-285 [2] didn't bring up any new > > items, I want to start vo

  1   2   3   4   5   >