[DISCUSS] FLIP-186: Refactor DataStreamUtils#reinterpretAsKeyedStream

2021-08-05 Thread Senhong Liu
Dear devs, I would like to start a discussion for FLIP-186: Refactor DataStreamUtils#reinterpretAsKeyedStream[1]. As we know, this API has a limitation that the partitioning of the base stream must be exactly the same as Flink's internal partitioning[2]. Currently, Flink would first select the k

Re: [DISCUSS]FLIP-170 Adding Checkpoint Rejection Mechanism

2021-07-13 Thread Senhong Liu
t; > Regarding the hybrid batch/streaming execution: The nice part about this > would be that no checkpoints are necessary during the batch phase. > > On Tue, Jul 13, 2021 at 5:51 AM Senhong Liu wrote: > > > Hi Stephan, > > > > Thank you so much for replying and su

Re: [DISCUSS]FLIP-170 Adding Checkpoint Rejection Mechanism

2021-07-12 Thread Senhong Liu
ent ways to create this behavior: Empty > checkpoints, Hybrid Source, etc. > - If no other way to implement this is possible, go with a > RejectionException as a temporary workaround. > - Invest into hybrid batch/streaming execution as the proper long-term &

Re: [DISCUSS]FLIP-170 Adding Checkpoint Rejection Mechanism

2021-06-09 Thread Senhong Liu
Piotrek > > [1] > > https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/streaming/api/environment/CheckpointConfig.html#setTolerableCheckpointFailureNumber-int- > > śr., 9 cze 2021 o 09:11 Senhong Liu napisał(a): > > > Here is some brief context

Re: [DISCUSS]FLIP-170 Adding Checkpoint Rejection Mechanism

2021-06-09 Thread Senhong Liu
Here is some brief context about the new feature. 1. Actively checkpoint rejecting by the operator. Follow by the current checkpoint mechanism, one more preliminary step is added to help the operator determine that if it is able to take snapshots. The preliminary step is a new API provided to the

[DISCUSS]FLIP-170 Adding Checkpoint Rejection Mechanism

2021-06-08 Thread Senhong Liu
Hi guys, We would like to start a discussion on the new FLIP about rejecting checkpoints on the operator level. The basic idea is to allow the operator to reject a checkpoint when it is not under a proper situation and returning a proper failure reason. http://cwiki.apache.org/confluence/display/

Apply for permission to edit wiki

2021-06-07 Thread Senhong Liu
Hello, My username is Senhong Liu (senhong...@gmail.com) and I want to apply for permission to propose a FLIP. Anyone who can help me? THX! Best, Senhong

[jira] [Created] (FLINK-22364) Translate the page of "Data Sources" to Chinese.

2021-04-19 Thread Senhong Liu (Jira)
Senhong Liu created FLINK-22364: --- Summary: Translate the page of "Data Sources" to Chinese. Key: FLINK-22364 URL: https://issues.apache.org/jira/browse/FLINK-22364 Project: Flink