[jira] [Created] (FLINK-21104) UnalignedCheckpointITCase.execute failed with "IllegalStateException"

2021-01-22 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-21104: Summary: UnalignedCheckpointITCase.execute failed with "IllegalStateException" Key: FLINK-21104 URL: https://issues.apache.org/jira/browse/FLINK-21104 Project: Flink

I want to contribute to Apache Flink

2021-01-22 Thread nicygan
Hi Guys, I want to contribute to Apache Flink. Would you please give me the permission as a contributor? My JIRA ID is nicygan. | | nicygan | | read3...@163.com | 签名由网易邮箱大师定制

Re: [DISCUSS] FLIP-160: Declarative scheduler

2021-01-22 Thread Steven Wu
Till, thanks a lot for the proposal. Even if the initial phase is only to support scale-up, maybe the "ScaleUpController" interface should be called "RescaleController" so that in the future scale-down can be added. On Fri, Jan 22, 2021 at 7:03 AM Till Rohrmann wrote: > Hi everyone, > > I

Re: [DISCUSS] FLIP-159: Reactive Mode

2021-01-22 Thread Steven Wu
Thanks a lot for the proposal, Robert and Till. > No fixed parallelism for any of the operators Regarding this limitation, can the scheduler only adjust the default parallelism? if some operators set parallelism explicitly (like always 1), just leave them unchanged. On Fri, Jan 22, 2021 at

[jira] [Created] (FLINK-21103) E2e tests time out on azure

2021-01-22 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21103: Summary: E2e tests time out on azure Key: FLINK-21103 URL: https://issues.apache.org/jira/browse/FLINK-21103 Project: Flink Issue Type: Bug

[DISCUSS] FLIP-159: Reactive Mode

2021-01-22 Thread Robert Metzger
Hi all, Till started a discussion about FLIP-160: Declarative scheduler [1] earlier today, the first major feature based on that effort will be FLIP-159: Reactive Mode. It allows users to operate Flink in a way that it reactively scales the job up or down depending on the provided resources:

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-22 Thread Chesnay Schepler
The FLIP seems to disregard the existence of dynamic properties, and I'm wondering why that is the case. Don't they fulfill similar roles, in that they allow config options to be set on the command-line? What use-case do they currently not support? I assume it's something along the lines of

[jira] [Created] (FLINK-21102) Add ScaleUpController

2021-01-22 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21102: - Summary: Add ScaleUpController Key: FLINK-21102 URL: https://issues.apache.org/jira/browse/FLINK-21102 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-21098) Add SlotAllocator

2021-01-22 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21098: - Summary: Add SlotAllocator Key: FLINK-21098 URL: https://issues.apache.org/jira/browse/FLINK-21098 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-21101) Set up cron job to run CI with declarative scheduler

2021-01-22 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21101: - Summary: Set up cron job to run CI with declarative scheduler Key: FLINK-21101 URL: https://issues.apache.org/jira/browse/FLINK-21101 Project: Flink Issue

[jira] [Created] (FLINK-21100) Add DeclarativeScheduler skeleton

2021-01-22 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21100: - Summary: Add DeclarativeScheduler skeleton Key: FLINK-21100 URL: https://issues.apache.org/jira/browse/FLINK-21100 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-21099) Introduce JobType to distinguish between batch and streaming jobs

2021-01-22 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21099: - Summary: Introduce JobType to distinguish between batch and streaming jobs Key: FLINK-21099 URL: https://issues.apache.org/jira/browse/FLINK-21099 Project: Flink

[DISCUSS] FLIP-160: Declarative scheduler

2021-01-22 Thread Till Rohrmann
Hi everyone, I would like to start a discussion about adding a new type of scheduler to Flink. The declarative scheduler will first declare the required resources and wait for them before deciding on the actual parallelism of a job. Thereby it can better handle situations where resources cannot

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-22 Thread Till Rohrmann
The updated design LGTM as well. Nice work Ingo! Cheers, Till On Fri, Jan 22, 2021 at 3:33 PM Ingo Bürk wrote: > Thanks, Ufuk. I think that makes sense, so I moved it from a footnote to an > addition to prevent that in the future as well. > > Ingo > > On Fri, Jan 22, 2021 at 3:10 PM Ufuk

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-22 Thread Ingo Bürk
Thanks, Ufuk. I think that makes sense, so I moved it from a footnote to an addition to prevent that in the future as well. Ingo On Fri, Jan 22, 2021 at 3:10 PM Ufuk Celebi wrote: > LGTM. Let's see what the others think... > > On Thu, Jan 21, 2021, at 11:37 AM, Ingo Bürk wrote: > > Regarding

Re: [DISCUSS] FLIP-161: Configuration through environment variables

2021-01-22 Thread Ufuk Celebi
LGTM. Let's see what the others think... On Thu, Jan 21, 2021, at 11:37 AM, Ingo Bürk wrote: > Regarding env.java.opts, what special handling is needed there? AFAICT only > the rejected alternative of substituting values would've had an effect on > this. Makes sense  >From the FLIP: > This

[jira] [Created] (FLINK-21097) Introduce e2e test framework for json plan test

2021-01-22 Thread godfrey he (Jira)
godfrey he created FLINK-21097: -- Summary: Introduce e2e test framework for json plan test Key: FLINK-21097 URL: https://issues.apache.org/jira/browse/FLINK-21097 Project: Flink Issue Type:

Re: [ANNOUNCE] 1.12.1 may still produce corrupted checkpoints

2021-01-22 Thread Arvid Heise
Hi Till, I completely agree with you. Best, Arvid On Fri, Jan 22, 2021 at 1:46 PM Till Rohrmann wrote: > Thanks for the update Arvid. This fix warrants a quick 1.12.2 release imo. > > Cheers, > Till > > On Fri, Jan 22, 2021 at 1:42 PM Arvid Heise wrote: > > > Hi Xintong, > > > > yes, I'm on

Re: [ANNOUNCE] 1.12.1 may still produce corrupted checkpoints

2021-01-22 Thread Till Rohrmann
Thanks for the update Arvid. This fix warrants a quick 1.12.2 release imo. Cheers, Till On Fri, Jan 22, 2021 at 1:42 PM Arvid Heise wrote: > Hi Xintong, > > yes, I'm on it. > > Best, > > Arvid > > On Fri, Jan 22, 2021 at 1:01 PM Xintong Song > wrote: > > > Hi Arvid, > > > > Thanks for the

Re: [ANNOUNCE] 1.12.1 may still produce corrupted checkpoints

2021-01-22 Thread Arvid Heise
Hi Xintong, yes, I'm on it. Best, Arvid On Fri, Jan 22, 2021 at 1:01 PM Xintong Song wrote: > Hi Arvid, > > Thanks for the announcement. > > I think we'd better also update the 1.12 release notes[1] and 1.12.1 > release blog post[2]. > Would you have time to help prepare the warning

Re: [ANNOUNCE] 1.12.1 may still produce corrupted checkpoints

2021-01-22 Thread Xintong Song
Hi Arvid, Thanks for the announcement. I think we'd better also update the 1.12 release notes[1] and 1.12.1 release blog post[2]. Would you have time to help prepare the warning messages? Thank you~ Xintong Song [1] https://github.com/apache/flink/blob/master/docs/release-notes/flink-1.12.md

Re:Re: [ANNOUNCE] Welcome Guowei Ma as a new Apache Flink Committer

2021-01-22 Thread pez1...@163.com
Congrats Guowei! 在 2021-01-22 16:48:19,"zhang hao" 写道: >Congrats Guowei! > >On Fri, Jan 22, 2021 at 10:40 AM Zhijiang > wrote: > >> Congrats, Guowei! >> >> >> Best, >> Zhijiang >> >> >> -- >> From:Biao Liu >> Send

[ANNOUNCE] 1.12.1 may still produce corrupted checkpoints

2021-01-22 Thread Arvid Heise
Dear users, Unfortunately, the bug in the unaligned checkpoint that we fixed in 1.12.1 still occurs under certain circumstances, such that we recommend to not use unaligned checkpoints in production until 1.12.2. While the normal processing is not affected by this bug, a recovery with corrupted

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-01-22 Thread Timo Walther
Hi everyone, let me answer the individual threads: >>> I know that the two series should be different at first glance, but >>> different SQL engines can have their own explanations,for example, >>> CURRENT_TIMESTAMP and LOCALTIMESTAMP are synonyms in Snowflake[1] and has >>> no difference, and

[jira] [Created] (FLINK-21096) Introduce ExecNodeGraphJsonPlanGenerator to serialize ExecNodeGraph to json plan and deserialize json plan to ExecNodeGraph

2021-01-22 Thread godfrey he (Jira)
godfrey he created FLINK-21096: -- Summary: Introduce ExecNodeGraphJsonPlanGenerator to serialize ExecNodeGraph to json plan and deserialize json plan to ExecNodeGraph Key: FLINK-21096 URL:

[jira] [Created] (FLINK-21095) Remove legacy slotmanagement profile

2021-01-22 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-21095: Summary: Remove legacy slotmanagement profile Key: FLINK-21095 URL: https://issues.apache.org/jira/browse/FLINK-21095 Project: Flink Issue Type:

Re: [ANNOUNCE] Welcome Guowei Ma as a new Apache Flink Committer

2021-01-22 Thread zhang hao
Congrats Guowei! On Fri, Jan 22, 2021 at 10:40 AM Zhijiang wrote: > Congrats, Guowei! > > > Best, > Zhijiang > > > -- > From:Biao Liu > Send Time:2021年1月21日(星期四) 14:45 > To:dev > Subject:Re: [ANNOUNCE] Welcome Guowei Ma as a new

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-01-22 Thread Kurt Young
Forgot one more thing. Continue with displaying in UTC. As a user, if Flink want to display the timestamp in UTC, why don't we offer something like UTC_TIMESTAMP? Best, Kurt On Fri, Jan 22, 2021 at 4:33 PM Kurt Young wrote: > Before jumping into technique details, let's take a step back to

Re: [DISCUSS] FLIP-162: Consistent Flink SQL time function behavior

2021-01-22 Thread Kurt Young
Before jumping into technique details, let's take a step back to discuss user experience. The first important question is what kind of date and time will Flink display when users call CURRENT_TIMESTAMP and maybe also PROCTIME (if we think they are similar). Should it always display the date and

[jira] [Created] (FLINK-21094) Support StreamExecSink json serialization/deserialization

2021-01-22 Thread godfrey he (Jira)
godfrey he created FLINK-21094: -- Summary: Support StreamExecSink json serialization/deserialization Key: FLINK-21094 URL: https://issues.apache.org/jira/browse/FLINK-21094 Project: Flink Issue