[jira] [Created] (FLINK-21332) Optimize releasing result partitions in RegionPartitionReleaseStrategy

2021-02-08 Thread Zhilong Hong (Jira)
Zhilong Hong created FLINK-21332: Summary: Optimize releasing result partitions in RegionPartitionReleaseStrategy Key: FLINK-21332 URL: https://issues.apache.org/jira/browse/FLINK-21332 Project:

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Shengkai Fang
Hi, all. I think it may cause user confused. The main problem is we have no means to detect the conflict configuration, e.g. users set the option true and use `TableResult#await` together. Best, Shengkai.

[jira] [Created] (FLINK-21331) Optimize calculating tasks to restart in RestartPipelinedRegionFailoverStrategy

2021-02-08 Thread Zhilong Hong (Jira)
Zhilong Hong created FLINK-21331: Summary: Optimize calculating tasks to restart in RestartPipelinedRegionFailoverStrategy Key: FLINK-21331 URL: https://issues.apache.org/jira/browse/FLINK-21331

[jira] [Created] (FLINK-21330) Optimization the initialization of PipelinedRegionSchedulingStrategy

2021-02-08 Thread Zhilong Hong (Jira)
Zhilong Hong created FLINK-21330: Summary: Optimization the initialization of PipelinedRegionSchedulingStrategy Key: FLINK-21330 URL: https://issues.apache.org/jira/browse/FLINK-21330 Project: Flink

[jira] [Created] (FLINK-21329) "Local recovery and sticky scheduling end-to-end test" does not finish within 600 seconds

2021-02-08 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-21329: -- Summary: "Local recovery and sticky scheduling end-to-end test" does not finish within 600 seconds Key: FLINK-21329 URL: https://issues.apache.org/jira/browse/FLINK-21329

[jira] [Created] (FLINK-21328) Optimize the initialization of DefaultExecutionTopology

2021-02-08 Thread Zhilong Hong (Jira)
Zhilong Hong created FLINK-21328: Summary: Optimize the initialization of DefaultExecutionTopology Key: FLINK-21328 URL: https://issues.apache.org/jira/browse/FLINK-21328 Project: Flink

[jira] [Created] (FLINK-21327) Support window TVF in batch mode

2021-02-08 Thread Jark Wu (Jira)
Jark Wu created FLINK-21327: --- Summary: Support window TVF in batch mode Key: FLINK-21327 URL: https://issues.apache.org/jira/browse/FLINK-21327 Project: Flink Issue Type: Sub-task

Re: Activate bloom filter in RocksDB State Backend via Flink configuration

2021-02-08 Thread Yun Tang
Hi Jun, Some predefined options would also activate bloom filters, e.g. PredefinedOptions#SPINNING_DISK_OPTIMIZED_HIGH_MEM, but I think offering configurable option is good idea. +1 for this. When talking about the bloom filter default value, I slight prefer to use full format [1] instead of

[jira] [Created] (FLINK-21326) Optimize building topology when initializing ExecutionGraph

2021-02-08 Thread Zhilong Hong (Jira)
Zhilong Hong created FLINK-21326: Summary: Optimize building topology when initializing ExecutionGraph Key: FLINK-21326 URL: https://issues.apache.org/jira/browse/FLINK-21326 Project: Flink

[jira] [Created] (FLINK-21325) NoResourceAvailableException while cancel then resubmit jobs

2021-02-08 Thread hayden zhou (Jira)
hayden zhou created FLINK-21325: --- Summary: NoResourceAvailableException while cancel then resubmit jobs Key: FLINK-21325 URL: https://issues.apache.org/jira/browse/FLINK-21325 Project: Flink

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Rui Li
Hi Jark, I agree it's more consistent if table API also respects this config. But on the other hand, it might make the `executeSql` API a little trickier to use, because now DDL, DQL and DML all behave differently from one another: - DDL: always sync - DQL: always async - DML: can be

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Jark Wu
Hi Rui, That's a good point. From the naming of the option, I prefer to get sync behavior. It would be very straightforward that it affects all the DMLs on SQL CLI and TableEnvironment (including `executeSql`, `StatementSet`, `Table#executeInsert`, etc.). This can also make SQL CLI easy to

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Rui Li
Hi, Glad to see we have reached consensus on option #2. +1 to it. Regarding the name, I'm fine with `table.dml-async`. But I wonder whether this config also applies to table API. E.g. if a user sets table.dml-async=false and calls TableEnvironment::executeSql to run a DML, will he get sync

[jira] [Created] (FLINK-21324) statefun-testutil can't assert messages function sends to itself

2021-02-08 Thread Timur (Jira)
Timur created FLINK-21324: - Summary: statefun-testutil can't assert messages function sends to itself Key: FLINK-21324 URL: https://issues.apache.org/jira/browse/FLINK-21324 Project: Flink Issue

Re: [DISCUSS] Releasing Apache Flink 1.12.2

2021-02-08 Thread Matthias Pohl
Fabian and I were investigating strange behavior with stop-with-savepoint not terminating when using the new fromSource to add a source to a job definition. I created FLINK-21323 [1] to cover the issue. This might not be a blocker for 1.12.2 since this bug would have been already around since

[jira] [Created] (FLINK-21323) Stop-with-savepoint is not supported by SourceOperatorStreamTask

2021-02-08 Thread Matthias (Jira)
Matthias created FLINK-21323: Summary: Stop-with-savepoint is not supported by SourceOperatorStreamTask Key: FLINK-21323 URL: https://issues.apache.org/jira/browse/FLINK-21323 Project: Flink

Re: [DISCUSS] Releasing Apache Flink 1.12.2

2021-02-08 Thread Thomas Weise
+1 for the 1.12.2 release On Mon, Feb 8, 2021 at 3:20 AM Matthias Pohl wrote: > Thanks Yuan for driving 1.12.2. > +1 for releasing 1.12.2 > > One comment about FLINK-21030 [1]: I hope to fix it this week. But there > are still some uncertainties. The underlying problem is older than 1.12. >

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Jark Wu
Ah, I just forgot the option name. I'm also fine with `table.dml-async`. What do you think @Rui Li @Shengkai Fang ? Best, Jark On Mon, 8 Feb 2021 at 23:06, Timo Walther wrote: > Great to hear that. Can someone update the FLIP a final time before we > start a vote? > > We should quickly

[jira] [Created] (FLINK-21322) Add ExecutionGraphHandler

2021-02-08 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-21322: Summary: Add ExecutionGraphHandler Key: FLINK-21322 URL: https://issues.apache.org/jira/browse/FLINK-21322 Project: Flink Issue Type: Improvement

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Timo Walther
Great to hear that. Can someone update the FLIP a final time before we start a vote? We should quickly discuss how we would like to name the config option for the async/sync mode. I heared voices internally that are strongly against calling it "detach" due to historical reasons with a Flink

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Jark Wu
Thanks Timo, I'm +1 for option#2 too. I think we have addressed all the concerns and can start a vote. Best, Jark On Mon, 8 Feb 2021 at 22:19, Timo Walther wrote: > Hi Jark, > > you are right. Nesting STATEMENT SET and ASYNC might be too verbose. > > So let's stick to the config option

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Timo Walther
Hi Jark, you are right. Nesting STATEMENT SET and ASYNC might be too verbose. So let's stick to the config option approach. However, I strongly believe that we should not use the batch/streaming mode for deriving semantics. This discussion is similar to time function discussion. We should

[jira] [Created] (FLINK-21321) Change RocksDB incremental checkpoint re-scaling to use deleteRange

2021-02-08 Thread Joey Pereira (Jira)
Joey Pereira created FLINK-21321: Summary: Change RocksDB incremental checkpoint re-scaling to use deleteRange Key: FLINK-21321 URL: https://issues.apache.org/jira/browse/FLINK-21321 Project: Flink

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Jark Wu
Hi Timo, Actually, I'm not in favor of explicit syntax `BEGIN ASYNC;... END;`. Because it makes submitting streaming jobs very verbose, every INSERT INTO and STATEMENT SET must be wrapped in the ASYNC clause which is not user-friendly and not backward-compatible. I agree we will have unified

Re: [DISCUSS] FLIP-164: Improve Schema Handling in Catalogs

2021-02-08 Thread Dawid Wysakowicz
Hi Timo, From my perspective the proposed changes look good. I agree it is an important step towards FLIP-129 and FLIP-136. Personally I feel comfortable voting on the document. Best, Dawid On 05/02/2021 16:09, Timo Walther wrote: > Hi everyone, > > you might have seen that we discussed a

Re: [DISCUSS] Releasing Apache Flink 1.12.2

2021-02-08 Thread Matthias Pohl
Thanks Yuan for driving 1.12.2. +1 for releasing 1.12.2 One comment about FLINK-21030 [1]: I hope to fix it this week. But there are still some uncertainties. The underlying problem is older than 1.12. Hence, the suggestion is to not block the 1.12.2 release because of FLINK-21030 [1]. I will

[jira] [Created] (FLINK-21320) Stream from FlinkKafkaShuffle.persistentKeyBy does not end if upstream ends

2021-02-08 Thread Kezhu Wang (Jira)
Kezhu Wang created FLINK-21320: -- Summary: Stream from FlinkKafkaShuffle.persistentKeyBy does not end if upstream ends Key: FLINK-21320 URL: https://issues.apache.org/jira/browse/FLINK-21320 Project:

Re: Flink 1.11.4 release? And usual plan for maintenance etc

2021-02-08 Thread Chesnay Schepler
The 1.13 feature freeze is scheduled for the end of march, and the release can be expected to happen between 2-4 weeks after that. On 2/8/2021 11:21 AM, Adam Roberts wrote: Hey Till, all - thanks for the information, very useful to know. So the JIRAs in particular are for upgrading the

Re: Flink 1.11.4 release? And usual plan for maintenance etc

2021-02-08 Thread Adam Roberts
Hey Till, all - thanks for the information, very useful to know. So the JIRAs in particular are for upgrading the version of Jackson https://issues.apache.org/jira/browse/FLINK-21020, and there's another for Netty as well https://issues.apache.org/jira/browse/FLINK-21019 but admittedly I'm not

Re: [DISCUSS]FLIP-163: SQL Client Improvements

2021-02-08 Thread Timo Walther
Hi Jark, Hi Rui, 1) How should we execute statements in CLI and in file? Should there be a difference? So it seems we have consensus here with unified bahavior. Even though this means we are breaking existing batch INSERT INTOs that were asynchronous before. 2) Should we have different

Re: Activate bloom filter in RocksDB State Backend via Flink configuration

2021-02-08 Thread Till Rohrmann
Hi Jun, Making things easier to use and configure is a good idea. Hence, +1 for this proposal. Maybe create a JIRA ticket for it. For the concrete default values it would be nice to hear the opinion of a RocksDB expert. Cheers, Till On Sun, Feb 7, 2021 at 7:23 PM Jun Qin wrote: > Hi, > >

Re: Flink upset-kaka connector not working with Avro confluent

2021-02-08 Thread Till Rohrmann
Hi Shamit, thanks for reaching out to the community. I am pulling in Timo who might know more about this problem. Cheers, Till On Sun, Feb 7, 2021 at 6:22 AM shamit jain wrote: > Hello Team, > > I am facing issue with "upsert-kafka" connector which should read the Avro > message serialized