[jira] [Created] (FLINK-21688) we use setIncreaseParallelism function, can cause slow flush in restore

2021-03-09 Thread xiaogang zhou (Jira)
xiaogang zhou created FLINK-21688: - Summary: we use setIncreaseParallelism function, can cause slow flush in restore Key: FLINK-21688 URL: https://issues.apache.org/jira/browse/FLINK-21688 Project:

[jira] [Created] (FLINK-21689) UnalignedCheckpointITCase does not terminate

2021-03-09 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-21689: Summary: UnalignedCheckpointITCase does not terminate Key: FLINK-21689 URL: https://issues.apache.org/jira/browse/FLINK-21689 Project: Flink Issue

[jira] [Created] (FLINK-21687) e2e "Run kubernetes session test (default input)" timed out due to k8s issue

2021-03-09 Thread Matthias (Jira)
Matthias created FLINK-21687: Summary: e2e "Run kubernetes session test (default input)" timed out due to k8s issue Key: FLINK-21687 URL: https://issues.apache.org/jira/browse/FLINK-21687 Project: Flink

[DISCUSS] FLIP-154: SQL Implicit Type Coercion

2021-03-09 Thread Danny Chan
Hello, fellows, long time no see ~ Here i want to fire a discussion about the SQL implicit type coercion, a required feature for SQL production but missed for Flink SQL for a long time ~ SQL implicit type coercion is very useful for these cases: - the inter-operation within builtin SQL

Re: Re: [DISCUSSION] Introduce a separated memory pool for the TM merge shuffle

2021-03-09 Thread Xintong Song
It was originally designed that the framework vs. task memory are distinguished by whether the memory owned by the slots. We have not emphasized this in user documents because the dynamic slot allocation is not yet available to the users, thus they do not need to worry about whether the memory is

Re: Re: [DISCUSSION] Introduce a separated memory pool for the TM merge shuffle

2021-03-09 Thread Guowei Ma
Hi, all Thanks all for your suggestions and feedback. I think it is a good idea that we increase the default size of the separated pool by testing. I am fine with adding the suffix(".size") to the config name, which makes it more clear to the user. But I am a little worried about adding a

[jira] [Created] (FLINK-21692) Update the Python SDK to reflect the latest changes in the request-reply protocol

2021-03-09 Thread Igal Shilman (Jira)
Igal Shilman created FLINK-21692: Summary: Update the Python SDK to reflect the latest changes in the request-reply protocol Key: FLINK-21692 URL: https://issues.apache.org/jira/browse/FLINK-21692

Re: QUESTION: Local Dev Setup

2021-03-09 Thread Till Rohrmann
Hi Jan, if you run mvn install of your modified Flink, then you can simply use the maven artefacts from the local repository. You should then either specify --no-snapshot-updates or assign a different maven version to your Flink build which you can reference from your test job. Cheers, Till On

Re: [ANNOUCE][DISCUSS] Roadmap Update for Website

2021-03-09 Thread Till Rohrmann
Thanks for creating the draft for the updated roadmap Stephan. It looks good to me. I also like the feature radar. This will help our users to navigate more safely in the growing Flink ecosystem. Cheers, Till On Mon, Mar 8, 2021 at 8:32 PM Stephan Ewen wrote: > Thanks, Robert! > > As far as I

[jira] [Created] (FLINK-21690) remove redundant tolerableCheckpointFailureNumber setting in CheckpointConfig

2021-03-09 Thread Zhengqi Zhang (Jira)
Zhengqi Zhang created FLINK-21690: - Summary: remove redundant tolerableCheckpointFailureNumber setting in CheckpointConfig Key: FLINK-21690 URL: https://issues.apache.org/jira/browse/FLINK-21690

[jira] [Created] (FLINK-21691) KafkaSource fails with NPE when setting it up

2021-03-09 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21691: - Summary: KafkaSource fails with NPE when setting it up Key: FLINK-21691 URL: https://issues.apache.org/jira/browse/FLINK-21691 Project: Flink Issue Type:

[jira] [Created] (FLINK-21693) TestStreamEnvironment does not implement executeAsync

2021-03-09 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-21693: - Summary: TestStreamEnvironment does not implement executeAsync Key: FLINK-21693 URL: https://issues.apache.org/jira/browse/FLINK-21693 Project: Flink

[jira] [Created] (FLINK-21694) Increase default value of "state.backend.rocksdb.checkpoint.transfer.thread.num"

2021-03-09 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-21694: Summary: Increase default value of "state.backend.rocksdb.checkpoint.transfer.thread.num" Key: FLINK-21694 URL: https://issues.apache.org/jira/browse/FLINK-21694

Re: [ANNOUCE][DISCUSS] Roadmap Update for Website

2021-03-09 Thread Robert Metzger
@Stephan: Indeed, we shouldn't add unreleased features there. +1 to update after each release. On Tue, Mar 9, 2021 at 3:05 PM Till Rohrmann wrote: > Thanks for creating the draft for the updated roadmap Stephan. It looks > good to me. I also like the feature radar. This will help our users to >

[jira] [Created] (FLINK-21695) Increase default value for number of KeyGroups

2021-03-09 Thread Stephan Ewen (Jira)
Stephan Ewen created FLINK-21695: Summary: Increase default value for number of KeyGroups Key: FLINK-21695 URL: https://issues.apache.org/jira/browse/FLINK-21695 Project: Flink Issue Type:

[jira] [Created] (FLINK-21697) Support StreamExecMiniBatchAssigner json serialization/deserialization

2021-03-09 Thread godfrey he (Jira)
godfrey he created FLINK-21697: -- Summary: Support StreamExecMiniBatchAssigner json serialization/deserialization Key: FLINK-21697 URL: https://issues.apache.org/jira/browse/FLINK-21697 Project: Flink

Re: [DISCUSS] Flink-RocketMQ connector

2021-03-09 Thread Nicholas Jiang
Hi Heng Du, Thanks for starting the discussion. >From the user's perspective, the integration between RocketMQ and Flink is user-friendly. I agree with you the point that polish the existing implementation and add new features, such as table connector support. But before that, you need to

Re: [DISCUSS] FLIP-162 follow-up discussion

2021-03-09 Thread Leonard Xu
Thank you all for your positive feedbacks. Considering that you’re binding votes for this FLIP and all of us agreed to remove this option, Thus I update this section of FLIP-162[1], I’ll start work on it next . Best, Leonard [1]

Re: [DISCUSS] Flink-RocketMQ connector

2021-03-09 Thread heng du
Dear Flink community, Thanks for @Nicholas's reply and bridge, I am a PMC member of RocketMQ, and developers in the RocketMQ community have been concerned about integration with Flink for a long time, and sincerely hope developers from both communities will work closely together to make this

[DISCUSS] Flink-RocketMQ connector

2021-03-09 Thread heng du
Apache RocketMQ[1], as a cloud-native messaging and streaming platform, has a large number of users [2] all over the world, especially in China. Recently, more and more users have begun to use RocketMQ to integrate with Flink, which has resulted in a large number of issues[3] and PRs[4]. So,

[Statefun] Interaction Protocol for Statefun

2021-03-09 Thread Guillaume Vauvert
Hi Statefun folk, Now that we have a powerful framework to manage stateful functions, a part of the algorithmic complexity has moved on the interaction between functions side. But implementing an interaction protocol is complex and error prone, so that could be great to provide an

[jira] [Created] (FLINK-21696) scala code in "Writing WatermarkGenerators" documentation missing/deprecated

2021-03-09 Thread Giacomo Gamba (Jira)
Giacomo Gamba created FLINK-21696: - Summary: scala code in "Writing WatermarkGenerators" documentation missing/deprecated Key: FLINK-21696 URL: https://issues.apache.org/jira/browse/FLINK-21696

[jira] [Created] (FLINK-21704) Support StreamExecChangelogNormalize json serde

2021-03-09 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-21704: --- Summary: Support StreamExecChangelogNormalize json serde Key: FLINK-21704 URL: https://issues.apache.org/jira/browse/FLINK-21704 Project: Flink Issue Type:

[jira] [Created] (FLINK-21705) Add a SliceType to the StateFun Java SDK

2021-03-09 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-21705: --- Summary: Add a SliceType to the StateFun Java SDK Key: FLINK-21705 URL: https://issues.apache.org/jira/browse/FLINK-21705 Project: Flink Issue

[jira] [Created] (FLINK-21699) SecurityUtils JaasModule make JVM dirty

2021-03-09 Thread tonychan (Jira)
tonychan created FLINK-21699: Summary: SecurityUtils JaasModule make JVM dirty Key: FLINK-21699 URL: https://issues.apache.org/jira/browse/FLINK-21699 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-21702) Support option `sql-client.verbos` to print the exception stack

2021-03-09 Thread Shengkai Fang (Jira)
Shengkai Fang created FLINK-21702: - Summary: Support option `sql-client.verbos` to print the exception stack Key: FLINK-21702 URL: https://issues.apache.org/jira/browse/FLINK-21702 Project: Flink

[jira] [Created] (FLINK-21703) Add option `table.dml-sync`

2021-03-09 Thread Shengkai Fang (Jira)
Shengkai Fang created FLINK-21703: - Summary: Add option `table.dml-sync` Key: FLINK-21703 URL: https://issues.apache.org/jira/browse/FLINK-21703 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-21698) Disable problematic cast conversion between NUMERIC type and TIMESTAMP type

2021-03-09 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-21698: -- Summary: Disable problematic cast conversion between NUMERIC type and TIMESTAMP type Key: FLINK-21698 URL: https://issues.apache.org/jira/browse/FLINK-21698 Project:

[jira] [Created] (FLINK-21700) Allow to disable fetching Hadoop delegation token on Yarn

2021-03-09 Thread Junfan Zhang (Jira)
Junfan Zhang created FLINK-21700: Summary: Allow to disable fetching Hadoop delegation token on Yarn Key: FLINK-21700 URL: https://issues.apache.org/jira/browse/FLINK-21700 Project: Flink

[jira] [Created] (FLINK-21701) Extend the `RESET` command in the sql client

2021-03-09 Thread Shengkai Fang (Jira)
Shengkai Fang created FLINK-21701: - Summary: Extend the `RESET` command in the sql client Key: FLINK-21701 URL: https://issues.apache.org/jira/browse/FLINK-21701 Project: Flink Issue Type:

[jira] [Created] (FLINK-21706) FlinkKafkaProducerITCase.testMigrateFromAtExactlyOnceToAtLeastOnce fails with "SchemaException: Error reading field 'api_keys': Error reading field 'api_key': java.nio.B

2021-03-09 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-21706: Summary: FlinkKafkaProducerITCase.testMigrateFromAtExactlyOnceToAtLeastOnce fails with "SchemaException: Error reading field 'api_keys': Error reading field 'api_key': java.nio.BufferUnderflowException"

Re: [ANNOUCE][DISCUSS] Roadmap Update for Website

2021-03-09 Thread Dawid Wysakowicz
Thank you Stephan for working on this. It's really nice to see the roadmap updated. I also find the "Feature Radar" extremely helpful. It looks good to be published from my side. Best, Dawid On 02/03/2021 15:38, Stephan Ewen wrote: > Hi all! > > The roadmap on the Flink website is quite

[jira] [Created] (FLINK-21707) Job is possible to hang when restarting a FINISHED task with POINTWISE BLOCKING consumers

2021-03-09 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-21707: --- Summary: Job is possible to hang when restarting a FINISHED task with POINTWISE BLOCKING consumers Key: FLINK-21707 URL: https://issues.apache.org/jira/browse/FLINK-21707

[jira] [Created] (FLINK-21686) Duplicate code in hive parser file should be abstracted into functions

2021-03-09 Thread humengyu (Jira)
humengyu created FLINK-21686: Summary: Duplicate code in hive parser file should be abstracted into functions Key: FLINK-21686 URL: https://issues.apache.org/jira/browse/FLINK-21686 Project: Flink

Re: [DISCUSS] Apache Flink Jira Process

2021-03-09 Thread Leonard Xu
Thanks Konstantin for driving this topic. Generally +1 for the proposal, I went through the doc and have two concerns here. Will the robot send all notifications to assignee/reporter/watchers ? I’m a little worried about too many push messages. Eg, I watched some issues that I want to

Re: [DISCUSS] FLIP-162 follow-up discussion

2021-03-09 Thread Jark Wu
Thanks Leonard, I'm also +1 with not introducing this fallback option. It's error-prone to mix the implementation of wrong behavior and correct behavior. And it's better to educate users the right way in one version instead of spanning multiple versions. Best, Jark On Tue, 9 Mar 2021 at 15:15,

Re: [DISCUSS] FLIP-162 follow-up discussion

2021-03-09 Thread Timo Walther
Hi Leonard, I'm fine with dropping the old buggy behavior immediatly. Users can still implement a UDF with the old bavhior if needed. I hope the new functions will be well-tested so that a fallback to the old functions is not necessary as a workaround. It will definitely avoid confusion for

Re: [DISCUSS] FLIP-162 follow-up discussion

2021-03-09 Thread Jingsong Li
+1 Let's go straight to the right behavior. Drop the option for the wrong behavior. Best, Jingsong On Tue, Mar 9, 2021 at 4:29 PM Timo Walther wrote: > Hi Leonard, > > I'm fine with dropping the old buggy behavior immediatly. Users can > still implement a UDF with the old bavhior if needed.