[jira] [Created] (FLINK-22488) KafkaSourceLegacyITCase.testOneToOneSources failed due to "OperatorEvent from an OperatorCoordinator to a task was lost"

2021-04-26 Thread Dong Lin (Jira)
Dong Lin created FLINK-22488: Summary: KafkaSourceLegacyITCase.testOneToOneSources failed due to "OperatorEvent from an OperatorCoordinator to a task was lost" Key: FLINK-22488 URL:

[jira] [Created] (FLINK-22487) Support `print` to print logs in PyFlink

2021-04-26 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-22487: Summary: Support `print` to print logs in PyFlink Key: FLINK-22487 URL: https://issues.apache.org/jira/browse/FLINK-22487 Project: Flink Issue Type: New

[jira] [Created] (FLINK-22486) Wrong results of the IN operator

2021-04-26 Thread Shengkai Fang (Jira)
Shengkai Fang created FLINK-22486: - Summary: Wrong results of the IN operator Key: FLINK-22486 URL: https://issues.apache.org/jira/browse/FLINK-22486 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-22485) Support client attach on application mode

2021-04-26 Thread Junfan Zhang (Jira)
Junfan Zhang created FLINK-22485: Summary: Support client attach on application mode Key: FLINK-22485 URL: https://issues.apache.org/jira/browse/FLINK-22485 Project: Flink Issue Type:

Re: [DISCUSS] Using timeouts in JUnit tests

2021-04-26 Thread Robert Metzger
I was actually recently wondering if we shouldn't rather use timeouts more aggressively in JUnit. There was recently a case where a number of tests accidentally ran for 5 minutes, because a timeout was increased to 5 minutes. If we had a global limit of 1 minute per test, we would have caught this

Re: Task Local Recovery with mountable disks in the cloud

2021-04-26 Thread Till Rohrmann
Hi Sonam, sorry for the late reply. We were a bit caught in the midst of the feature freeze for the next major Flink release. In general, I think it is a very good idea to disaggregate the local state storage to make it reusable across TaskManager failures. However, it is also not trivial to do.

[jira] [Created] (FLINK-22484) Buit-in functions for collections

2021-04-26 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-22484: --- Summary: Buit-in functions for collections Key: FLINK-22484 URL: https://issues.apache.org/jira/browse/FLINK-22484 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-22483) Recover checkpoints when JobMaster gains leadership

2021-04-26 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-22483: -- Summary: Recover checkpoints when JobMaster gains leadership Key: FLINK-22483 URL: https://issues.apache.org/jira/browse/FLINK-22483 Project: Flink

[jira] [Created] (FLINK-22482) ParquetFileSystemITCase fails on Azure

2021-04-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22482: Summary: ParquetFileSystemITCase fails on Azure Key: FLINK-22482 URL: https://issues.apache.org/jira/browse/FLINK-22482 Project: Flink Issue Type:

[jira] [Created] (FLINK-22481) RocksDBStateBackendWindowITCase times out on Azure

2021-04-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22481: Summary: RocksDBStateBackendWindowITCase times out on Azure Key: FLINK-22481 URL: https://issues.apache.org/jira/browse/FLINK-22481 Project: Flink

[jira] [Created] (FLINK-22480) KafkaChangelogTableITCase hangs on Azure

2021-04-26 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22480: Summary: KafkaChangelogTableITCase hangs on Azure Key: FLINK-22480 URL: https://issues.apache.org/jira/browse/FLINK-22480 Project: Flink Issue Type:

[jira] [Created] (FLINK-22478) confluent schema registry avro (de)serialization https support

2021-04-26 Thread Lars Bachmann (Jira)
Lars Bachmann created FLINK-22478: - Summary: confluent schema registry avro (de)serialization https support Key: FLINK-22478 URL: https://issues.apache.org/jira/browse/FLINK-22478 Project: Flink

[jira] [Created] (FLINK-22477) Remove DefaultSlotPoolServiceFactory and SlotPoolImpl

2021-04-26 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-22477: - Summary: Remove DefaultSlotPoolServiceFactory and SlotPoolImpl Key: FLINK-22477 URL: https://issues.apache.org/jira/browse/FLINK-22477 Project: Flink

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

2021-04-26 Thread Tony Wei
Hi Till, I have created the ticket to extend the description of `execution.targe`. https://issues.apache.org/jira/browse/FLINK-22476 best regards, Tony Wei 於 2021年4月26日 週一 下午5:24寫道: > Hi Till, Yangze, > > I think FLINK-15852 should solve my problem. > It is my fault that my flink version is

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

2021-04-26 Thread Tony Wei
Hi Till, Yangze, I think FLINK-15852 should solve my problem. It is my fault that my flink version is not 100% consistent with the community version, and FLINK-15852 is the one I missed. Thanks for your information. best regards, Till Rohrmann 於 2021年4月26日 週一 下午5:14寫道: > I think you are right

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

2021-04-26 Thread Yangze Guo
If the GenericCLI is selected, then the execution.target should have been overwritten to "yarn-application" in GenericCLI#toConfiguration. It is odd that why the GenericCLI#isActive return false as the execution.target is defined in both flink-conf and command line. Best, Yangze Guo On Mon, Apr

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

2021-04-26 Thread Till Rohrmann
I think you are right that the `GenericCLI` should be the first choice. >From the top of my head I do not remember why FlinkYarnSessionCli is still used. Maybe it is in order to support some Yarn specific cli option parsing. I assume it is either an oversight or some parsing has not been

Re: [DISCUSS] Using timeouts in JUnit tests

2021-04-26 Thread Till Rohrmann
+1. I think this rule makes a lot of sense. Cheers, Till On Mon, Apr 26, 2021 at 10:08 AM Arvid Heise wrote: > +1 from my side. > > We should probably double-check if we really need 4h timeouts on test tasks > in AZP. It feels like 2h be enough. > > On Mon, Apr 26, 2021 at 9:54 AM Dawid

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

2021-04-26 Thread Till Rohrmann
Hi Tony, I think you are right that Flink's cli does not behave super consistent at the moment. Case 2. should definitely work because `-t yarn-application` should overwrite what is defined in the Flink configuration. The problem seems to be that we don't resolve the configuration wrt the

Re: [DISCUSS] Using timeouts in JUnit tests

2021-04-26 Thread Arvid Heise
+1 from my side. We should probably double-check if we really need 4h timeouts on test tasks in AZP. It feels like 2h be enough. On Mon, Apr 26, 2021 at 9:54 AM Dawid Wysakowicz wrote: > Hi devs! > > I wanted to bring up something that was discussed in a few independent > groups of people in

[DISCUSS] Using timeouts in JUnit tests

2021-04-26 Thread Dawid Wysakowicz
Hi devs! I wanted to bring up something that was discussed in a few independent groups of people in the past days. I'd like to revise using timeouts in our JUnit tests. The suggestion would be not to use them anymore. The problem with timeouts is that we have no thread dump and stack traces of

[jira] [Created] (FLINK-22474) Display job configuration in web-ui

2021-04-26 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-22474: - Summary: Display job configuration in web-ui Key: FLINK-22474 URL: https://issues.apache.org/jira/browse/FLINK-22474 Project: Flink Issue Type:

[jira] [Created] (FLINK-22473) Kinesis connector options are not listed by its factory

2021-04-26 Thread Jira
Ingo Bürk created FLINK-22473: - Summary: Kinesis connector options are not listed by its factory Key: FLINK-22473 URL: https://issues.apache.org/jira/browse/FLINK-22473 Project: Flink Issue

[jira] [Created] (FLINK-22472) The real partition data produced time is behind meta(_SUCCESS) file produced

2021-04-26 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-22472: -- Summary: The real partition data produced time is behind meta(_SUCCESS) file produced Key: FLINK-22472 URL: https://issues.apache.org/jira/browse/FLINK-22472 Project:

[jira] [Created] (FLINK-22471) Use formatted descriptions for connector options

2021-04-26 Thread Jira
Ingo Bürk created FLINK-22471: - Summary: Use formatted descriptions for connector options Key: FLINK-22471 URL: https://issues.apache.org/jira/browse/FLINK-22471 Project: Flink Issue Type:

[jira] [Created] (FLINK-22470) The root cause of the exception encountered during compiling the job was not exposed to users in certain cases

2021-04-26 Thread Dian Fu (Jira)
Dian Fu created FLINK-22470: --- Summary: The root cause of the exception encountered during compiling the job was not exposed to users in certain cases Key: FLINK-22470 URL:

Re: when should `FlinkYarnSessionCli` be included for parsing CLI arguments?

2021-04-26 Thread Yangze Guo
Hi, Tony. What is the version of your flink-dist. AFAIK, this issue should be addressed in FLINK-15852[1]. Could you give the client log of case 2(set the log level to DEBUG would be better). [1] https://issues.apache.org/jira/browse/FLINK-15852 Best, Yangze Guo On Sun, Apr 25, 2021 at 11:33