Re: [DISCUSS] Backport HybridSource to 1.13.x

2021-08-20 Thread Till Rohrmann
Hi everyone, if the HybridSource is self contained, then I don't see a lot of harm in doing the backport. We should probably explicitly state in the docs that a certain minor version is required for this source to be available because our docs don't distinguish between minor versions and having

[jira] [Created] (FLINK-23894) Resolve/hide akka warning regarding ActorRefProvider

2021-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23894: Summary: Resolve/hide akka warning regarding ActorRefProvider Key: FLINK-23894 URL: https://issues.apache.org/jira/browse/FLINK-23894 Project: Flink

[jira] [Created] (FLINK-23895) Upsert materializer is not inserted for all sink providers

2021-08-20 Thread Timo Walther (Jira)
Timo Walther created FLINK-23895: Summary: Upsert materializer is not inserted for all sink providers Key: FLINK-23895 URL: https://issues.apache.org/jira/browse/FLINK-23895 Project: Flink

[jira] [Created] (FLINK-23887) Check thread pollution for 1.14

2021-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23887: Summary: Check thread pollution for 1.14 Key: FLINK-23887 URL: https://issues.apache.org/jira/browse/FLINK-23887 Project: Flink Issue Type:

[jira] [Created] (FLINK-23890) CepOperator may create a large number of timers and cause performance problems

2021-08-20 Thread Yue Ma (Jira)
Yue Ma created FLINK-23890: -- Summary: CepOperator may create a large number of timers and cause performance problems Key: FLINK-23890 URL: https://issues.apache.org/jira/browse/FLINK-23890 Project: Flink

[jira] [Created] (FLINK-23897) Fix obsolete doc about creating hive table with flink

2021-08-20 Thread Rui Li (Jira)
Rui Li created FLINK-23897: -- Summary: Fix obsolete doc about creating hive table with flink Key: FLINK-23897 URL: https://issues.apache.org/jira/browse/FLINK-23897 Project: Flink Issue Type: Bug

Projection pushdown for metadata columns

2021-08-20 Thread Ingo Bürk
Hi everyone, according to the SupportsReadableMetadata interface, the planner is supposed to project required metadata columns prior to applying them: > The planner will select required metadata columns (i.e. perform projection push down) and will call applyReadableMetadata(List, DataType) with

[jira] [Created] (FLINK-23891) Support 'sink.partition-commit.policy.kind'='metastore,success-file' config for batch Hive sink

2021-08-20 Thread Yik San Chan (Jira)
Yik San Chan created FLINK-23891: Summary: Support 'sink.partition-commit.policy.kind'='metastore,success-file' config for batch Hive sink Key: FLINK-23891 URL: https://issues.apache.org/jira/browse/FLINK-23891

[jira] [Created] (FLINK-23898) WARN log level not mapped correctly to akka log levels

2021-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23898: Summary: WARN log level not mapped correctly to akka log levels Key: FLINK-23898 URL: https://issues.apache.org/jira/browse/FLINK-23898 Project: Flink

Re: [DISCUSS] FLIP-171: Async Sink

2021-08-20 Thread Hausmann, Steffen
Hi Till, I've updated the wiki page as per the discussion on flip-177. I hope it makes more sense now. Cheers, Steffen On 16.07.21, 18:28, "Till Rohrmann" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can

[jira] [Created] (FLINK-23889) Unnecessary logging from FineGrainedSlotManager

2021-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23889: Summary: Unnecessary logging from FineGrainedSlotManager Key: FLINK-23889 URL: https://issues.apache.org/jira/browse/FLINK-23889 Project: Flink

[jira] [Created] (FLINK-23896) The new KafkaSink drops data if job fails between checkpoint and transaction commit.

2021-08-20 Thread Jira
David Morávek created FLINK-23896: - Summary: The new KafkaSink drops data if job fails between checkpoint and transaction commit. Key: FLINK-23896 URL: https://issues.apache.org/jira/browse/FLINK-23896

[jira] [Created] (FLINK-23899) Translate the "Elastic Scaling" page into chinese

2021-08-20 Thread Wenhao Ji (Jira)
Wenhao Ji created FLINK-23899: - Summary: Translate the "Elastic Scaling" page into chinese Key: FLINK-23899 URL: https://issues.apache.org/jira/browse/FLINK-23899 Project: Flink Issue Type:

[jira] [Created] (FLINK-23903) ZooKeeperLeaderRetrievalConnectionHandlingTest.testSuspendedConnectionDoesNotClearLeaderInformationIfClearanceOnLostConnection fails on azure

2021-08-20 Thread Xintong Song (Jira)
Xintong Song created FLINK-23903: Summary: ZooKeeperLeaderRetrievalConnectionHandlingTest.testSuspendedConnectionDoesNotClearLeaderInformationIfClearanceOnLostConnection fails on azure Key: FLINK-23903 URL:

[jira] [Created] (FLINK-23888) Flink azure fs doesn't work

2021-08-20 Thread Liviu Firu (Jira)
Liviu Firu created FLINK-23888: -- Summary: Flink azure fs doesn't work Key: FLINK-23888 URL: https://issues.apache.org/jira/browse/FLINK-23888 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-23892) FineGrainedSlotManager is not sending signal for lacking resources

2021-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23892: Summary: FineGrainedSlotManager is not sending signal for lacking resources Key: FLINK-23892 URL: https://issues.apache.org/jira/browse/FLINK-23892 Project:

[jira] [Created] (FLINK-23901) Fink with HA, upgrade from 1.12.1 to 1.13.2 causes NPE

2021-08-20 Thread Parag Somani (Jira)
Parag Somani created FLINK-23901: Summary: Fink with HA, upgrade from 1.12.1 to 1.13.2 causes NPE Key: FLINK-23901 URL: https://issues.apache.org/jira/browse/FLINK-23901 Project: Flink Issue

[jira] [Created] (FLINK-23902) Support Hive version 2.3.8

2021-08-20 Thread Jeff Yang (Jira)
Jeff Yang created FLINK-23902: - Summary: Support Hive version 2.3.8 Key: FLINK-23902 URL: https://issues.apache.org/jira/browse/FLINK-23902 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-23893) Fine-grained resource allocation may fail depending on slot allocation order

2021-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23893: Summary: Fine-grained resource allocation may fail depending on slot allocation order Key: FLINK-23893 URL: https://issues.apache.org/jira/browse/FLINK-23893

[jira] [Created] (FLINK-23900) Migrate to ClusterActorRefProvider

2021-08-20 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23900: Summary: Migrate to ClusterActorRefProvider Key: FLINK-23900 URL: https://issues.apache.org/jira/browse/FLINK-23900 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-171: Async Sink

2021-08-20 Thread Till Rohrmann
Thanks for the update Steffen. I'll try to take a look at it asap. Cheers, Till On Fri, Aug 20, 2021 at 1:34 PM Hausmann, Steffen wrote: > Hi Till, > > I've updated the wiki page as per the discussion on flip-177. I hope it > makes more sense now. > > Cheers, Steffen > > On 16.07.21, 18:28,

[jira] [Created] (FLINK-23904) Add a parameter to flink's startup command that points to the directory named jobid containing all versions of checkpoint

2021-08-20 Thread liwei li (Jira)
liwei li created FLINK-23904: Summary: Add a parameter to flink's startup command that points to the directory named jobid containing all versions of checkpoint Key: FLINK-23904 URL:

Scala 2.12 End to End Tests

2021-08-20 Thread Danny Cranmer
Hello, I am working on reviewing a PR [1] to add JSON support for the AWS Glue Schema Registry integration. The module only builds for Scala 2.12 due to a dependency incompatibility. The end-to-end tests have been implemented using the newer Java framework rather than bash scripts. However, it

[jira] [Created] (FLINK-23905) Reduce the load on JobManager when submitting large-scale job with a big user jar

2021-08-20 Thread huntercc (Jira)
huntercc created FLINK-23905: Summary: Reduce the load on JobManager when submitting large-scale job with a big user jar Key: FLINK-23905 URL: https://issues.apache.org/jira/browse/FLINK-23905 Project: