Re: [DISCUSS] Creating an external connector repository

Hi all, My name is Kyle and I’m an open source developer primarily focused on Apache Iceberg. I’m happy to help clarify or elaborate on any aspect of our experience working on a relatively decoupled connector that is downstream and pretty popular. I’d also love to be able to contribute or

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

Hi Jingsong, thanks for the proposal, providing a built-in storage solution for users will make flink SQL much more easier to use in production. I have some questions which may be missed in the FLIP, but may be important IMO: 1. Is it possible to read historical data from the file store first and

[jira] [Created] (FLINK-24607) SourceCoordinator may miss to close SplitEnumerator when failover frequently

Jark Wu created FLINK-24607: --- Summary: SourceCoordinator may miss to close SplitEnumerator when failover frequently Key: FLINK-24607 URL: https://issues.apache.org/jira/browse/FLINK-24607 Project: Flink

[jira] [Created] (FLINK-24606) AvroDeserializationSchema buffer is not clean

heyu dou created FLINK-24606: Summary: AvroDeserializationSchema buffer is not clean Key: FLINK-24606 URL: https://issues.apache.org/jira/browse/FLINK-24606 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-24605) org.apache.flink.kafka.shaded.org.apache.kafka.clients.consumer.NoOffsetForPartitionException: Undefined offset with no reset policy for partitions

Abhijit Talukdar created FLINK-24605: Summary: org.apache.flink.kafka.shaded.org.apache.kafka.clients.consumer.NoOffsetForPartitionException: Undefined offset with no reset policy for partitions Key: FLINK-24605

"streaming.tests.avro does not exist"

Hello everyone, I am trying to set up IntelliJ IDEA so that I can start contributing but I am running into the following error: flink/flink-end-to-end-tests/flink-datastream-allround-test/src/main/java/org/apache/flink/streaming/tests/DataStreamAllroundTestProgram.java:34:45 java: package

Re: "streaming.tests.avro does not exist"

What you're lacking are generated files. Run mvn generate-sources (or mvn clean install) on the command-line. On 20/10/2021 20:22, Saad Ur Rahman wrote: Hello everyone, I am trying to set up IntelliJ IDEA so that I can start contributing but I am running into the following error:

Re: [DISCUSS] Creating an external connector repository

Hi, I see the stable core Flink API as a prerequisite for modularity. And for connectors it is not just the source and sink API (source being stable as of 1.14), but everything that is required to build and maintain a connector downstream, such as the test utilities and infrastructure. Without

"streaming.tests.avro does not exist"

Hello everyone, I am trying to set up IntelliJ IDEA so that I can start contributing but I am running into the following error: flink/flink-end-to-end-tests/flink-datastream-allround-test/src/main/java/org/apache/flink/streaming/tests/DataStreamAllroundTestProgram.java:34:45 java: package

[jira] [Created] (FLINK-24604) Failing tests for casting decimals to boolean

Marios Trivyzas created FLINK-24604: --- Summary: Failing tests for casting decimals to boolean Key: FLINK-24604 URL: https://issues.apache.org/jira/browse/FLINK-24604 Project: Flink Issue

Looking back at the Apache Flink 1.14 development cycle / getting better for 1.15

Dear Flink community, In preparation for the 1.15 development cycle of Apache Flink (it already started) and preparing the release management we are collecting feedback from the community. If you didn’t have a chance to look at the release announcement you might want to do that now [1] Also

[jira] [Created] (FLINK-24603) Add E2E test

Chesnay Schepler created FLINK-24603: Summary: Add E2E test Key: FLINK-24603 URL: https://issues.apache.org/jira/browse/FLINK-24603 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

Hi Jingsong, thank you for writing up the proposal. The benefits such a mechanism will bring will be very valuable! I haven't yet looked into this in detail, but one question came to my mind immediately: The DDL for these tables seems to rely on there not being a 'connector' option. However,

[jira] [Created] (FLINK-24602) Include diffs for configuration documentation page

Zichen Liu created FLINK-24602: -- Summary: Include diffs for configuration documentation page Key: FLINK-24602 URL: https://issues.apache.org/jira/browse/FLINK-24602 Project: Flink Issue Type:

Re: [DISCUSS] Creating an external connector repository

Hi Konstantin, > the connectors need to be adopted and require at least one release per Flink minor release. However, this will make the releases of connectors slower, e.g. maintain features for multiple branches and release multiple branches. I think the main purpose of having an external

Re: [DISCUSS] FLIP-176: Unified Iteration to Support Algorithms (Flink ML)

Hi all, With some offline discussion with Becket and Zhipeng, we have made some modification to the API to provide a unified method for iteration on the bounded streams, and also allow users to construct the iteration with mixed types of operator lifecycle (namely whether we re-create users'

[DISCUSS] FLIP-188: Introduce Built-in Dynamic Table Storage

Hi all, Kurt and I propose to introduce built-in storage support for dynamic table, a truly unified changelog & table representation, from Flink SQL’s perspective. We believe this kind of storage will improve the usability a lot. We want to highlight some characteristics about this storage: -

[jira] [Created] (FLINK-24601) Error in submitting the job

Shyam created FLINK-24601: -- Summary: Error in submitting the job Key: FLINK-24601 URL: https://issues.apache.org/jira/browse/FLINK-24601 Project: Flink Issue Type: Bug Reporter: Shyam

One week until Flink Forward! -- Info & special announcements

Hi Everyone, We have just one week before Flink Forward Global 2021!! If you (or a friend) still need to register, you can do so here for free. Since we’re virtual, you can join from anywhere in the world. I am excited to announce that the

[jira] [Created] (FLINK-24600) job checkpoint page has two p99 which is duplicated

donglei created FLINK-24600: --- Summary: job checkpoint page has two p99 which is duplicated Key: FLINK-24600 URL: https://issues.apache.org/jira/browse/FLINK-24600 Project: Flink Issue Type: New

[jira] [Created] (FLINK-24599) Make checking for type root and family less verbose

Timo Walther created FLINK-24599: Summary: Make checking for type root and family less verbose Key: FLINK-24599 URL: https://issues.apache.org/jira/browse/FLINK-24599 Project: Flink Issue

[jira] [Created] (FLINK-24598) Current IT case do not cover fallback path for hash aggregate

Shuo Cheng created FLINK-24598: -- Summary: Current IT case do not cover fallback path for hash aggregate Key: FLINK-24598 URL: https://issues.apache.org/jira/browse/FLINK-24598 Project: Flink

Re: [DISCUSS] Creating an external connector repository

Hi everyone, regarding the stability of the APIs. I think everyone agrees that connector APIs which are stable across minor versions (1.13->1.14) are the mid-term goal. But: a) These APIs are still quite young, and we shouldn't make them @Public prematurely either. b) Isn't this *mostly*

Re: [DISCUSS] Creating an external connector repository

Hi, I think Thomas raised very good questions and would like to know your opinions if we want to move connectors out of flink in this version. (1) is the connector API already stable? > Separate releases would only make sense if the core Flink surface is > fairly stable though. As evident from

[DISCUSS] FLIP-187: Adaptive Batch Job Scheduler

Hi all, Zhu Zhu and I propose to introduce a new job scheduler to Flink: adaptive batch job scheduler. The new scheduler can automatically decide parallelisms of job vertices for batch jobs, according to the size of data volume each vertex needs to process. Major benefits of this scheduler

[jira] [Created] (FLINK-24597) RocksdbStateBackend getKeysAndNamespaces would return duplicate data when using MapState

Yue Ma created FLINK-24597: -- Summary: RocksdbStateBackend getKeysAndNamespaces would return duplicate data when using MapState Key: FLINK-24597 URL: https://issues.apache.org/jira/browse/FLINK-24597