Re: [VOTE] Apache Flink Kafka connector version 3.0.1, RC1

2023-10-28 Thread mystic lama
+1 (non-binding) - verified signatures - build with Java 8 and Java 11 - build success Minor observation - RAT check flagged that README.md is missing ASL On Fri, 27 Oct 2023 at 23:40, Xianxun Ye wrote: > +1(non-binding) > > - Started a local Flink 1.18 cluster, read and wrote with Kafka and

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-10-28 Thread Becket Qin
Thanks for digging into the git history, Jark. I agree it makes sense to deprecate this API in 2.0. Cheers, Jiangjie (Becket) Qin On Fri, Oct 27, 2023 at 5:47 PM Jark Wu wrote: > Hi Becket, > > I checked the history of " > *table.optimizer.source.predicate-pushdown-enabled*", > it seems it

Re: Proposal for Implementing Keyed Watermarks in Apache Flink

2023-10-28 Thread Tawfek Yasser Tawfek
Thanks, Alexander for your reply. Our solution initiated from this inquiry on Stack Overflow: https://stackoverflow.com/questions/52179898/does-flink-support-keyed-watermarks-if-not-is-there-any-plan-of-implementing-i The idea is to generate a watermark for each key (sub-stream), in order to

Re: Re: [DISCUSS] FLIP-376: Add DISTRIBUTED BY clause

2023-10-28 Thread Benchao Li
Thanks Timo for preparing the FLIP. Regarding "By default, DISTRIBUTED BY assumes a list of columns for an implicit hash partitioning." Do you think it's useful to add some extensibility for the hash strategy. One scenario I can foresee is if we write bucketed data into Hive, and if Flink's hash

[Discuss] Proposal to add TTL Configuration in JobResultStore for cleanJobResults

2023-10-28 Thread Samrat Deb
Hi All, The JobResultStore interface includes an Abstract implementation known as `AbstractThreadsafeJobResultStore`. This implementation is extended by two concrete implementations: `FileSystemJobResultStore` and `EmbeddedJobResultStore`. Both of these concrete implementations handle two types

Re: [VOTE] Apache Flink Kafka connector version 3.0.1, RC1

2023-10-28 Thread Xianxun Ye
+1(non-binding) - Started a local Flink 1.18 cluster, read and wrote with Kafka and Upsert Kafka connector successfully to Kafka 2.2 cluster One minor question: should we update the dependency manual of these two documentations[1][2]? [1]

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-10-28 Thread Jiabao Sun
Thanks Venkatakrishnan for the feedback. Taking MySQL as an example, if the pushed-down filter does not hit an index, it will result in a full table scan. For a table with a large amount of data, a full table scan can consume a significant amount of CPU resources, increase response time, hold