flink 1.17 connector adapt

2023-03-30 Thread tian tian
Hi, Flink 1.17 has been released, but elasticsearch and rabbitmq have not been adapted to 1.17. Is there any plan to adapt it?

Re: Unexpected behaviour when configuring both `withWatermarkAlignment` & `withIdleness`

2023-03-30 Thread Reem Razak via user
This sounds very much like our issue, thank you! We will follow along with the bug. Much appreciated, - Reem On Thu, Mar 30, 2023 at 9:20 AM Martijn Visser wrote: > Hi Reem > > My thinking is that this might be related to recently reported > https://issues.apache.org/jira/browse/FLINK-31632. >

Re: KafkaSource consumer group

2023-03-30 Thread Tzu-Li (Gordon) Tai
Sorry, I meant to say "Hi Ben" :-) On Thu, Mar 30, 2023 at 9:52 AM Tzu-Li (Gordon) Tai wrote: > Hi Robert, > > This is a design choice. Flink's KafkaSource doesn't rely on consumer > groups for assigning partitions / rebalancing / offset tracking. It > manually assigns whatever partitions are

Re: KafkaSource consumer group

2023-03-30 Thread Tzu-Li (Gordon) Tai
Hi Robert, This is a design choice. Flink's KafkaSource doesn't rely on consumer groups for assigning partitions / rebalancing / offset tracking. It manually assigns whatever partitions are in the specified topic across its consumer instances, and rebalances only when the Flink job / KafkaSink is

KafkaSource consumer group

2023-03-30 Thread Roberts, Ben (Senior Developer) via user
Hi, Is there a way to run multiple flink jobs with the same Kafka group.id and have them join the same consumer group? It seems that setting the group.id using KafkaSource.builder().set_group_id() does not have the effect of creating an actual consumer group in Kafka. Running the same flink

Re: [Discussion] - Release major Flink version to support JDK 17 (LTS)

2023-03-30 Thread Piotr Nowojski
Hey, > 1. The Flink community agrees that we upgrade Kryo to a later version, which means breaking all checkpoint/savepoint compatibility and releasing a Flink 2.0 with Java 17 support added and Java 8 and Flink Scala API support dropped. This is probably the quickest way, but would still mean

Re: [Discussion] - Release major Flink version to support JDK 17 (LTS)

2023-03-30 Thread Alexis Sarda-Espinosa
Hi Martijn, just to be sure, if all state-related classes use a POJO serializer, Kryo will never come into play, right? Given FLINK-16686 [1], I wonder how many users actually have jobs with Kryo and RocksDB, but even if there aren't many, that still leaves those who don't use RocksDB for

Re: [Discussion] - Release major Flink version to support JDK 17 (LTS)

2023-03-30 Thread Martijn Visser
Hi all, I also saw a thread on this topic from Clayton Wohl [1] on this topic, which I'm including in this discussion thread to avoid that it gets lost. >From my perspective, there's two main ways to get to Java 17: 1. The Flink community agrees that we upgrade Kryo to a later version, which

[ANNOUNCE] TAC supporting Berlin Buzzwords

2023-03-30 Thread Martijn Visser
Hi everyone, I'm forwarding the following information from the ASF Travel Assistance Committee (TAC): --- Hi All, The ASF Travel Assistance Committee is supporting taking up to six (6) people to attend Berlin Buzzwords [1] In June this year. This includes Conference passes, and travel &

Re: [ANNOUNCE] Flink Table Store Joins Apache Incubator as Apache Paimon(incubating)

2023-03-30 Thread Jane Chan
Congratulations! Best regards, Jane On Thu, Mar 30, 2023 at 1:38 PM Jiadong Lu wrote: > Congratulations !!! > > Best, > Jiadong Lu > > On 2023/3/27 17:23, Yu Li wrote: > > Dear Flinkers, > > > > > > > As you may have noticed, we are pleased to announce that Flink Table Store > has joined the

Re: Unexpected behaviour when configuring both `withWatermarkAlignment` & `withIdleness`

2023-03-30 Thread Martijn Visser
Hi Reem My thinking is that this might be related to recently reported https://issues.apache.org/jira/browse/FLINK-31632. Best regards, Martijn On Wed, Mar 29, 2023 at 7:07 PM Reem Razak via user wrote: > Hey Martijn, > > The version is 1.16.0 > > On Wed, Mar 29, 2023 at 5:43 PM Martijn