Re: Kafka incremental sticky rebalancing.

2021-06-21 Thread Luke Chen
Hi Mazen, I think you can ignore this message as the message said, you can try completing the rebalance by calling poll() and then retry the operation. You should be able to complete the offset commit in the next poll round. Thanks. Luke On Mon, Jun 21, 2021 at 10:19 PM Mazen Ezzeddine <

Re: How to avoid storing password in clear text in server.properties file

2021-06-21 Thread Dhirendra Singh
I am using apache kafka. any such feature available in it ? On Mon, Jun 21, 2021 at 7:17 PM Ran Lupovich wrote: > Using Confluent Platform you can use feature called Secrets , I am not > familiar with open source solution for this. > >

Re: Consumer Group Stuck on "Completing-Rebalance" State

2021-06-21 Thread Ran Lupovich
https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-12890 Check out this jira ticket בתאריך יום ב׳, 21 ביוני 2021, 22:15, מאת Tao Huang ‏< sandy.huang...@gmail.com>: > Hi There, > > I am experiencing intermittent issue when consumer group stuck on > "Completing-Reblalance" state.

Consumer Group Stuck on "Completing-Rebalance" State

2021-06-21 Thread Tao Huang
Hi There, I am experiencing intermittent issue when consumer group stuck on "Completing-Reblalance" state. When this is happening, client throws error as below: 2021-06-18 13:55:41,086 ERROR io.mylab.adapter.KafkaListener [edfKafkaListener:CIO.PandC.CIPG.InternalLoggingMetadataInfo] Exception on

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-21 Thread Bill Bejeck
Congrats Konstantine! -Bill On Mon, Jun 21, 2021 at 12:44 PM Tom Bentley wrote: > Congratulations Konstantine! > > On Mon, Jun 21, 2021 at 5:33 PM David Jacot > wrote: > > > Congrats, Konstantine. Well deserved! > > > > Best, > > David > > > > On Mon, Jun 21, 2021 at 6:14 PM Ramesh Krishnan

Embedded Kafka connector in data ingestion service

2021-06-21 Thread Boyang Chen
Hey there, I'm wondering if anyone has the need to use an embedded Kafka connector module. The goal we want to achieve is to avoid letting customers maintain a separate component when they stream data from their Kafka cluster to our service, so that they just need to provide the cluster

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-21 Thread Tom Bentley
Congratulations Konstantine! On Mon, Jun 21, 2021 at 5:33 PM David Jacot wrote: > Congrats, Konstantine. Well deserved! > > Best, > David > > On Mon, Jun 21, 2021 at 6:14 PM Ramesh Krishnan > wrote: > > > Congrats Konstantine > > > > On Mon, 21 Jun 2021 at 8:58 PM, Mickael Maison > > wrote: >

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-21 Thread Ramesh Krishnan
Congrats Konstantine On Mon, 21 Jun 2021 at 8:58 PM, Mickael Maison wrote: > Hi, > > It's my pleasure to announce that Konstantine Karantasis is now a > member of the Kafka PMC. > > Konstantine has been a Kafka committer since Feb 2020. He has remained > active in the community since becoming a

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-21 Thread David Jacot
Congrats, Konstantine. Well deserved! Best, David On Mon, Jun 21, 2021 at 6:14 PM Ramesh Krishnan wrote: > Congrats Konstantine > > On Mon, 21 Jun 2021 at 8:58 PM, Mickael Maison > wrote: > > > Hi, > > > > It's my pleasure to announce that Konstantine Karantasis is now a > > member of the

Re: [ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-21 Thread Israel Ekpo
Congratulations Konstantine and thank you for your service! On Mon, Jun 21, 2021 at 11:28 AM Mickael Maison wrote: > Hi, > > It's my pleasure to announce that Konstantine Karantasis is now a > member of the Kafka PMC. > > Konstantine has been a Kafka committer since Feb 2020. He has remained >

[ANNOUNCE] New Kafka PMC Member: Konstantine Karantasis

2021-06-21 Thread Mickael Maison
Hi, It's my pleasure to announce that Konstantine Karantasis is now a member of the Kafka PMC. Konstantine has been a Kafka committer since Feb 2020. He has remained active in the community since becoming a committer. Congratulations Konstantine! Mickael, on behalf of the Apache Kafka PMC

Re: How to avoid storing password in clear text in server.properties file

2021-06-21 Thread Ryanne Dolan
Take a look at the ConfigProvider interface. On Mon, Jun 21, 2021, 8:26 AM Dhirendra Singh wrote: > Hi All, > I am currently storing various passwords like "ssl.keystore.password", > "ssl.truststore.password", SASL plain user password in cleartext in > server.properties file. > is there any way

Re: Kafka incremental sticky rebalancing.

2021-06-21 Thread Mazen Ezzeddine
Hi Luke, I am committing the offsets manually, and after processing each of the records returned by one poll loop iteration using consumer.commitSync(). Regards, From: Luke Chen Sent: Monday, June 21, 2021 3:06 PM To: Kafka Users Subject: Re: Kafka

Re: How to avoid storing password in clear text in server.properties file

2021-06-21 Thread Ran Lupovich
Using Confluent Platform you can use feature called Secrets , I am not familiar with open source solution for this. https://docs.confluent.io/platform/current/security/secrets.html בתאריך יום ב׳, 21 ביוני 2021, 16:26, מאת Dhirendra Singh ‏< dhirendr...@gmail.com>: > Hi All, > I am currently

How to avoid storing password in clear text in server.properties file

2021-06-21 Thread Dhirendra Singh
Hi All, I am currently storing various passwords like "ssl.keystore.password", "ssl.truststore.password", SASL plain user password in cleartext in server.properties file. is there any way to store the password in encrypted text ? i am using kafka version 2.5.0

Re: Kafka incremental sticky rebalancing.

2021-06-21 Thread Luke Chen
Hi Mazen, Did you auto commit offsets or manually? When do you commit offsets? Thanks Luke Mazen Ezzeddine 於 2021年6月21日 週一 下午7:42 寫道: > I am running Kafka on Kubernetes using the Kafka Strimzi operator. I am > using incremental sticky rebalance strategy by configuring my consumers > with the

Kafka incremental sticky rebalancing.

2021-06-21 Thread Mazen Ezzeddine
I am running Kafka on Kubernetes using the Kafka Strimzi operator. I am using incremental sticky rebalance strategy by configuring my consumers with the following: ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG,

Replica selection in unclean leader election and min.insync.replicas=2

2021-06-21 Thread Oleksandr Shulgin
Hi, We are running Apache Kafka v2.7.0 in production in a 3-rack setup (3 AZs in a single AWS region) with the per-topic replication factor of 3 and the following global settings: unclean.leader.election.enable=false min.insync.replicas=2 replica.lag.time.max.ms=1