Re: Ruquest for Jira assign permissions

2023-08-09 Thread Luke Chen
Hi Romulo, You should be able to assign tasks to yourselves. Thanks. Luke On Sat, Aug 5, 2023 at 9:10 PM Romulo Oliveira wrote: > I added a new Jira task in order to contribute to a new feature for Kafka > Connectors TimestampConverter transformer: > https://issues.apache.org/jira/browse/KAFKA

Re: Single Broker Constantly Reaches 100% CPU

2023-08-09 Thread sunil chaudhari
Point 2 may impact if the size of partitions is too big. too many log segments will cause those many iops I am not expert though On Wed, 9 Aug 2023 at 6:43 PM, Tiansu Yu wrote: > 1. We use cruise-control to actively balance the partitions across all > brokers. So point 1 could be ruled out. > 2.

Re: Single Broker Constantly Reaches 100% CPU

2023-08-09 Thread Tiansu Yu
The Kafka version is 3.2. Replication factor is defined per topic, but minimum 3. We have no control over every producer, but we do recommend our users to set acks as all. Another angle we are considering is that there might be some misbehaving clients using Kafka low level APIs that might not

Lag in one of Kafka broker partition

2023-08-09 Thread Karnam, Sudheer
Hi team, We are using kafka version-2.6.0,has 6brokers and cluster is running for last 3 years. We have a topic with 30 partitions and recently we are observing lag in one of the partition(1,22,000),Other 29 partitions are not having lag. We restarted kafka brokers still lag not reducing in one t

Re: Single Broker Constantly Reaches 100% CPU

2023-08-09 Thread Samuel Cantero
Which kafka version are you running? Which replication factor is being used? are producers using acks=all? On Wed, Aug 9, 2023 at 9:14 AM Tiansu Yu wrote: > 1. We use cruise-control to actively balance the partitions across all > brokers. So point 1 could be ruled out. > 2. I am not sure how muc

Re: Single Broker Constantly Reaches 100% CPU

2023-08-09 Thread Tiansu Yu
1. We use cruise-control to actively balance the partitions across all brokers. So point 1 could be ruled out. 2. I am not sure how much this would impact the broker, as we do have some exceptionally large partitions around. I have to check to know if they live on the aforementioned broker. So

kafka streams consumer group reporting lag even on source topics removed from topology

2023-08-09 Thread Pushkar Deole
Hi All, I have a streams application with 3 instances with application-id set to applicationV1. The application uses processor API with reading from source topics, processing the data and writing to destination topic. Currently it consumes from 6 source topics however we don't need to process data

Re: Single Broker Constantly Reaches 100% CPU

2023-08-09 Thread sunil chaudhari
Hi I can guess two problems here. 1. Either too many partition’s concentrated on this broker compared to other broker 2. The partitions on this broker might have larger size as compared to the parition on other brokers please chech if all brokers are evenly balanced in terms of number of partition

Single Broker Constantly Reaches 100% CPU

2023-08-09 Thread Tiansu Yu
Hi Kafka community, We have an issue with our Kafka cluster from time to time, that a single (one and only one) broker (leader) in the cluster reaches 100% CPU utilisation. We could not see any apparent issue from the metrics. There is no heap memory usage increase, no excessive connections ma