Re: Cleanup unused topics

2018-08-21 Thread Brett Rann
These are in regex form for DataDog's JMX collector, but it should get you started: bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=TotalProduceRequestsPerSec,topic=.*' bean_regex: 'kafka\.server:type=BrokerTopicMetrics,name=TotalFetchRequestsPerSec,topic=.*'

Cleanup unused topics

2018-08-21 Thread Luigi Tagliamonte
I've got a sandbox cluster (without any control on topic creation) full of unused topics and I'd like to cleanup all the those topics. I was able to identify all the topics with 0 messages with the following lines: *#!/bin/bash* *set -e* *BROKERS="10.16.208.220:9092

Kakfa embedded cluster rebalance scenario test

2018-08-21 Thread pradeep s
Hi , I would like to add a integration test for kafka rebalance scenario and make sure retries are working as expected if enough replicas are not there in my kafka streams application . Do you have any info on how can i achieve this . I was checking TestUtils class in org.apache.kafka.test, but

Re: Improve error message when trying to produce message without key for compacted topic

2018-08-21 Thread Matthias J. Sax
That makes sense to me. Please file a Jira. Thanks a lot for reporting this issue! -Matthias On 8/21/18 3:29 AM, Patrik Kleindl wrote: > Hello > > Yesterday we had the following exception: > > Exception thrown when sending a message with key='null' and payload='...' > to topic sometopic::

Improve error message when trying to produce message without key for compacted topic

2018-08-21 Thread Patrik Kleindl
Hello Yesterday we had the following exception: Exception thrown when sending a message with key='null' and payload='...' to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: This message has failed its CRC checksum, exceeds the valid size, or is otherwise corrupt. The

Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

2018-08-21 Thread Viktor Somogyi-Vass
Congrats Dong! :) On Tue, Aug 21, 2018 at 10:09 AM James Cheng wrote: > Congrats Dong! > > -James > > > On Aug 20, 2018, at 3:54 AM, Ismael Juma wrote: > > > > Hi everyone, > > > > Dong Lin became a committer in March 2018. Since then, he has remained > > active in the community and

Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

2018-08-21 Thread James Cheng
Congrats Dong! -James > On Aug 20, 2018, at 3:54 AM, Ismael Juma wrote: > > Hi everyone, > > Dong Lin became a committer in March 2018. Since then, he has remained > active in the community and contributed a number of patches, reviewed > several pull requests and participated in numerous KIP

Re: [ANNOUNCE] New Kafka PMC member: Dong Lin

2018-08-21 Thread Dong Lin
Thank you everyone!! It has been my great pleasure to contribute to and be part of the Apache Kafka community! On Mon, Aug 20, 2018 at 3:50 PM, Mayuresh Gharat wrote: > Congrats Dong !!! > > Thanks, > > Mayuresh > > On Mon, Aug 20, 2018 at 1:36 PM Gwen Shapira wrote: > > > Congrats Dong Lin!

Re: Kafka SASL (Kerberos) - UNKNOWN_TOPIC_OR_PARTITION

2018-08-21 Thread Manikumar
"allow.everyone.if.no.acl.found" config is applicable for Kafka's inbuilt authorizer implementation (SimpleAclAuthorizer). What is the Kafka version? You can check authorizer debug logs starting with "No acl found for resource..." make sure no acls for ANONYMOUS user. Are you using any custom