Thanks Alberto. On Fri, Jan 13, 2017 at 9:02 AM, <alberto.fo...@natwestmarkets.com.invalid> wrote:
> Sure, done: KAFKA-4629 > > Thanks, > Alberto > > > -----Original Message----- > From: Guozhang Wang [mailto:wangg...@gmail.com] > Sent: 13 January 2017 16:32 > To: users@kafka.apache.org > Subject: Re: Per topic MBeans leaking > > ********************************************* > " This message originates from outside our organisation. Consider > carefully whether you should click on any links, open any attachments or > reply. If in doubt, forward to ~ Phishing" > ********************************************* > > The way you used admin tools to create and delete topics looks reasonable > to me. I suspect there may be some race conditions causing the metrics to > not be de-registered. > > Could you file a JIRA to track this issue so we can investigate? > > > Guozhang > > On Fri, Jan 13, 2017 at 8:12 AM, <Alberto.Forti@natwestmarkets. > com.invalid> > wrote: > > > Hi, > > > > In our application we create and delete topics dynamically. In our > > dev/test environment, where we create and delete a lot of topics, I've > > noticed that the memory usage of the Kafka brokers slowly increases > > and then noticed the huge amount of MBeans related to topics that > > should have been deleted (e.g. > > kafka.server:type=BrokerTopicMetrics,name= > > TotalProduceRequestsPerSec,topic=dw_06b5f828-e452-4e22- > 89c9-67849a65603d). > > > > Also, deleting a topic often produces (what I think is) noise in the > > logs at WARN level, for example: > > 2017-01-13 14:52:46 WARN > > PartitionStateMachine$DeleteTopicsListener:83 - [DeleteTopicsListener > > on 1]: Ignoring request to delete non-existing topics > > dw_fe8ff14b-aa9b-4f24-9bc1-6fbce15d20d2 > > > > This behaviour is easy to reproduce with a basic Kafka cluster with > > two brokers, just create and delete topics few times. Sometimes the > > MBeans for the topic are deleted and sometimes not. > > > > Maybe I'm deleting the topics in the wrong way? I'm using the > > AdminUtils class in the Java API: > > AdminUtils.deleteTopic(zkUtils, topicName); > > > > Topics are created using the AdminUtils class as well: > > AdminUtils.createTopic(zkUtils, topicName, partitions, > > replicationFactor, topicConfiguration, > > kafka.admin.RackAwareMode.Enforced$.MODULE$); > > > > The Kafka version is 0.10.0.1. > > > > Thanks, > > Alberto > > > > > > ****************************************************************** > > > > NatWest Markets is a marketing name of The Royal Bank of Scotland plc. > > > > This communication and any attachments are confidential and intended > > solely for the addressee. If you are not the intended recipient please > > advise us immediately and delete it. Unless specifically stated in the > > message or otherwise indicated, you may not duplicate, redistribute or > > forward this message and any attachments are not intended for > > distribution to, or use by any person or entity in any jurisdiction or > > country where such distribution or use would be contrary to local law > > or regulation. The Royal Bank Of Scotland plc or any affiliated entity > > ("RBS") accepts no responsibility for any changes made to this message > after it was sent. > > > > Unless otherwise specifically indicated, the contents of this > > communication and its attachments are for information purposes only > > and should not be regarded as an offer or solicitation to buy or sell > > a product or service, confirmation of any transaction, a valuation, > > indicative price or an official statement. Where this communication > > has been prepared by an RBS trading desk, that desk may have a > > position or interest in the products or services mentioned that is > > inconsistent with any views expressed in this message. In evaluating > > the information contained in this message, you should know that it > > could have been previously provided to other clients and/or internal RBS > personnel, who could have already acted on it. > > > > RBS cannot provide absolute assurances that all electronic > > communications (sent or received) are secure, error free, not > > corrupted, incomplete or virus free and/or that they will not be lost, > > mis-delivered, destroyed, delayed or intercepted/decrypted by others. > > Therefore RBS disclaims all liability with regards to electronic > > communications (and the contents > > therein) if they are corrupted, lost destroyed, delayed, incomplete, > > mis-delivered, intercepted, decrypted or otherwise misappropriated by > > others. > > > > Any electronic communication that is conducted within or through RBS > > systems will be subject to being archived, monitored and produced to > > regulators and in litigation in accordance with RBS's policy and local > > laws, rules and regulations. Unless expressly prohibited by local law, > > electronic communications may be archived in countries other than the > > country in which you are located, and may be treated in accordance > > with the laws and regulations of the country of each individual > > included in the entire chain. > > > > Copyright 2014 The Royal Bank of Scotland plc. All rights reserved. > > See http://www.natwestmarkets.com/legal/s-t-discl.html for further > > risk disclosure. > > > > ****************************************************************** > > > > > > -- > -- Guozhang > ****************************************************************** > > NatWest Markets is a marketing name of The Royal Bank of Scotland plc. > > This communication and any attachments are confidential and intended > solely for the addressee. If you are not the intended recipient please > advise us immediately and delete it. Unless specifically stated in the > message or otherwise indicated, you may not duplicate, redistribute or > forward this message and any attachments are not intended for distribution > to, or use by any person or entity in any jurisdiction or country where > such distribution or use would be contrary to local law or regulation. The > Royal Bank Of Scotland plc or any affiliated entity ("RBS") accepts no > responsibility for any changes made to this message after it was sent. > > Unless otherwise specifically indicated, the contents of this > communication and its attachments are for information purposes only and > should not be regarded as an offer or solicitation to buy or sell a product > or service, confirmation of any transaction, a valuation, indicative price > or an official statement. Where this communication has been prepared by an > RBS trading desk, that desk may have a position or interest in the products > or services mentioned that is inconsistent with any views expressed in this > message. In evaluating the information contained in this message, you > should know that it could have been previously provided to other clients > and/or internal RBS personnel, who could have already acted on it. > > RBS cannot provide absolute assurances that all electronic communications > (sent or received) are secure, error free, not corrupted, incomplete or > virus free and/or that they will not be lost, mis-delivered, destroyed, > delayed or intercepted/decrypted by others. Therefore RBS disclaims all > liability with regards to electronic communications (and the contents > therein) if they are corrupted, lost destroyed, delayed, incomplete, > mis-delivered, intercepted, decrypted or otherwise misappropriated by > others. > > Any electronic communication that is conducted within or through RBS > systems will be subject to being archived, monitored and produced to > regulators and in litigation in accordance with RBS's policy and local > laws, rules and regulations. Unless expressly prohibited by local law, > electronic communications may be archived in countries other than the > country in which you are located, and may be treated in accordance with the > laws and regulations of the country of each individual included in the > entire chain. > > Copyright 2014 The Royal Bank of Scotland plc. All rights reserved. See > http://www.natwestmarkets.com/legal/s-t-discl.html for further risk > disclosure. > > ****************************************************************** > -- -- Guozhang
