The way you used admin tools to create and delete topics looks reasonable
to me. I suspect there may be some race conditions causing the metrics to
not be de-registered.
Could you file a JIRA to track this issue so we can investigate?
Guozhang
On Fri, Jan 13, 2017 at 8:12 AM, <alberto.fo...@natwestmarkets.com.invalid>
wrote:
> Hi,
>
> In our application we create and delete topics dynamically. In our
> dev/test environment, where we create and delete a lot of topics, I've
> noticed that the memory usage of the Kafka brokers slowly increases and
> then noticed the huge amount of MBeans related to topics that should have
> been deleted (e.g. kafka.server:type=BrokerTopicMetrics,name=
> TotalProduceRequestsPerSec,topic=dw_06b5f828-e452-4e22-89c9-67849a65603d).
>
> Also, deleting a topic often produces (what I think is) noise in the logs
> at WARN level, for example:
> 2017-01-13 14:52:46 WARN PartitionStateMachine$DeleteTopicsListener:83 -
> [DeleteTopicsListener on 1]: Ignoring request to delete non-existing topics
> dw_fe8ff14b-aa9b-4f24-9bc1-6fbce15d20d2
>
> This behaviour is easy to reproduce with a basic Kafka cluster with two
> brokers, just create and delete topics few times. Sometimes the MBeans for
> the topic are deleted and sometimes not.
>
> Maybe I'm deleting the topics in the wrong way? I'm using the AdminUtils
> class in the Java API:
> AdminUtils.deleteTopic(zkUtils, topicName);
>
> Topics are created using the AdminUtils class as well:
> AdminUtils.createTopic(zkUtils, topicName, partitions, replicationFactor,
> topicConfiguration, kafka.admin.RackAwareMode.Enforced$.MODULE$);
>
> The Kafka version is 0.10.0.1.
>
> Thanks,
> Alberto
>
>
> ******************************************************************
>
> NatWest Markets is a marketing name of The Royal Bank of Scotland plc.
>
> This communication and any attachments are confidential and intended
> solely for the addressee. If you are not the intended recipient please
> advise us immediately and delete it. Unless specifically stated in the
> message or otherwise indicated, you may not duplicate, redistribute or
> forward this message and any attachments are not intended for distribution
> to, or use by any person or entity in any jurisdiction or country where
> such distribution or use would be contrary to local law or regulation. The
> Royal Bank Of Scotland plc or any affiliated entity ("RBS") accepts no
> responsibility for any changes made to this message after it was sent.
>
> Unless otherwise specifically indicated, the contents of this
> communication and its attachments are for information purposes only and
> should not be regarded as an offer or solicitation to buy or sell a product
> or service, confirmation of any transaction, a valuation, indicative price
> or an official statement. Where this communication has been prepared by an
> RBS trading desk, that desk may have a position or interest in the products
> or services mentioned that is inconsistent with any views expressed in this
> message. In evaluating the information contained in this message, you
> should know that it could have been previously provided to other clients
> and/or internal RBS personnel, who could have already acted on it.
>
> RBS cannot provide absolute assurances that all electronic communications
> (sent or received) are secure, error free, not corrupted, incomplete or
> virus free and/or that they will not be lost, mis-delivered, destroyed,
> delayed or intercepted/decrypted by others. Therefore RBS disclaims all
> liability with regards to electronic communications (and the contents
> therein) if they are corrupted, lost destroyed, delayed, incomplete,
> mis-delivered, intercepted, decrypted or otherwise misappropriated by
> others.
>
> Any electronic communication that is conducted within or through RBS
> systems will be subject to being archived, monitored and produced to
> regulators and in litigation in accordance with RBS's policy and local
> laws, rules and regulations. Unless expressly prohibited by local law,
> electronic communications may be archived in countries other than the
> country in which you are located, and may be treated in accordance with the
> laws and regulations of the country of each individual included in the
> entire chain.
>
> Copyright 2014 The Royal Bank of Scotland plc. All rights reserved. See
> http://www.natwestmarkets.com/legal/s-t-discl.html for further risk
> disclosure.
>
> ******************************************************************
>
--
-- Guozhang
