Hmm.. that's right. completely forgot about that.
On Mon, Jan 26, 2015 at 01:49:33PM -0800, Jun Rao wrote:
> Joel,
>
> That's probably because console consumer always uses wildcard for
> consumption.
>
> Thanks,
>
> Jun
>
> On Mon, Jan 26, 2015 at 1:44 PM, Joel Koshy <jjkosh...@gmail.com> wrote:
>
> > I think this should work even on non-wildcard consumers. (i.e., I just
> > tried it locally and it appears to work fine).
> >
> > Here is what I did:
> >
> > - Leave a console consumer up and running, bounce the broker to turn
> > off auto-create
> > - Delete the topic
> > - The consumer should rebalance and stop consuming that topic
> > - Bounce the broker to turn on auto-create
> >
> > The only issue is that while your broker has auto-create off you will
> > see errors on some producers - which you can announce as a brief
> > outage to your users.
> >
> > Joel
> >
> > On Mon, Jan 26, 2015 at 01:02:57PM -0800, Jun Rao wrote:
> > > Actually, I think the deletion of a topic will trigger a rebalance on all
> > > wildcard subscribers. The time to complete the rebalance depends on the #
> > > of topic/partitions.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Jan 26, 2015 at 12:26 PM, Jason Rosenberg <j...@squareup.com>
> > wrote:
> > >
> > > > Yeah Joel,
> > > >
> > > > I just thought of that idea too (e.g. temporarily disable auto topic
> > > > creation). I can probably make that work (it may result in unexpected
> > lack
> > > > of topic creation, but I can message this as a maintenance downtime,
> > etc.).
> > > >
> > > > So, that was my next question, so a consumer rebalance will trigger a
> > fresh
> > > > topic pull from the consumers? How long is 'long enough' to ensure a
> > > > rebalance has occurred everywhere?
> > > >
> > > > Jason
> > > >
> > > > On Mon, Jan 26, 2015 at 3:07 PM, Joel Koshy <jjkosh...@gmail.com>
> > wrote:
> > > >
> > > > > Hey Jason,
> > > > >
> > > > > Is it an option for you to do the following:
> > > > >
> > > > > - Bounce in a config change to the brokers to turn off auto-create
> > > > > - (Batch)-delete the topic(s)
> > > > > - Wait long enough for consumers to rebalance (after which they will
> > > > > no longer consume the topic(s))
> > > > > - Bounce in a config change to the brokers to turn on auto-create
> > > > >
> > > > > Joel
> > > > >
> > > > > On Mon, Jan 26, 2015 at 12:01:59PM -0800, Jun Rao wrote:
> > > > > > Hi, Jason,
> > > > > >
> > > > > > I am not sure that we should delay the 0.8.2 release. The reasons
> > are
> > > > (1)
> > > > > > There are other features such as the new java producer and
> > Kafka-based
> > > > > > offset management that are potentially useful to people. (2) It may
> > > > take
> > > > > > some time to completely fix the issue with deleting topic since it
> > > > > involves
> > > > > > wire protocol changes.
> > > > > >
> > > > > > What we can do is probably just to document the limitation of
> > delete
> > > > > topic
> > > > > > in 0.8.2.0 in our site.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > > On Mon, Jan 26, 2015 at 11:05 AM, Jason Rosenberg <
> > j...@squareup.com>
> > > > > wrote:
> > > > > >
> > > > > > > I think this shortcoming should probably delay release of 0.8.2
> > until
> > > > > > > resolved, no?
> > > > > > >
> > > > > > > On Mon, Jan 26, 2015 at 2:01 PM, Guozhang Wang <
> > wangg...@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > It then seems to me that delete-topic will not actually work
> > > > > "smoothly"
> > > > > > > > until create topic request is added since it is too much to
> > require
> > > > > > > people
> > > > > > > > to turn off their clients while deleting topics. In this case
> > shall
> > > > > we
> > > > > > > make
> > > > > > > > it clear in the release docs of in 0.8.2, or even still mark
> > it as
> > > > > > > > not-supported?
> > > > > > > >
> > > > > > > > Guozhang
> > > > > > > >
> > > > > > > > On Mon, Jan 26, 2015 at 8:14 AM, Harsha <ka...@harsha.io>
> > wrote:
> > > > > > > >
> > > > > > > > > Jun,
> > > > > > > > > I made an attempt at fixing that issue as part of this
> > JIRA
> > > > > > > > > https://issues.apache.org/jira/browse/KAFKA-1507 .
> > > > > > > > > As Jay pointed out there should be admin api if there is more
> > > > info
> > > > > on
> > > > > > > > > this api I am interested in adding/fixing this issue.
> > > > > > > > > Thanks,
> > > > > > > > > Harsha
> > > > > > > > >
> > > > > > > > > On Mon, Jan 26, 2015, at 07:28 AM, Jun Rao wrote:
> > > > > > > > > > Yes, that's the issue. Currently, topics can be
> > auto-created on
> > > > > > > > > > TopicMetadataRequest, which can be issued from both the
> > > > producer
> > > > > and
> > > > > > > > the
> > > > > > > > > > consumer. To prevent that, you would need to stop the
> > producer
> > > > > and
> > > > > > > the
> > > > > > > > > > consumer before deleting a topic. We plan to address this
> > issue
> > > > > once
> > > > > > > we
> > > > > > > > > > have a separate request for creating topics.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > >
> > > > > > > > > > Jun
> > > > > > > > > >
> > > > > > > > > > On Mon, Jan 26, 2015 at 7:21 AM, Harsha <ka...@harsha.io>
> > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > There could be another case where if you have
> > > > > > > > auto.create.topics.enable
> > > > > > > > > > > to set to true ( its true by default) . Any
> > > > > TopicMetadataRequest
> > > > > > > can
> > > > > > > > > > > recreate topics. So if you issued a delete topic command
> > and
> > > > > you
> > > > > > > have
> > > > > > > > > > > producers running or consumers? too which is issuing a
> > > > > > > > > > > TopicMetadataRequest than the topic will be recreated.
> > > > > > > > > > > -Harsha
> > > > > > > > > > >
> > > > > > > > > > > On Sun, Jan 25, 2015, at 11:26 PM, Jason Rosenberg wrote:
> > > > > > > > > > > > cversion did change (incremented by 2) when I issue the
> > > > > delete
> > > > > > > > > command.
> > > > > > > > > > > >
> > > > > > > > > > > > From the logs on the conroller broker (also the leader
> > for
> > > > > the
> > > > > > > > > topic), it
> > > > > > > > > > > > looks like the delete proceeds, and then the topic gets
> > > > > recreated
> > > > > > > > > > > > immediately (highlighted in yellow). It appears maybe
> > it’s
> > > > > due
> > > > > > > to a
> > > > > > > > > > > > consumer client app trying to consume the topic. Also,
> > the
> > > > > > > consumer
> > > > > > > > > is
> > > > > > > > > > > > not
> > > > > > > > > > > > yet updated to 0.8.2 (it’s using 0.8.1.1), perhaps
> > that’s
> > > > > part of
> > > > > > > > the
> > > > > > > > > > > > problem?
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > 2015-01-26 07:02:14,281 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine$DeleteTopicsListener -
> > > > > > > > > > > > [DeleteTopicsListener on 6]: Starting topic deletion
> > for
> > > > > topics
> > > > > > > > > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,282 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Handling deletion for topics
> > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,286 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Deletion of topic mytopic
> > > > > (re)started
> > > > > > > > > > > > 2015-01-26 07:02:14,286 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager - [Topic Deletion
> > Manager
> > > > 6],
> > > > > > > Topic
> > > > > > > > > > > > deletion callback for mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,289 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager - [Topic Deletion
> > Manager
> > > > 6],
> > > > > > > > > > > > Partition deletion callback for [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,295 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to OfflineReplica for
> > replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,303 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New
> > leader and
> > > > > ISR
> > > > > > > for
> > > > > > > > > > > > partition [mytopic,0] is
> > > > > {"leader":6,"leader_epoch":1,"isr":[6]}
> > > > > > > > > > > > 2015-01-26 07:02:14,312 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New
> > leader and
> > > > > ISR
> > > > > > > for
> > > > > > > > > > > > partition [mytopic,0] is
> > > > > {"leader":-1,"leader_epoch":2,"isr":[]}
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to ReplicaDeletionStarted for
> > > > > replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-5]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7]
> > > > > > > > > > > > log.OffsetIndex - Deleting index
> > > > > > > > > > > > /mypath/mytopic-0/00000000000000000000.index
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-7]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Deleted log for partition [mytopic,0] in
> > > > /mypath/mytopic-0.
> > > > > > > > > > > > 2015-01-26 07:02:14,314 INFO
> > > > > > > > [Controller-6-to-broker-6-send-thread]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to ReplicaDeletionSuccessful
> > for
> > > > > > > replicas
> > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,314 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Handling deletion for topics
> > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,316 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Deletion for replicas 7 for
> > > > > partition
> > > > > > > > > > > > [mytopic,0] of topic mytopic in progress
> > > > > > > > > > > > 2015-01-26 07:02:14,316 INFO
> > > > > > > > [Controller-6-to-broker-7-send-thread]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to ReplicaDeletionSuccessful
> > for
> > > > > > > replicas
> > > > > > > > > > > > [Topic=mytopic,Partition=0,Replica=7]
> > > > > > > > > > > > 2015-01-26 07:02:14,316 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Handling deletion for topics
> > > > > mytopic
> > > > > > > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to NonExistentReplica for
> > > > replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > [Topic=mytopic,Partition=0,Replica=6],[Topic=mytopic,Partition=0,Replica=7]
> > > > > > > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to
> > OfflinePartition
> > > > for
> > > > > > > > > > > > partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,318 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to
> > > > NonExistentPartition
> > > > > for
> > > > > > > > > > > > partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,325 WARN [kafka-request-handler-2]
> > > > > > > > > > > > server.ReplicaManager - [Replica Manager on Broker 6]:
> > > > Fetch
> > > > > > > > request
> > > > > > > > > > > > with correlation id 2619442 from client
> > > > > ReplicaFetcherThread-0-6
> > > > > > > on
> > > > > > > > > > > > partition [mytopic,0] failed due to Partition
> > [mytopic,0]
> > > > > doesn't
> > > > > > > > > > > > exist on 6
> > > > > > > > > > > > 2015-01-26 07:02:14,371 INFO [delete-topics-thread-6]
> > > > > > > > > > > > controller.TopicDeletionManager$DeleteTopicsThread -
> > > > > > > > > > > > [delete-topics-thread-6], Deletion of topic mytopic
> > > > > successfully
> > > > > > > > > > > > completed
> > > > > > > > > > > > 2015-01-26 07:02:14,376 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine$TopicChangeListener -
> > > > > > > > > > > > [TopicChangeListener on Controller 6]: New topics:
> > [Set()],
> > > > > > > deleted
> > > > > > > > > > > > topics: [Set()], new partition replica assignment
> > [Map()]
> > > > > > > > > > > > 2015-01-26 07:02:14,403 WARN [request-expiration-task]
> > > > > > > > > > > > server.ReplicaManager - [Replica Manager on Broker 6]:
> > > > Fetch
> > > > > > > > request
> > > > > > > > > > > > with correlation id 2054626 from client
> > > > > > > > > > > > myconsumer-1415333962444-cc78e67e-0-6 on partition
> > > > > [mytopic,0]
> > > > > > > > failed
> > > > > > > > > > > > due to Partition [mytopic,0] doesn't exist on 6
> > > > > > > > > > > > 2015-01-26 07:02:14,446 INFO [kafka-request-handler-5]
> > > > > > > > > > > > admin.AdminUtils$ - Topic creation
> > > > > > > > > > > > {"version":1,"partitions":{"0":[7,6]}}
> > > > > > > > > > > > 2015-01-26 07:02:14,457 INFO [kafka-request-handler-5]
> > > > > > > > > > > > server.KafkaApis - [KafkaApi-6] Auto creation of topic
> > > > > mytopic
> > > > > > > > with 1
> > > > > > > > > > > > partitions and replication factor 2 is successful!
> > > > > > > > > > > > 2015-01-26 07:02:14,458 INFO
> > > > [kafka-network-thread-27330-2]
> > > > > > > > > > > > network.Processor - Closing socket connection to /
> > > > 10.1.10.4.
> > > > > > > > > > > > 2015-01-26 07:02:14,466 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine$TopicChangeListener -
> > > > > > > > > > > > [TopicChangeListener on Controller 6]: New topics:
> > > > > > > [Set(mytopic)],
> > > > > > > > > > > > deleted topics: [Set()], new partition replica
> > assignment
> > > > > > > > > > > > [Map([mytopic,0] -> List(7, 6))]
> > > > > > > > > > > > 2015-01-26 07:02:14,466 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New topic
> > > > > creation
> > > > > > > > > > > > callback for [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,466 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.KafkaController - [Controller 6]: New
> > partition
> > > > > > > creation
> > > > > > > > > > > > callback for [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,466 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to NewPartition
> > for
> > > > > > > partitions
> > > > > > > > > > > > [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,467 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to NewReplica for replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,468 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.PartitionStateMachine - [Partition state
> > machine
> > > > > on
> > > > > > > > > > > > Controller 6]: Invoking state change to
> > OnlinePartition for
> > > > > > > > > partitions
> > > > > > > > > > > > [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,530 INFO
> > > > > > > > > > > > [ZkClient-EventThread-21-myzkserver:12345/mynamespace]
> > > > > > > > > > > > controller.ReplicaStateMachine - [Replica state
> > machine on
> > > > > > > > controller
> > > > > > > > > > > > 6]: Invoking state change to OnlineReplica for replicas
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > [Topic=mytopic,Partition=0,Replica=7],[Topic=mytopic,Partition=0,Replica=6]
> > > > > > > > > > > > 2015-01-26 07:02:14,532 INFO [kafka-request-handler-5]
> > > > > log.Log -
> > > > > > > > > > > > Completed load of log mytopic-0 with log end offset 0
> > > > > > > > > > > > 2015-01-26 07:02:14,533 INFO [kafka-request-handler-5]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Created log for partition [mytopic,0] in /mypath with
> > > > > > > properties
> > > > > > > > > > > > {segment.index.bytes -> 10485760, file.delete.delay.ms
> > ->
> > > > > 60000,
> > > > > > > > > > > > segment.bytes -> 1073741824, flush.ms ->
> > > > > 9223372036854775807,
> > > > > > > > > > > > delete.retention.ms -> 86400000, index.interval.bytes
> > ->
> > > > > 4096,
> > > > > > > > > > > > retention.bytes -> 50000000000, min.insync.replicas ->
> > 1,
> > > > > > > > > > > > cleanup.policy -> delete,
> > unclean.leader.election.enable ->
> > > > > true,
> > > > > > > > > > > > segment.ms -> 604800000, max.message.bytes -> 1000012,
> > > > > > > > > flush.messages
> > > > > > > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio ->
> > 0.5,
> > > > > > > > > retention.ms
> > > > > > > > > > > > -> 86400000, segment.jitter.ms -> 0}.
> > > > > > > > > > > > 2015-01-26 07:02:14,540 INFO [kafka-request-handler-5]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,540 INFO [kafka-request-handler-5]
> > > > > log.Log -
> > > > > > > > > > > > Truncating log mytopic-0 to offset 0.
> > > > > > > > > > > > 2015-01-26 07:02:14,547 INFO [kafka-request-handler-5]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 6]
> > > > > > > > > > > > Added fetcher for partitions List([[mytopic,0],
> > initOffset
> > > > 0
> > > > > to
> > > > > > > > > broker
> > > > > > > > > > > > id:7,host:mybroker:54321] )
> > > > > > > > > > > >
> > > > > > > > > > > > Here’s the log in the other broker with a replica of
> > the
> > > > > topic.
> > > > > > > > > > > >
> > > > > > > > > > > > 2015-01-26 07:02:14,312 INFO [kafka-request-handler-7]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 7]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 7]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2]
> > > > > > > > > > > > log.OffsetIndex - Deleting index
> > > > > > > > > > > > /mypath/mytopic-0/00000000000000000000.index
> > > > > > > > > > > > 2015-01-26 07:02:14,313 INFO [kafka-request-handler-2]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Deleted log for partition [mytopic,0] in
> > > > /mypath/mytopic-0.
> > > > > > > > > > > > 2015-01-26 07:02:14,530 INFO [kafka-request-handler-0]
> > > > > > > > > > > > server.ReplicaFetcherManager - [ReplicaFetcherManager
> > on
> > > > > broker
> > > > > > > 7]
> > > > > > > > > > > > Removed fetcher for partitions [mytopic,0]
> > > > > > > > > > > > 2015-01-26 07:02:14,532 INFO [kafka-request-handler-0]
> > > > > log.Log -
> > > > > > > > > > > > Completed load of log mytopic-0 with log end offset 0
> > > > > > > > > > > > 2015-01-26 07:02:14,533 INFO [kafka-request-handler-0]
> > > > > > > > > log.LogManager
> > > > > > > > > > > > - Created log for partition [mytopic,0] in /mypath with
> > > > > > > properties
> > > > > > > > > > > > {segment.index.bytes -> 10485760, file.delete.delay.ms
> > ->
> > > > > 60000,
> > > > > > > > > > > > segment.bytes -> 1073741824, flush.ms ->
> > > > > 9223372036854775807,
> > > > > > > > > > > > delete.retention.ms -> 86400000, index.interval.bytes
> > ->
> > > > > 4096,
> > > > > > > > > > > > retention.bytes -> 50000000000, min.insync.replicas ->
> > 1,
> > > > > > > > > > > > cleanup.policy -> delete,
> > unclean.leader.election.enable ->
> > > > > true,
> > > > > > > > > > > > segment.ms -> 604800000, max.message.bytes -> 1000012,
> > > > > > > > > flush.messages
> > > > > > > > > > > > -> 9223372036854775807, min.cleanable.dirty.ratio ->
> > 0.5,
> > > > > > > > > retention.ms
> > > > > > > > > > > > -> 86400000, segment.jitter.ms -> 0}.
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, Jan 26, 2015 at 1:26 AM, Jun Rao <
> > j...@confluent.io
> > > > >
> > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Could you do a get on /admin/deleted_topics and see if
> > its
> > > > > > > cversion
> > > > > > > > > > > > changes
> > > > > > > > > > > > > after issuing the delete topic command? This will
> > tell us
> > > > > > > whether
> > > > > > > > > the
> > > > > > > > > > > > > path /admin/deleted_topics//mytopic
> > > > > > > > > > > > > is actually created or not. Anything in the
> > controller
> > > > log
> > > > > that
> > > > > > > > > > > indicates a
> > > > > > > > > > > > > delete topic event is fired?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Jun
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Sun, Jan 25, 2015 at 9:55 PM, Jason Rosenberg <
> > > > > > > > j...@squareup.com
> > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > yes
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Mon, Jan 26, 2015 at 12:18 AM, Jun Rao <
> > > > > j...@confluent.io>
> > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Do you have delete.topic.enable turned on in all
> > > > > brokers?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Jun
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Sun, Jan 25, 2015 at 7:56 PM, Jason Rosenberg
> > <
> > > > > > > > > j...@squareup.com
> > > > > > > > > > > >
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > So far, I have been unable to get delete topic
> > to
> > > > > work,
> > > > > > > > with
> > > > > > > > > > > release
> > > > > > > > > > > > > > > > candidate 2 for 0.8.2.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > It worked ok when I ran it in the debugger
> > locally,
> > > > > on a
> > > > > > > > > single
> > > > > > > > > > > node
> > > > > > > > > > > > > > > > instance. But when I run it in our staging
> > > > > environment,
> > > > > > > it
> > > > > > > > > is not
> > > > > > > > > > > > > > > > successfully even marking the topic for
> > delete, for
> > > > > some
> > > > > > > > > reason.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I am setting delete.topic.enable to true (and
> > see
> > > > the
> > > > > > > > > > > confirmation of
> > > > > > > > > > > > > > > this
> > > > > > > > > > > > > > > > in the startup logs, e.g.):
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > INFO [main] utils.VerifiableProperties -
> > Property
> > > > > > > > > > > delete.topic.enable
> > > > > > > > > > > > > > > > is overridden to true
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I run this command:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand
> > > > --zookeeper
> > > > > > > > > > > > > > > > myzkconnect:12345/mynamespace --delete --topic
> > > > > mytopic
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection).
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread).
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > Topic mytopic is marked for deletion.
> > > > > > > > > > > > > > > > Note: This will have no impact if
> > > > > delete.topic.enable is
> > > > > > > > not
> > > > > > > > > set
> > > > > > > > > > > to
> > > > > > > > > > > > > > true.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I then do a —list which should at least show
> > the
> > > > > topic
> > > > > > > > > marked for
> > > > > > > > > > > > > > > deletion:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > java -cp app.jar kafka.admin.TopicCommand
> > > > --zookeeper
> > > > > > > > > > > > > > > > myzkconnect:12345/mynamespace --list --topic
> > > > mytopic
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkConnection).
> > > > > > > > > > > > > > > > log4j:WARN No appenders could be found for
> > logger
> > > > > > > > > > > > > > > > (org.I0Itec.zkclient.ZkEventThread).
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN Please initialize the log4j system
> > > > > properly.
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > log4j:WARN See
> > > > > > > > > > > http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > > > > > > > > > > > > > > > for more info.
> > > > > > > > > > > > > > > > mytopic
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Note, it doesn’t list it as ‘marked for
> > deletion’.
> > > > > > > > > Furthermore,
> > > > > > > > > > > > > waiting
> > > > > > > > > > > > > > > > multiple hours still doesn’t result in the
> > topic
> > > > > being
> > > > > > > > > deleted.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The topic has 1 partition, 2 replicas, and no
> > data
> > > > > > > stored.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > In zookeeper, the /admin/deleted_topics/ path
> > is
> > > > > empty.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > The zookeeper code looks pretty
> > straightforward,
> > > > but
> > > > > for
> > > > > > > > some
> > > > > > > > > > > reason
> > > > > > > > > > > > > is
> > > > > > > > > > > > > > > not
> > > > > > > > > > > > > > > > writing the deleted_topics path. We are
> > running
> > > > > > > zookeeper
> > > > > > > > > 3.4.6.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thoughts?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Jason
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > -- Guozhang
> > > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > >
> >
> >
