Re: Issue with topic deletion

2015-02-05 Thread Joel Koshy
Thanks will take a look.. On Wed, Feb 04, 2015 at 11:33:03PM -0800, Sumit Rangwala wrote: > > > > Any idea why you have session expirations? This is typically due to GC > >> and/or flaky network. Regardless, we should be handling that scenario > >> as well. However, your logs seem incomplete. Can

Re: Issue with topic deletion

2015-02-04 Thread Sumit Rangwala
> > Any idea why you have session expirations? This is typically due to GC >> and/or flaky network. Regardless, we should be handling that scenario >> as well. However, your logs seem incomplete. Can you redo this and >> perhaps keep the set up running a little longer and send over those >> logs? >

Re: Issue with topic deletion

2015-02-04 Thread Sumit Rangwala
On Wed, Feb 4, 2015 at 10:16 PM, Harsha wrote: > >whats your zookeeper.session.timeout.ms value > 3 (30sec) Sumit > > On Wed, Feb 4, 2015, at 09:35 PM, Sumit Rangwala wrote: > > On Wed, Feb 4, 2015 at 6:14 PM, Joel Koshy wrote: > > > > > I took a look at your logs. I agree with

Re: Issue with topic deletion

2015-02-04 Thread Harsha
whats your zookeeper.session.timeout.ms value On Wed, Feb 4, 2015, at 09:35 PM, Sumit Rangwala wrote: > On Wed, Feb 4, 2015 at 6:14 PM, Joel Koshy wrote: > > > I took a look at your logs. I agree with Harsh that the logs seem > > truncated. The basic issue though is that you have session

Re: Issue with topic deletion

2015-02-04 Thread Sumit Rangwala
On Wed, Feb 4, 2015 at 6:14 PM, Joel Koshy wrote: > I took a look at your logs. I agree with Harsh that the logs seem > truncated. The basic issue though is that you have session expirations > and controller failover. Broker 49554 was the controller and hosted > some partition(s) of LAX1-GRIFFIN-

Re: Issue with topic deletion

2015-02-04 Thread Joel Koshy
I took a look at your logs. I agree with Harsh that the logs seem truncated. The basic issue though is that you have session expirations and controller failover. Broker 49554 was the controller and hosted some partition(s) of LAX1-GRIFFIN-r13-1423001701601. After controller failover the new control

Re: Issue with topic deletion

2015-02-04 Thread Sumit Rangwala
> > >> I have since stopped the container so I cannot say if > LAX1-GRIFFIN-r45-142388317 was one of the topic in "marked for > deletion" forever. However, there were many topics (at least 10 of them) > that were perennially in "marked for deletion" state. > > I have the setup to recreate the

Re: Issue with topic deletion

2015-02-03 Thread Sumit Rangwala
On Tue, Feb 3, 2015 at 9:47 PM, Harsha wrote: > Sumit, > I grepped logs for this topic "LAX1-GRIFFIN-r13-1423001701601" > it looks like topic partitions are getting deleted in > state-change.log and this happens around 22:59 and server.log > has data till 22:5

Re: Issue with topic deletion

2015-02-03 Thread Sumit Rangwala
On Tue, Feb 3, 2015 at 9:47 PM, Harsha wrote: > Sumit, > I grepped logs for this topic "LAX1-GRIFFIN-r13-1423001701601" > it looks like topic partitions are getting deleted in > state-change.log and this happens around 22:59 and server.log > has data till 22:5

Re: Issue with topic deletion

2015-02-03 Thread Sumit Rangwala
On Tue, Feb 3, 2015 at 9:19 PM, Harsha wrote: > you are probably handling it but there is a case where you call > deleteTopic and kafka goes through delete topic process but your > consumer is running probably made a TopicMetadataRequest for the same > topic which can re-create the topic with the

Re: Issue with topic deletion

2015-02-03 Thread Harsha
Sumit, I grepped logs for this topic "LAX1-GRIFFIN-r13-1423001701601" it looks like topic partitions are getting deleted in state-change.log and this happens around 22:59 and server.log has data till 22:59. I looked for other deleted topic "LAX1-GRIFFIN-r45-1423

Re: Issue with topic deletion

2015-02-03 Thread Harsha
you are probably handling it but there is a case where you call deleteTopic and kafka goes through delete topic process but your consumer is running probably made a TopicMetadataRequest for the same topic which can re-create the topic with the default num.partitions and replication.factor. Did you

Re: Issue with topic deletion

2015-02-03 Thread Sumit Rangwala
On Tue, Feb 3, 2015 at 6:48 PM, Harsha wrote: > Sumit, >lets say you are deleting a older topic "test1" do you have any >consumers running simultaneously for the topic "test1" while >deletion of topic going on. > Yes it is the case. However, after a small period of time

Re: Issue with topic deletion

2015-02-03 Thread Harsha
Sumit, lets say you are deleting a older topic "test1" do you have any consumers running simultaneously for the topic "test1" while deletion of topic going on. -Harsha On Tue, Feb 3, 2015, at 06:17 PM, Joel Koshy wrote: > Thanks for the logs - will take a look tomorrow unless

Re: Issue with topic deletion

2015-02-03 Thread Joel Koshy
Thanks for the logs - will take a look tomorrow unless someone else gets a chance to get to it today. Joel On Tue, Feb 03, 2015 at 04:11:57PM -0800, Sumit Rangwala wrote: > On Tue, Feb 3, 2015 at 3:37 PM, Joel Koshy wrote: > > > Hey Sumit, > > > > I thought you would be providing the actual ste

Re: Issue with topic deletion

2015-02-03 Thread Sumit Rangwala
On Tue, Feb 3, 2015 at 3:37 PM, Joel Koshy wrote: > Hey Sumit, > > I thought you would be providing the actual steps to reproduce :) > I want to but some proprietary code prevents me to do it. > Nevertheless, can you get all the relevant logs: state change logs and > controller logs at the ver

Re: Issue with topic deletion

2015-02-03 Thread Joel Koshy
Hey Sumit, I thought you would be providing the actual steps to reproduce :) Nevertheless, can you get all the relevant logs: state change logs and controller logs at the very least and if possible server logs and send those over? Joel On Tue, Feb 03, 2015 at 03:27:43PM -0800, Sumit Rangwala wro

Issue with topic deletion

2015-02-03 Thread Sumit Rangwala
In my setup kafka brokers are set for auto topic creation. In the scenario below a node informs other nodes (currently 5 in total) about a number of new (non-existent) topics, and all the nodes almost simultaneously open a consumer for each of those topics. Sometime later another node informs all