Hi Grant, Thanks a lot for the explanation.This is great. It was little confusing with KIP-4 and KAFKA-1507 both talking about the somewhat same thing and KAFKA-1507 has been lying around since last August and not checked in or iterated on.
Here is what I understand is : 1) KAFKA-2945 will implement the CreateTopic Request that will block on the server until the topic is successfully created. This will return success or should return failure on timeout. 2) Once this is done, whether we want to have a config in Kafka Producer and Consumer OR we want to let the users create the topic explicitly using the createTopicRequest API is still under discussion. Am I right on both the above points? The reason I want to confirm this is because it is very important for us when we want to delete topics in a pipeline and there are several clients producing, consuming and mirror-makers and monitoring/auditing systems running at the same time. Thanks, Mayuresh On Fri, Jan 8, 2016 at 8:41 AM, Grant Henke <ghe...@cloudera.com> wrote: > Hi Mayuresh, > > I am working on KIP-4 and the create topic wire protocol is a part of that. > I have a patch but it needs to be changed to be synchronous on the server > side yet. You can see the discussion on the pull request. > That work can be tracked here: > https://issues.apache.org/jira/browse/KAFKA-2945 > > Once the create topic work is done, clients can then create missing topics > themselves and we wont depend on the server auto creating them. the client, > auto creation could be baked into the client, or users could use the error > codes to make the call themselves, that work and details has not been > vetted yet. > There is a jira to track that here: > https://issues.apache.org/jira/browse/KAFKA-2410 > > Hope that helps. > > Thanks, > Grant > > On Thu, Jan 7, 2016 at 6:27 PM, Mayuresh Gharat < > gharatmayures...@gmail.com> > wrote: > > > Hi, > > > > I might have missed out the discussions on this :( > > > > I had some more questions : > > > > 1) We need a config in KafkaProducer for this. So when the KafkaProducer > > issues a TMR for a topic and receives a response that the topic does not > > exist, depending on the value of this config it should use the > CreateTopic > > request to create the topic or exit. Is there a ticket for this? > > > > 2) We need a config in KafkaConsumer for this. So when the > > KafkaConsumer(old and new) issues a TMR for a topic and receives a > response > > that the topic does not exist, depending on the value of this config it > > should use the CreateTopic request to create the topic or exit. Is there > a > > ticket for this? > > > > Thanks, > > > > Mayuresh > > > > On Thu, Jan 7, 2016 at 3:27 PM, Mayuresh Gharat < > > gharatmayures...@gmail.com> > > wrote: > > > > > Hi Jason, > > > > > > Thanks. I had found this ticket before but the KAFKA-1507 also has some > > > context about this and I was confused basically exactly which patch is > > > going to go in. > > > Thanks a lot for confirming. > > > > > > Thanks, > > > > > > Mayuresh > > > > > > On Thu, Jan 7, 2016 at 3:08 PM, Jason Gustafson <ja...@confluent.io> > > > wrote: > > > > > >> Hey Mayuresh, > > >> > > >> The ticket that Grant Henke has been working on is here: > > >> https://issues.apache.org/jira/browse/KAFKA-2945. Is that what you > were > > >> looking for? > > >> > > >> -Jason > > >> > > >> On Thu, Jan 7, 2016 at 1:50 PM, Mayuresh Gharat < > > >> gharatmayures...@gmail.com> > > >> wrote: > > >> > > >> > + dev > > >> > > > >> > Hi > > >> > > > >> > There has been discussion on the ticket : > > >> > https://issues.apache.org/jira/browse/KAFKA-2887, that we are going > > to > > >> > deprecate auto-creation of topic or at least turn it off by default > > >> once we > > >> > have the CreateTopics API. It also says the patch is available for > > this. > > >> > > > >> > The only other ticket that I came across on the ticket is > > >> > https://issues.apache.org/jira/browse/KAFKA-1507. > > >> > > > >> > I wanted to confirm that > > >> https://issues.apache.org/jira/browse/KAFKA-1507 > > >> > is the ticket that has the CreateTopics API patch. > > >> > <%28862%29%20250-7125> > > >> > > > >> > > > >> > -- > > >> > -Regards, > > >> > Mayuresh R. Gharat > > >> > (862) 250-7125 > > >> > > > >> > > > > > > > > > > > > -- > > > -Regards, > > > Mayuresh R. Gharat > > > (862) 250-7125 > > > > > > > > > > > -- > > -Regards, > > Mayuresh R. Gharat > > (862) 250-7125 > > > > > > -- > Grant Henke > Software Engineer | Cloudera > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > -- -Regards, Mayuresh R. Gharat (862) 250-7125
