Hey Venkat, The logs will help, but I'll probably need to see some code to understand what is going on (consumer usage can be a little subtle sometimes). You're welcome to open a ticket if you like or post here.
Thanks, Jason On Mon, Feb 22, 2016 at 5:09 PM, Venkatesan Subramanian < vsubraman...@lendingclub.com> wrote: > Jason: > > Thanks for your reply. > > I did turn on logging for kafka as well when I ran the test. I do not see > any partition assignment for the topic happening. > If you don’t see anything obviously wrong in what I am doing, I can file a > ticket with Confluent and attach the logs to it. > > Regards > Venkat > > > > On 2/19/16, 11:22 AM, "Jason Gustafson" <ja...@confluent.io> wrote: > > >Hi Venkatesan, > > > >Autocreation of topics happens when the broker receives a topic metadata > >request. That should mean that both topics get created when the consumer > >does the initial poll() since that is the first time that topic metadata > >would be fetched (fetching topic metadata allows the consumer and producer > >to determine the number of partitions in the topic and the current > >partition leaders). Are you sure that partitions from both topics have not > >been assigned at that time? > > > >Thanks, > >Jason > > > >On Fri, Feb 19, 2016 at 10:43 AM, Venkatesan Subramanian < > >vsubraman...@lendingclub.com> wrote: > > > >> I am running into a curious problem with auto creation of topics and > would > >> appreciate any insight into what could be causing it. > >> > >> Some context first: We are using Confluent 2.0 platform and I have an > >> integration test where the following happens: > >> > >> > >> 1. A worker thread is spun up and it creates a kafka consumer that > >> subscribes to two topics: Topic A and Topic B. Both of these topics do > not > >> exist at this time. > >> 2. The kafka consumer polls the two topics periodically. > >> 3. A kafka producer sends an event into Topic A (and hence creates > the > >> topic through auto topic creation) > >> 4. The kafka consumer created previously is assigned a partition from > >> Topic A (no assignment for Topic B since it does not exist yet). > >> 5. worker polls and receives the event from Topic A partition. It > >> processes the event and produces an event into Topic B (and hence the > topic > >> B is created at this time) > >> 6. I am expected to see a partition from Topic B getting assigned to > >> the consumer next time a poll happens but nothing happens. > >> > >> Note that the same integration test works fine if I pre-create the two > >> topics before running the test. > >> > >> ________________________________ > >> DISCLAIMER: The information transmitted is intended only for the person > or > >> entity to which it is addressed and may contain confidential and/or > >> privileged material. Any review, re-transmission, dissemination or other > >> use of, or taking of any action in reliance upon this information by > >> persons or entities other than the intended recipient is prohibited. If > you > >> received this in error, please contact the sender and destroy any > copies of > >> this document and any attachments. > >> >
