Re: Partition and consumer configuration

2015-08-10 Thread Manikumar Reddy
Hi, 1. Will Kafka distribute the 100 serialized files randomly say 20 files go to Partition 1, 25 to Partition 2 etc or do I have an option to configure how many files go to which partition . Assuming you are using new producer, All keyed messages will be distributed based on the

Re: Partition and consumer configuration

2015-08-10 Thread Shashidhar Rao
Thanks Kumar, for explaining in depth. On Mon, Aug 10, 2015 at 1:37 PM, Manikumar Reddy ku...@nmsworks.co.in wrote: Hi, 1. Will Kafka distribute the 100 serialized files randomly say 20 files go to Partition 1, 25 to Partition 2 etc or do I have an option to configure how many

Partition and consumer configuration

2015-08-10 Thread Shashidhar Rao
Hi, Could somebody help me whether my understanding is correct as I am very new to kafka. 1. Topic name- ProdCategory, with 4 partitions. All the messages are XML files . And consumer numbers also 4. Multi-Broker -4. 2. XML files vary in size from 10 KB- 1 MB. 3. Say if there are 100 XML