Thanks Kumar, for explaining in depth. On Mon, Aug 10, 2015 at 1:37 PM, Manikumar Reddy <[email protected]> wrote:
> Hi, > > > > > 1. Will Kafka distribute the 100 serialized files randomly say 20 files > go > > to Partition 1, 25 to Partition 2 etc or do I have an option to > configure > > how many files go to which partition . > > > > Assuming you are using new producer, > > All keyed messages will be distributed based on the hash of the key (or) > All non-keyed message will be distributed in round-robin fashion. > (or) You can specify partition number for each message using > "ProducerRecord" object. > > > > 2. How to configure each consumer to read from a particular partition > > only. Say consumer 1 to read from partition 4 only likewise. > > > > > You can use SimpleConsumer to consumer from a particular partition. > https://kafka.apache.org/documentation.html#simpleconsumerapi > > > > 2. If I have not set the max size what will happen to that 1MB file, I > read > > in the documentation that default size is 10KB. > > > Default "max.request.size" on new producer and "message.max.bytes" on > broker is 1MB. > Anything above these limits, you will get exception. > > > Kumar >
