Hi Puneet, Yes you will need to bump up the maxMessageSize in server.KafkaConfig and fetchSize in consumer.ConsumerConfig.
server.KafkaConfig.maxMessageSize can be same as producer.ProducerConfig.maxMessageSize. You can set consumer.ConsumerConfig.fetchsize to a value greater than or equal to producer.ProducerConfig.maxMessageSize. Thanks, Swapnil On 9/24/12 3:51 PM, "Puneet Mehta" <mehta.p...@gmail.com> wrote: >Hi all, > >We are using kafka 0.7.1 > > I am seeing this error in the producer -> > >kafka.common.MessageSizeTooLargeException >at >kafka.producer.SyncProducer$$anonfun$kafka$producer$SyncProducer$$verifyMe >ssageSize$1.apply(SyncProducer.scala:141) >at >kafka.producer.SyncProducer$$anonfun$kafka$producer$SyncProducer$$verifyMe >ssageSize$1.apply(SyncProducer.scala:139) >at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:30) >at kafka.message.MessageSet.foreach(MessageSet.scala:87) >at >kafka.producer.SyncProducer.kafka$producer$SyncProducer$$verifyMessageSize >(SyncProducer.scala:139) >at kafka.producer.SyncProducer.send(SyncProducer.scala:113) >at >kafka.producer.ProducerPool$$anonfun$send$1.apply$mcVI$sp(ProducerPool.sca >la:116) >at >kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102) >at >kafka.producer.ProducerPool$$anonfun$send$1.apply(ProducerPool.scala:102) >at kafka.producer.ProducerPool.send(ProducerPool.scala:102) >at kafka.producer.Producer.zkSend(Producer.scala:143) >at kafka.producer.Producer.send(Producer.scala:105) >at kafka.javaapi.producer.Producer.send(Producer.scala:104) > >We are using the max message size as 1 <1000000>000000 <1000000> bytes. > >I am planning to bump this up to say 5 <5000000>000000 <5000000> bytes. > >I am just wondering do I need to change any other properties in >producer/broker/consumer that may get impacted by this bump up? > >Also, I came across this thread, which relates to fetch size relative to >max message size -> https://issues.apache.org/jira/browse/KAFKA-247 > >Could anyone of you advise me on places I may get impacted and change >accordingly? > > >Thanks, >Puneet Mehta
