Hi Zuoning, since compression is a per-message(set) attribute a topic can have both compressed and uncompressed messages, as Guozhang says, and yes, this is supported by both the broker and client (librdkafka in this case).
Regards, Magnus 2014-10-31 17:14 GMT+01:00 Zuoning Yin <zuoning....@gmail.com>: > Hi Guozhang, > Just want to double check: does this have some requirement on the Kafka > version (we are using 0.8.0) or the kafka client (we are using librdkafka)? > > Thanks, > --Zuoning > > On Fri, Oct 31, 2014 at 12:09 PM, Zuoning Yin <zuoning....@gmail.com> > wrote: > > > Thanks so much for the reply, Guozhang! > > > > On Fri, Oct 31, 2014 at 12:06 PM, Guozhang Wang <wangg...@gmail.com> > > wrote: > > > >> Hi Zuoning, > >> > >> Yes, the same topic's partitions can hold both compressed and > uncompressed > >> data, and the consumer is able to read them in mixed mode. > >> > >> Guozhang > >> > >> On Fri, Oct 31, 2014 at 8:40 AM, Zuoning Yin <zuoning....@gmail.com> > >> wrote: > >> > >> > Hey Guys, > >> > If at the beginning, I configure the producer to not use > >> compression > >> > and produce a number of message to a topic. Then later, I reconfigure > >> the > >> > producer to use compression and produce another batch of messages to > the > >> > same topic. Can a consumer correctly read all the messages in the > topic? > >> > Will the consumer stop reading correctly when it reach the compressed > >> part? > >> > > >> > Thanks so much, > >> > --Zuoning > >> > > >> > >> > >> > >> -- > >> -- Guozhang > >> > > > > >
