Hi David, Yes, you're right. I've bumped the version of record batch, and describe the down-conversion will happen like what we do for message format v1 now when old consumers consuming records.
> Overall, I wonder if the bandwidth saving is worth this change given that it will put more pressure on the brokers. Actually, I'm not 100% sure. So I'd also like to hear what the community thought about it. But if the producers and consumers all existed in the same organization, which means upgrading producers/consumers for the org's cost saving, should be a reasonable motivation. Thanks. Luke On Fri, May 12, 2023 at 3:43 PM David Jacot <dja...@confluent.io.invalid> wrote: > Hi Luke, > > Thanks for the KIP. > > What do we do in the case where a batch is written with > `ignoreMessageAttributes` set to 1, which means that messages won't have > the `attributes`, and is consumed by a consumer which does not understand > this new format? I suppose that we would need to introduce a new version > for the message format (v3) and that we will have to downconvert records > from the new format version to v2 in this case. This is not clear in the > KIP. Could you elaborate a bit more on this? Overall, I wonder if the > bandwidth saving is worth this change given that it will put more pressure > on the brokers. > > Best, > David > > On Fri, May 12, 2023 at 9:04 AM Luke Chen <show...@gmail.com> wrote: > > > Hi all, > > > > I'd like to start a discussion for the KIP-931: Flag to ignore unused > > message attribute field. This KIP is to add a flag in the batch header to > > indicate if messages inside the batch have attribute field or not, to > > reduce the message size, thus, save network traffic and storage size (and > > money, of course). > > > > Please check the link for more detail: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-931%3A+Flag+to+ignore+unused+message+attribute+field > > > > Any feedback is welcome. > > > > Thank you. > > Luke > > >
