Hi David, I know what you mean. Let's hear what others' thoughts about it. :)
Luke On Fri, May 12, 2023 at 4:53 PM David Jacot <dja...@confluent.io.invalid> wrote: > Thanks, Luke. > > > But if the producers and consumers all existed in the same organization, > which means upgrading producers/consumers for the org's cost saving, should > be a reasonable motivation. > > Yeah, that works in this case. However, Kafka is often used as a service > (on premise or in cloud) nowadays and in this case the producers/consumers > versions are completely out of control thus my concern. > > BR, > David > > On Fri, May 12, 2023 at 10:47 AM Luke Chen <show...@gmail.com> wrote: > > > Hi David, > > > > Yes, you're right. I've bumped the version of record batch, and describe > > the down-conversion will happen like what we do for message format v1 now > > when old consumers consuming records. > > > > > Overall, I wonder if the bandwidth saving is worth this change given > that > > it will put more pressure on the brokers. > > Actually, I'm not 100% sure. So I'd also like to hear what the community > > thought about it. > > But if the producers and consumers all existed in the same organization, > > which means upgrading producers/consumers for the org's cost saving, > should > > be a reasonable motivation. > > > > Thanks. > > Luke > > > > > > On Fri, May 12, 2023 at 3:43 PM David Jacot <dja...@confluent.io.invalid > > > > wrote: > > > > > Hi Luke, > > > > > > Thanks for the KIP. > > > > > > What do we do in the case where a batch is written with > > > `ignoreMessageAttributes` set to 1, which means that messages won't > have > > > the `attributes`, and is consumed by a consumer which does not > understand > > > this new format? I suppose that we would need to introduce a new > version > > > for the message format (v3) and that we will have to downconvert > records > > > from the new format version to v2 in this case. This is not clear in > the > > > KIP. Could you elaborate a bit more on this? Overall, I wonder if the > > > bandwidth saving is worth this change given that it will put more > > pressure > > > on the brokers. > > > > > > Best, > > > David > > > > > > On Fri, May 12, 2023 at 9:04 AM Luke Chen <show...@gmail.com> wrote: > > > > > > > Hi all, > > > > > > > > I'd like to start a discussion for the KIP-931: Flag to ignore unused > > > > message attribute field. This KIP is to add a flag in the batch > header > > to > > > > indicate if messages inside the batch have attribute field or not, to > > > > reduce the message size, thus, save network traffic and storage size > > (and > > > > money, of course). > > > > > > > > Please check the link for more detail: > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-931%3A+Flag+to+ignore+unused+message+attribute+field > > > > > > > > Any feedback is welcome. > > > > > > > > Thank you. > > > > Luke > > > > > > > > > >
