bq. we can perform down-conversion when Records.writeTo is called. Wouldn't this delay the network thread (though maybe the duration is short) ?
Can you expand on the structure of LazyDownConvertedRecords in more detail ? bq. even if it exceeds fetch.max.bytes I did a brief search but didn't see the above config. Did you mean message.max.bytes ? bq. with possibility to grow if the allocation After the buffers grow, is there a way to trim them down if subsequent down-conversion doesn't need that much memory ? Thanks On Fri, Apr 6, 2018 at 2:56 PM, Dhruvil Shah <dhru...@confluent.io> wrote: > Hi, > > I created a KIP to help mitigate out of memory issues during > down-conversion. The KIP proposes introducing a configuration that can > prevent down-conversions altogether, and also describes a design for > efficient memory usage for down-conversion. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 283%3A+Efficient+Memory+Usage+for+Down-Conversion > > Suggestions and feedback are welcome! > > Thanks, > Dhruvil >
