Thanks Omer for answer. It helps to understand. Couple of follow up questions
1. Is there a way to confirm this using any kafka logs (message with encrypted info). 2. Is this error intermittent or happens always? Because I do see plaintext calls succeed for the same client and topic Regards, Nanda -----Original Message----- From: Ömer Şiar Baysal <osiarbay...@gmail.com> Sent: Thursday, May 29, 2025 1:19 PM To: users@kafka.apache.org Cc: rradu...@gmail.com Subject: Re: [EXTERNAL] Re: What config to avoid outofmemory exceptions happening in kafka server logs Hi, This is a known issue, https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-4493 Basically when broker receives first few bytes of the encrypted message on its plaintext listener, it may end up trying to allocate a huge buffer for the message, because it assumes the received bytes are for plaintext listener. Since encrypted message is no longer a clear text, its first 4 bytes can resemble anything (sometimes a huge number) thus resulting in OOM. Hope that clears it up for you. Ömer Şiar Baysal On Thu, May 29, 2025, 18:18 Nanda Naga <nandan...@microsoft.com.invalid> wrote: > Thanks Radu for the response. > > Wondering why it is out of memory? Any config to resolve this or any > change in client to resolve this? > > Also, we use librdkafka client to connect to plaintext port. The > memory errors seems intermittent as I see produces happening to plaintext as > well. > > Regards, > Nanda > > -----Original Message----- > From: Radu Radutiu <rradu...@gmail.com> > Sent: Wednesday, May 28, 2025 3:08 AM > To: users@kafka.apache.org > Subject: [EXTERNAL] Re: What config to avoid outofmemory exceptions > happening in kafka server logs > > [You don't often get email from rradu...@gmail.com. Learn why this is > important at https://aka.ms/LearnAboutSenderIdentification ] > > The error usually occurs when you try to connect with a client > speaking TLS to the plain text port of the Kafka broker. > > Radu > > On Wed, May 28, 2025 at 12:37 AM Nanda Naga <nandan...@microsoft.com > .invalid> > wrote: > > > I see this exception in our environment in multiple machines. What > > config will help to resolve this error? > > > > The exception I get is at > > > > java.lang.OutOfMemoryError: Unable to allocate 369295624 bytes at > > java.base/jdk.internal.misc.Unsafe.allocateMemory(Unsafe.java:632) > > at > > java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:125 > > ) at > > java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:332) > > at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:228) > > at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:293) > > at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:259) > > at > > java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:4 > > 17 > > ) > > at > > org.apache.kafka.common.network.PlaintextTransportLayer.read(Plainte > > xt > > TransportLayer.java:103) > > at > > org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkRecei > > ve > > .java:118) > > at > > org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.ja > > va > > :452) at > > org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java: > > 40 > > 2) at > > org.apache.kafka.common.network.Selector.attemptRead(Selector.java:6 > > 74 > > ) > > at > > org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector. > > ja > > va:576) at > > org.apache.kafka.common.network.Selector.poll(Selector.java:481) > > at kafka.network.Processor.poll(SocketServer.scala:1144) > > at kafka.network.Processor.run(SocketServer.scala:1047) > > at java.base/java.lang.Thread.run(Thread.java:840) > > > > Regards, > > Nanda > > > > >
