[
https://issues.apache.org/jira/browse/KAFKA-5814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Petrovsky updated KAFKA-5814:
---------------------------------------
Description:
Hello!
We have in our production
- kafka_2.12-0.10.2.1
- # java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)
And we had strange behavior with kafka which result to big outage:
{code:java}
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: [2017-08-30
21:17:19,266] ERROR Processor got uncaught exception. (kafka.network.Processor)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]:
java.lang.OutOfMemoryError: Direct buffer memory
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.nio.Bits.reserveMemory(Bits.java:694)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
sun.nio.ch.IOUtil.read(IOUtil.java:195)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:355)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.Selector.poll(Selector.java:303)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
kafka.network.Processor.poll(SocketServer.scala:494)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
kafka.network.Processor.run(SocketServer.scala:432)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.lang.Thread.run(Thread.java:748)
{code}
Only kafka cluster restart helps to fix this problem.
I wonder, why this problem have take a place, cause we running kafka with
follow options Environment="KAFKA_HEAP_OPTS=-Xmx5G -Xms5G
-XX:MaxDirectMemorySize=2G"
was:
Hello!
We have in our production
- kafka_2.12-0.10.2.1
- # java -version
openjdk version "1.8.0_131"
OpenJDK Runtime Environment (build 1.8.0_131-b12)
OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)
And we had strange behavior with kafka with result to big outage:
{code:java}
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: [2017-08-30
21:17:19,266] ERROR Processor got uncaught exception. (kafka.network.Processor)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]:
java.lang.OutOfMemoryError: Direct buffer memory
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.nio.Bits.reserveMemory(Bits.java:694)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
sun.nio.ch.IOUtil.read(IOUtil.java:195)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:355)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
org.apache.kafka.common.network.Selector.poll(Selector.java:303)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
kafka.network.Processor.poll(SocketServer.scala:494)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
kafka.network.Processor.run(SocketServer.scala:432)
авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
java.lang.Thread.run(Thread.java:748)
{code}
Only kafka cluster restart helps to fix this problem.
I wonder, why this problem have take a place, cause we running kafka with
follow options Environment="KAFKA_HEAP_OPTS=-Xmx5G -Xms5G
-XX:MaxDirectMemorySize=2G"
> java.lang.OutOfMemoryError: Direct buffer memory
> ------------------------------------------------
>
> Key: KAFKA-5814
> URL: https://issues.apache.org/jira/browse/KAFKA-5814
> Project: Kafka
> Issue Type: Bug
> Components: core
> Reporter: Alexander Petrovsky
>
> Hello!
> We have in our production
> - kafka_2.12-0.10.2.1
> - # java -version
> openjdk version "1.8.0_131"
> OpenJDK Runtime Environment (build 1.8.0_131-b12)
> OpenJDK 64-Bit Server VM (build 25.131-b12, mixed mode)
> And we had strange behavior with kafka which result to big outage:
> {code:java}
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: [2017-08-30
> 21:17:19,266] ERROR Processor got uncaught exception.
> (kafka.network.Processor)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]:
> java.lang.OutOfMemoryError: Direct buffer memory
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> java.nio.Bits.reserveMemory(Bits.java:694)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> sun.nio.ch.IOUtil.read(IOUtil.java:195)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:110)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:97)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:169)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:150)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:355)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> org.apache.kafka.common.network.Selector.poll(Selector.java:303)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> kafka.network.Processor.poll(SocketServer.scala:494)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> kafka.network.Processor.run(SocketServer.scala:432)
> авг 30 21:17:19 2kafka01.rtty.in kafka-server-start.sh[97716]: at
> java.lang.Thread.run(Thread.java:748)
> {code}
> Only kafka cluster restart helps to fix this problem.
> I wonder, why this problem have take a place, cause we running kafka with
> follow options Environment="KAFKA_HEAP_OPTS=-Xmx5G -Xms5G
> -XX:MaxDirectMemorySize=2G"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
