[
https://issues.apache.org/jira/browse/ARTEMIS-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17308709#comment-17308709
]
mahendra anil sonawale edited comment on ARTEMIS-2623 at 3/25/21, 2:21 PM:
---------------------------------------------------------------------------
Thank you [~nigrofranz] for your response.
We will accommodate the additional outofmemory parameter changes into our
artemis.profile configuration.
We are also looking for solution why broker failed with OOM and not much
broker log printed for us apart from mentioned exceptions.
our set-up has 4GB allocated to artemis service on dual core m5.large AWS EC2
(8GB in total)
was (Author: mahendra s):
Thank you [~nigrofranz] for your response.
We will accommodate the additional outofmemory parameter changes into our
artemis.profile configuration.
We are also looking for solution why broker failed with OOM and not much broker
log printed for us apart from mentioned exceptions.
> Artemis continues to run in broken state after OOME (thread)
> ------------------------------------------------------------
>
> Key: ARTEMIS-2623
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2623
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.11.0
> Environment: openjdk 11.0.3 2019-04-16 LTS
> OpenJDK Runtime Environment 18.9 (build 11.0.3+7-LTS)
> OpenJDK 64-Bit Server VM 18.9 (build 11.0.3+7-LTS, mixed mode, sharing)
> Reporter: Keith Wall
> Priority: Major
>
> If Artemis runs out of threads with the JVM signalling this with an OOME
> error, the process continues in a broken state logging exceptions. It would
> be better if the process exited so the running environment (Kubernetes, job
> control etc) can restart it.
> Note that stack trace shows a {{ClassCastException}} where an attempt to made
> to assign the OOME (Error) to an Exception.
> I read from [JDK-8155004|https://bugs.openjdk.java.net/browse/JDK-8155004]
> that the JVM flag {{XxxxOnOutOfMemoryError}} is not applied to this sort of
> OOME.
> {noformat}
> [104223.528s][warning][os,thread] Failed to start thread - pthread_create
> failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
> 2020-02-19T14:41:04.437Z WARN [ProtonServerSenderContext]
> java.lang.ClassCastException: class java.lang.OutOfMemoryError cannot be cast
> to class java.lang.Exception (java.lang.OutOfMemoryError and
> java.lang.Exception are in module java.base of loader 'bootstrap')
> java.lang.ClassCastException: class java.lang.OutOfMemoryError cannot be cast
> to class java.lang.Exception (java.lang.OutOfMemoryError and
> java.lang.Exception are in module java.base of loader 'bootstrap')
> at
> org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.individualAcknowledge(ServerConsumerImpl.java:998)
> at
> org.apache.activemq.artemis.protocol.amqp.broker.AMQPSessionCallback.ack(AMQPSessionCallback.java:399)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.ProtonServerSenderContext.onMessage(ProtonServerSenderContext.java:621)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext.onDelivery(AMQPConnectionContext.java:583)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.Events.dispatch(Events.java:92)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.dispatch(ProtonHandler.java:485)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.flush(ProtonHandler.java:285)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.inputBuffer(ProtonHandler.java:242)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext.inputBuffer(AMQPConnectionContext.java:180)
> at
> org.apache.activemq.artemis.protocol.amqp.broker.ActiveMQProtonRemotingConnection.bufferReceived(ActiveMQProtonRemotingConnection.java:145)
> at
> org.apache.activemq.artemis.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:654)
> at
> org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
> at
> io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
> at
> io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:796)
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:427)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> 2020-02-19T14:41:04.437Z WARN [ProtonHandler] AMQ119007: error acknowledging
> message
> org.apache.activemq.artemis.protocol.amqp.broker.AMQPMessage@51dd2f4c, class
> java.lang.OutOfMemoryError cannot be cast to class java.lang.Exception
> (java.lang.OutOfMemoryError and java.lang.Exception are in module java.base
> of loader 'bootstrap')
> org.apache.activemq.artemis.protocol.amqp.exceptions.ActiveMQAMQPIllegalStateException:
> AMQ119007: error acknowledging message
> org.apache.activemq.artemis.protocol.amqp.broker.AMQPMessage@51dd2f4c, class
> java.lang.OutOfMemoryError cannot be cast to class java.lang.Exception
> (java.lang.OutOfMemoryError and java.lang.Exception are in module java.base
> of loader 'bootstrap')
> at
> org.apache.activemq.artemis.protocol.amqp.proton.ProtonServerSenderContext.onMessage(ProtonServerSenderContext.java:624)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext.onDelivery(AMQPConnectionContext.java:583)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.Events.dispatch(Events.java:92)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.dispatch(ProtonHandler.java:485)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.flush(ProtonHandler.java:285)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.handler.ProtonHandler.inputBuffer(ProtonHandler.java:242)
> at
> org.apache.activemq.artemis.protocol.amqp.proton.AMQPConnectionContext.inputBuffer(AMQPConnectionContext.java:180)
> at
> org.apache.activemq.artemis.protocol.amqp.broker.ActiveMQProtonRemotingConnection.bufferReceived(ActiveMQProtonRemotingConnection.java:145)
> at
> org.apache.activemq.artemis.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:654)
> at
> org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
> at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1470)
> at
> io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1219)
> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266)
> at
> io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498)
> at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437)
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377)
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363)
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:796)
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:427)
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328)
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
> at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
> [104223.571s][warning][os,thread] Failed to start thread - pthread_create
> failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached.
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
