from:"Phillip Jenkins \(JIRA\)"

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-22 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410786#comment-16410786
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram]

I said I would get back to you. The load test went well and Artemis did well. 
This bug is resolved.

Thank you again for the hard work. Looking forward to the next release! 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-21 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408380#comment-16408380
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic]
Thank you for getting back to me again. We require using these large messages 
for some of our use cases. There's just no getting around that. We will be out 
front testing anything Artemis does in support of large messages. 

If I understand you correctly, what you are saying and what you are not saying, 
we will want to: 
- Use QoS=1 to make sure the message gets there.
- Update the logging.properties file to quiet the log output. 

So far the testing is going fine. I'll be hitting the broker with significant 
load later today to verify how it performs. I will report back here how it went.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-21 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408098#comment-16408098
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram]

My central question is around why does this method sendLargeMessageContinuation 
in MQTTSessionCallback get called?  The journal buffer size in my test setup is 
10M and I don't see (anymore) the files I saw previously in the paging 
directory. In fact *paging* and *large-messages* are empty. If the large 
journal-buffer-size prevents writing to disk then that is OK and 
understandable. Again my concern is around the warning. I don't want to block 
the warning (via logging.properties) if it's trying to tell me something I need 
to know. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-20 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407234#comment-16407234
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

Perhaps I'm just a bit dense but I'm not following what you're trying to say. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-20 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16407209#comment-16407209
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

That sounds a bit scary. Currently the messages are sent using QoS = 0 and they 
appear to be working properly. If I move to QoS = 1 then will the message 
transmission will finish (to complete the QoS contract) and eliminate the 
warning? 

Looking at the Configuration Reference page min-large-message-size is part of 
bridging. We are not doing any bridging. Do I misread the documentation? Also 
your suggestion for the journal-size is not clear enough. Did you mean 
journal-buffer-size? I set that to 
10M and let the broker auto-reload. 
The message is still coming out. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-20 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406939#comment-16406939
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram],

Thank you for getting back to me. I see the logging statement in code so that's 
why I wasn't sure of the approach to quiet that down. Something new in the log 
which is mostly OK. I think that others will likely need to know how to quiet 
this down. The logging properties are not dynamically loaded? 

My testing continues to be good so far. We'll be increasing the load of these 
large messages soon to see how well the broker handles them. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-20 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16406682#comment-16406682
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram],

The volume of "Sending LARGE MESSAGE" coming out of MQTTSessionCallback is 
tremendous. Do I now need to change the logging properties to quiet that down 
or configure such noise thru broker.xml?

Thanks,
Phillip 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-19 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405777#comment-16405777
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram]

Pulled the Artemis code from GitHub. The server environment is defaulting to an 
out-of-box setup with no special settings. I made 2 test runs back to back 
successfully. My broker remains up and running to continue testing tomorrow.

The log contains MANY messages like: 
{quote}*2018-03-19 17:54:48,837 WARN 
[org.apache.activemq.artemis.core.protocol.mqtt] Sending LARGE MESSAGE*
{quote}
 I think I've seen this kind of thing before in my testing. Shouldn't I be able 
to configure my broker.xml and eliminate the extra messaging in the log?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB MQTT messages.

2018-03-17 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403716#comment-16403716
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram]

Yes I will take a look at the PR plus pull the code and try it out ASAP. I've 
been on holiday a few days but will respond ASAP.

Note: Phillip has two L's. Thanks!

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB MQTT messages.
> -
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: clebert suconic
>Priority: Blocker
> Fix For: 2.5.1
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-03-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399436#comment-16399436
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram]

I was doing my testing using Justin's branch and his example code which was 
based on my example code. Hence we were on the same page. :D

That's great news [~jbertram]! You have made me very happy! I was beginning to 
wonder if I was hallucinating the problem. When the time comes I will be VERY 
happy to assist testing. 

Gentlemen thank you very much for your patience and help.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-03-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16399134#comment-16399134
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram],

I've reproduced the issue using 100Mb. 

I have the server in a separate console to see the error easily. This time I 
watched the java process that is running the server and noticed it growing 
during the test. As the messages were published during the test it continued to 
grow. That looks wrong and appears to show the root cause of the OODME error. 
Why does it grow? What sets the limit for that? The message QoS = 0 and I 
expected the messages to disappear as they are consumed. Also, the paging 
directory shows LOTS of PAGE files that appear to be my messages. Why doesn't 
that directory remove the consumed message? 

If [~jbertram] doesn't get the error then lets look at why my process grows and 
the paging directory isn't cleaned up. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-03-08 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391653#comment-16391653
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic] and [~jbertram],

I was using the default global-max-size which comes out at around 500MB. I 
found that by increasing the memory I could delay the error. However, it was 
just a delay and eventually the error would appear. I can suggest increasing 
the loop count to 500 to see if it will appear. Something to run over lunch so 
it doesn't impact you too much. Until you reproduce you're stuck which means 
I'm stuck. 

It's a big deal here so I must brainstorm with others for possible next steps. 
Any suggestions from you would be most welcome indeed. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-03-07 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390267#comment-16390267
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], let me see if I understand you. My Windows workstation with 16G of 
memory is running out of memory? I have little understanding of netty and no 
understanding of what causes the OODME. Are you saying that because it's a 
"direct memory error" it must be an issue with how much available free memory I 
have? Despite having gigabytes of available RAM sending 3MB messages will cause 
my to consume available memory? Please clarify. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-03-05 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387314#comment-16387314
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], would it be possible to get together over webex or similar to walk 
thru a demo of the error?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-03-04 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385619#comment-16385619
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], that totally makes no sense. Hmm... I ran and repeated it twice. 
Running the server in a separate console makes the issue appear very clearly in 
the server console. At around message 100 it will throw the OODME exception. 
You did let it run that long didn't you? How long did you let it run for?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
>

[jira] [Comment Edited] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-03-01 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381504#comment-16381504
 ] 

Phillip Jenkins edited comment on ARTEMIS-1286 at 3/1/18 3:11 PM:
--

[~jbertram], I've attached an updated MQTTBasicPubSubExample2 class file that 
will cause the OODME in netty. Please try it at your earliest convenience.
Many thanks!

Note: I ran the broker in a separate console than the example (use -PnoServer) 
to clearly observe the error come out.


was (Author: pwjenkins):
[~jbertram], I've attached an updated MQTTBasicPubSubExample2 class file that 
will cause the OODME in netty. Please try it at your earliest convenience.
Many thanks!

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-28 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16381504#comment-16381504
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], I've attached an updated MQTTBasicPubSubExample2 class file that 
will cause the OODME in netty. Please try it at your earliest convenience.
Many thanks!

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
>

[jira] [Updated] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-28 Thread Phillip Jenkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phillip Jenkins updated ARTEMIS-1286:
-
Attachment: MQTTBasicPubSubExample2.zip

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: MQTTBasicPubSubExample2.zip, artemis log showing v2.4 
> still having trouble handling large messages.log, broker.xml, 
> image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-26 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377173#comment-16377173
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], I set aside the issue in my last comment to focus only on getting 
the example code running in the Artemis 2.2.x source code from July 2017. I did 
not have these issues last July when I assembled the example code. So I'm 
struggling to get Artemis up and running now. This is about getting Artemis 
running not my code. I can see the SNAPSHOT in the m2 repository so this is 
quite frustrating. Why doesn't Maven see the jar is there and succeed? I'm 
grateful for any suggestions to get this working.

*Running:*
C:\Projects\activemq-artemis-master-July\examples\protocols\mqtt>mvn -Pexamples 
verify

*Results:*
[ERROR] Failed to execute goal 
org.apache.activemq:artemis-maven-plugin:2.2.0-SNAPSHOT:create (create) on 
project artemis-mqtt-publish-example: org/apache/activemq/artemis/boot/Artemis 
-> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal 
org.apache.activemq:artemis-maven-plugin:2.2.0-SNAPSHOT:create (create) on 
project artemis-mqtt-publish-example: org/apache/activemq/artemis/boot/Artemis
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at 
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
...

[~clebertsuconic]  [~martyntaylor]

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: artemis log showing v2.4 still having trouble handling 
> large messages.log, broker.xml, image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-24 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375887#comment-16375887
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], I've been trying to get Artemis up and running. You mentioned 
having trouble getting it up and running. I've reverted back to the code base 
of July to try to get this running. It shows the following failure I've yet to 
resolve. Suggestions would be most welcome.

!image-2018-02-24-19-54-00-892.png!

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: artemis log showing v2.4 still having trouble handling 
> large messages.log, broker.xml, image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
>

[jira] [Updated] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-24 Thread Phillip Jenkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phillip Jenkins updated ARTEMIS-1286:
-
Attachment: image-2018-02-24-19-54-00-892.png

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: artemis log showing v2.4 still having trouble handling 
> large messages.log, broker.xml, image-2018-02-24-19-54-00-892.png, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Comment Edited] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-22 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373427#comment-16373427
 ] 

Phillip Jenkins edited comment on ARTEMIS-1286 at 2/22/18 9:31 PM:
---

[~jbertram], I just ran our code against the v2.4 release. Sadly I got a 
different error message but the error still relates to large message. I've 
attached the log.

I've pulled your branch and working thru the project Readme.md file 
instructions for getting the examples to run. I've got errors in Prelease to 
resolve.


was (Author: pwjenkins):
[~jbertram], I just ran our code against the v2.4 release. Sadly I got a 
different error message but the error still relates to large message. I've 
attached the log.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: artemis log showing v2.4 still having trouble handling 
> large messages.log, broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-22 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373427#comment-16373427
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], I just ran our code against the v2.4 release. Sadly I got a 
different error message but the error still relates to large message. I've 
attached the log.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: artemis log showing v2.4 still having trouble handling 
> large messages.log, broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
>

[jira] [Updated] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-22 Thread Phillip Jenkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phillip Jenkins updated ARTEMIS-1286:
-
Attachment: artemis log showing v2.4 still having trouble handling large 
messages.log

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: artemis log showing v2.4 still having trouble handling 
> large messages.log, broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2018-02-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364639#comment-16364639
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~jbertram], apologies for finally get back to you. Thank you for looking into 
this. It's greatly appreciated!

I found the issue was reproduced with the sample code I provided last July on 
the Artemis code at the time. Since you could not reproduce the issue I'm going 
to jump up to v2.4 and try reproducing the issue. If I can i will pull your 
branch and look into your example as you suggest. I'll respond back here with 
my test results.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: unscheduled
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-08-09 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16121055#comment-16121055
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~tabish121]  Please let me sincerely apologize for any rudeness on my part. 
Clebert indicated 2 weeks ago he might get to look at this. We need it and are 
trying to plan. I wish we could fix it. We have contributed to Artemis in the 
past but this issue is beyond us. I'm here to help in anyway I can.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-08-07 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16117874#comment-16117874
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~martyntaylor], checking to see if there's been any work on this yet. Any 
progress? Any estimate on when it might get to resolution?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-08-06 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115962#comment-16115962
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~martyntaylor], checking to see if there's been any work on this yet. Any 
progress? Any estimate on when it might get to resolution?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-08-03 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16113923#comment-16113923
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~martyntaylor], checking to see if there's been any work on this yet. Any 
progress? Any estimate on when it might get to resolution?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Assignee: Martyn Taylor
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-08-02 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112140#comment-16112140
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], checking to see if there's been any work on this yet. Any 
progress? Any estimate on when it might get to resolution?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-08-01 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110318#comment-16110318
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], checking to see if there's been any work on this yet. Any 
progress? Any estimate on when it might get to resolution? 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-31 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107799#comment-16107799
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], checking to see if this is in progress?  Any estimate of 
when you might get to resolution? 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Comment Edited] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-27 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103366#comment-16103366
 ] 

Phillip Jenkins edited comment on ARTEMIS-1286 at 7/27/17 3:45 PM:
---

Just checking in to see if there was any review of this since last reply 
comment. Any estimate of when you might get to this?


was (Author: pwjenkins):
Just checking in to see if there was any review of this since last reply 
comment. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-27 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16103366#comment-16103366
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

Just checking in to see if there was any review of this since last reply 
comment. 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.3.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-25 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16100369#comment-16100369
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], just checking in to see if there was any review of this? Any 
discussion of a resolution release? 

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.2.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-19 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16093138#comment-16093138
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], have you tried using the attached project to reproduce the 
exception?
[~eduda], please consider fixing this issue as part of 2.2 release.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.2.0
>
> Attachments: broker.xml, mqtt.zip
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-17 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090676#comment-16090676
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], I have reproduced the exception. I started with the 
*ActiveMQ Artemis MQTT Examples* under 
*activemq-artemis-master\examples\protocols\mqtt* and modified from there.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.2.0
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16088102#comment-16088102
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

OK. Give me a bit to work that out. I'll get back ASAP.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0
>Reporter: Phillip Jenkins
>Priority: Blocker
> Fix For: 2.2.0
>
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087962#comment-16087962
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

I'm working thru the test now. I'll get back ASAP.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Phillip Jenkins
>Priority: Blocker
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Comment Edited] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087913#comment-16087913
 ] 

Phillip Jenkins edited comment on ARTEMIS-1286 at 7/14/17 7:46 PM:
---

[~clebertsuconic], since you pointed to a configuration issue I went back to 
v2.1. However, you raise a valid question. Should I run this against v2.2 so 
that I pickup the OODME fixes in master? I can test against v2.2 and get back 
to you in less than one hour. Please specify.


was (Author: pwjenkins):
[~clebertsuconic], since you pointed to a configuration issue I went back to 
v2.1. However, you raise a valid question. Should I run this against v2.2 so 
that I pickup the OODME fixes in master? I can test against v2.2 and get back 
to you in less than one hour.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Phillip Jenkins
>Priority: Blocker
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
>

[jira] [Comment Edited] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087913#comment-16087913
 ] 

Phillip Jenkins edited comment on ARTEMIS-1286 at 7/14/17 7:45 PM:
---

[~clebertsuconic], since you pointed to a configuration issue I went back to 
v2.1. However, you raise a valid question. Should I run this against v2.2 so 
that I pickup the OODME fixes in master? I can test against v2.2 and get back 
to you in less than one hour.


was (Author: pwjenkins):
[~clebertsuconic], since you pointed to a configuration issue I went back to 
v2.1. However, you raise a valid question. Should I run this against v2.2 so 
that I pickup the OODME fixes in master?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Phillip Jenkins
>Priority: Blocker
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087913#comment-16087913
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], since you pointed to a configuration issue I went back to 
v2.1. However, you raise a valid question. Should I run this against v2.2 so 
that I pickup the OODME fixes in master?

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Phillip Jenkins
>Priority: Blocker
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-14 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16087873#comment-16087873
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

[~clebertsuconic], I've been trying various configurations. I'm not convinced. 
I can see paging is doing something. I see messages in the log and the paging 
folder has 29 bytes of data. Here's my broker.xml settings of interest.
global-max-size = 100Mb
max-size-bytes = 2Mb
page-size-bytes = 512000

The exception thrown is as follows:
14:27:03,090 WARN  [org.apache.activemq.artemis.core.server] AMQ222038: 
Starting paging on address 'SOAP.S.PRN.ADE0E203D58FBD00'; size is currently: 
4,194,817 bytes; max-size-bytes: 2,097,152
14:27:09,351 WARN  [org.apache.activemq.artemis.core.server] AMQ222038: 
Starting paging on address 'SOAP.S.PRN.ADE0E203D58FBD00'; size is currently: 
4,194,817 bytes; max-size-bytes: 2,097,152
14:27:09,366 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
removing consumer which did not handle a message, consumer=ServerConsumerImpl 
[id=15, filter=null, binding=LocalQueueBinding 
[address=SOAP.S.PRN.ADE0E203D58FBD00, 
queue=QueueImpl[name=ADE0E203D58FBD00.SOAP.S.PRN.ADE0E203D58FBD00, 
postOffice=PostOfficeImpl 
[server=ActiveMQServerImpl::serverUUID=cbf29555-68c8-11e7-a9f0-54ee7505882d], 
temp=false]@7b992a73, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
name=ADE0E203D58FBD00.SOAP.S.PRN.ADE0E203D58FBD00, 
clusterName=ADE0E203D58FBD00.SOAP.S.PRN.ADE0E203D58FBD00cbf29555-68c8-11e7-a9f0-54ee7505882d]],
 message=PagedReferenceImpl [position=PagePositionImpl [pageNr=72, messageNr=0, 
recordID=-1], message=PagedMessageImpl [queueIDs=[13], transactionID=-1, 
message=CoreMessage[messageID=2500,durable=false,userID=null,priority=0, 
timestamp=0,expiration=0, durable=false, 
address=SOAP.S.PRN.ADE0E203D58FBD00,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1737757493],
 deliveryTime=null, persistedCount=0, deliveryCount=1, 
subscription=PageSubscriptionImpl [cursorId=13, 
queue=QueueImpl[name=ADE0E203D58FBD00.SOAP.S.PRN.ADE0E203D58FBD00, 
postOffice=PostOfficeImpl 
[server=ActiveMQServerImpl::serverUUID=cbf29555-68c8-11e7-a9f0-54ee7505882d], 
temp=false]@7b992a73, filter = FilterImpl [sfilterString=NOT ((AMQAddress = 
'activemq.management') OR (AMQAddress = 'activemq.notifications'))]]]: 
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
byte(s) of direct memory (used: 1073248181, max: 1073741824)
at 
io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
[netty-all-4.1.9.Final.jar:4.1.9.Final]
at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
[netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
 [artemis-commons-2.1.0.jar:2.1.0]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:274)
 [artemis-mqtt-protocol-2.1.0.jar:]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:119)
 [artemis-mqtt-protocol-2.1.0.jar:]
...

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.

[jira] [Commented] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-13 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16086188#comment-16086188
 ] 

Phillip Jenkins commented on ARTEMIS-1286:
--

Thanks for getting back. I will pursue that and get back.

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Phillip Jenkins
>Priority: Blocker
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Updated] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-13 Thread Phillip Jenkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phillip Jenkins updated ARTEMIS-1286:
-
Priority: Blocker  (was: Major)

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Phillip Jenkins
>Priority: Blocker
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, the server throws the following OutOfDirectMemoryError 
> exception. The server stops responding and will not (any longer) accept 
> connections without throwing exceptions in an infinite loop.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTSessionCallback.sendMessage(MQTTSessionCallback.java:52)
>

[jira] [Updated] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-13 Thread Phillip Jenkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phillip Jenkins updated ARTEMIS-1286:
-
Description: 
Originally seen in v2.1 and present in v2.2.
If you send and receive a lot of 2MB messages in short time thru Artemis via 
Netty connector, the server throws the following OutOfDirectMemoryError 
exception. The server stops responding and will not (any longer) accept 
connections without throwing exceptions in an infinite loop.

{code:java}
11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server started 
at http://localhost:8161
11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
REST API available at http://localhost:8161/jolokia
11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
removing consumer which did not handle a message, consumer=ServerConsumerImpl 
[id=2229, filter=null, binding=LocalQueueBinding 
[address=SOAP.S.PRN.ACBCAF6238234680, 
queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
postOffice=PostOfficeImpl 
[server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
 
message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
 timestamp=0,expiration=0, durable=false, 
address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
 io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
byte(s) of direct memory (used: 1070952692, max: 1073741824)
at 
io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
[netty-all-4.1.9.Final.jar:4.1.9.Final]
at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
[netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
 [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
 [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
 [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTSessionCallback.sendMessage(MQTTSessionCallback.java:52)
 [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
at 
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.deliverStandardMessage(ServerConsumerImpl.java:1078)
 [artemis-server-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:450)
 [artemis-server-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2688)
 [artemis-server-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2673)

[jira] [Updated] (ARTEMIS-1286) Server stops responding and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-13 Thread Phillip Jenkins (JIRA)


 [ 
https://issues.apache.org/jira/browse/ARTEMIS-1286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phillip Jenkins updated ARTEMIS-1286:
-
Summary: Server stops responding and throws OutOfDirectMemoryError when 
sending & receiving lots of 2MB messages.  (was: Server stops and throws 
OutOfDirectMemoryError when sending & receiving lots of 2MB messages.)

> Server stops responding and throws OutOfDirectMemoryError when sending & 
> receiving lots of 2MB messages.
> 
>
> Key: ARTEMIS-1286
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker, MQTT
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Phillip Jenkins
>
> Originally seen in v2.1 and present in v2.2.
> If you send and receive a lot of 2MB messages in short time thru Artemis via 
> Netty connector, server throws the following OutOfDirectMemoryError exception.
> {code:java}
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server 
> started at http://localhost:8161
> 11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
> REST API available at http://localhost:8161/jolokia
> 11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
> removing consumer which did not handle a message, consumer=ServerConsumerImpl 
> [id=2229, filter=null, binding=LocalQueueBinding 
> [address=SOAP.S.PRN.ACBCAF6238234680, 
> queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> postOffice=PostOfficeImpl 
> [server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
> temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
> 'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
> name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
> clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
>  
> message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
>  timestamp=0,expiration=0, durable=false, 
> address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
>  io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
> byte(s) of direct memory (used: 1070952692, max: 1073741824)
>   at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
>  [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
> [netty-all-4.1.9.Final.jar:4.1.9.Final]
>   at 
> org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
>  [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
> org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
>  [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
>   at 
>

[jira] [Created] (ARTEMIS-1286) Server stops and throws OutOfDirectMemoryError when sending & receiving lots of 2MB messages.

2017-07-13 Thread Phillip Jenkins (JIRA)

Phillip Jenkins created ARTEMIS-1286:


 Summary: Server stops and throws OutOfDirectMemoryError when 
sending & receiving lots of 2MB messages.
 Key: ARTEMIS-1286
 URL: https://issues.apache.org/jira/browse/ARTEMIS-1286
 Project: ActiveMQ Artemis
  Issue Type: Bug
  Components: Broker, MQTT
Affects Versions: 2.1.0, 2.2.0
Reporter: Phillip Jenkins


Originally seen in v2.1 and present in v2.2.
If you send and receive a lot of 2MB messages in short time thru Artemis via 
Netty connector, server throws the following OutOfDirectMemoryError exception.

{code:java}
11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241001: HTTP Server started 
at http://localhost:8161
11:24:07,434 INFO  [org.apache.activemq.artemis] AMQ241002: Artemis Jolokia 
REST API available at http://localhost:8161/jolokia
11:58:35,991 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: 
removing consumer which did not handle a message, consumer=ServerConsumerImpl 
[id=2229, filter=null, binding=LocalQueueBinding 
[address=SOAP.S.PRN.ACBCAF6238234680, 
queue=QueueImpl[name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
postOffice=PostOfficeImpl 
[server=ActiveMQServerImpl::serverUUID=afa7de73-67e7-11e7-a231-54ee7505882d], 
temp=false]@46adc2a5, filter=FilterImpl [sfilterString=NOT ((AMQAddress = 
'activemq.management') OR (AMQAddress = 'activemq.notifications'))], 
name=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680, 
clusterName=ACBCAF6238234680.SOAP.S.PRN.ACBCAF6238234680afa7de73-67e7-11e7-a231-54ee7505882d]],
 
message=Reference[3551]:NON-RELIABLE:CoreMessage[messageID=3551,durable=false,userID=null,priority=0,
 timestamp=0,expiration=0, durable=false, 
address=SOAP.S.PRN.ACBCAF6238234680,properties=TypedProperties[mqtt.message.retain=false,mqtt.qos.level=0]]@1305748199:
 io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 3729415 
byte(s) of direct memory (used: 1070952692, max: 1073741824)
at 
io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:117)
 [netty-all-4.1.9.Final.jar:4.1.9.Final]
at io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828) 
[netty-all-4.1.9.Final.jar:4.1.9.Final]
at io.netty.buffer.WrappedByteBuf.readBytes(WrappedByteBuf.java:616) 
[netty-all-4.1.9.Final.jar:4.1.9.Final]
at 
org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readBytes(ChannelBufferWrapper.java:315)
 [artemis-commons-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendServerMessage(MQTTPublishManager.java:277)
 [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:121)
 [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
at 
org.apache.activemq.artemis.core.protocol.mqtt.MQTTSessionCallback.sendMessage(MQTTSessionCallback.java:52)
 [artemis-mqtt-protocol-2.2.0-SNAPSHOT.jar:]
at 
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.deliverStandardMessage(ServerConsumerImpl.java:1078)
 [artemis-server-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:450)
 [artemis-server-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
at 
org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2688)

[jira] [Commented] (ARTEMIS-1223) OutOfDirectMemoryError raised from TimedBuffer

2017-07-11 Thread Phillip Jenkins (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16082509#comment-16082509
 ] 

Phillip Jenkins commented on ARTEMIS-1223:
--

I am experiencing an issue in v2.1 that is extremely similar to this. Do we 
know when this fix will be released?

> OutOfDirectMemoryError raised from TimedBuffer
> --
>
> Key: ARTEMIS-1223
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1223
> Project: ActiveMQ Artemis
>  Issue Type: Bug
>  Components: Broker
>Affects Versions: 1.5.5, 2.1.0
> Environment: IBM JDK
> java version "1.8.0"
> Java(TM) SE Runtime Environment (build pxa6480sr3fp20-20161019_02(SR3 FP20))
> IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 
> 20161013_322271 (JIT enabled, AOT enabled)
> J9VM - R28_Java8_SR3_20161013_1635_B322271
> JIT  - tr.r14.java.green_20161011_125790
> GC   - R28_Java8_SR3_20161013_1635_B322271_CMPRSS
> J9CL - 20161013_322271)
> JCL - 20161018_01 based on Oracle jdk8u111-b14
>Reporter: Erich Duda
>Priority: Minor
> Fix For: 1.5.6, 2.2.0
>
>
> I see a lot of OutOfDirectMemoryError \[1\] in Artemis test suite when it 
> runs on IBM JDK. I don't see it with Oracle or OpenJDK.
> For reproducing the issue run {{LargeMessageCompressTest}} on IBM JDK.
> \[1\]
> {code}
> io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 501760 
> byte(s) of direct memory (used: 66876815, max: 67108864)
> at 
> io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:585)
> at 
> io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:539)
> at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
> at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:169)
> at 
> io.netty.buffer.UnpooledUnsafeDirectByteBuf.(UnpooledUnsafeDirectByteBuf.java:68)
> at 
> io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
> at 
> io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.(UnpooledByteBufAllocator.java:164)
> at 
> io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:73)
> at 
> io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:181)
> at io.netty.buffer.Unpooled.directBuffer(Unpooled.java:145)
> at 
> org.apache.activemq.artemis.core.io.buffer.TimedBuffer.(TimedBuffer.java:103)
> at 
> org.apache.activemq.artemis.core.io.AbstractSequentialFileFactory.(AbstractSequentialFileFactory.java:78)
> at 
> org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory.(AIOSequentialFileFactory.java:78)
> at 
> org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.init(JournalStorageManager.java:136)
> at 
> org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.(AbstractJournalStorageManager.java:217)
> at 
> org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.(JournalStorageManager.java:103)
> at 
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.createStorageManager(ActiveMQServerImpl.java:2014)
> at 
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.initialisePart1(ActiveMQServerImpl.java:2151)
> at 
> org.apache.activemq.artemis.core.server.impl.LiveOnlyActivation.run(LiveOnlyActivation.java:63)
> at 
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.internalStart(ActiveMQServerImpl.java:517)
> at 
> org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.start(ActiveMQServerImpl.java:465)
> at 
> org.apache.activemq.artemis.tests.integration.client.LargeMessageCompressTest.testLargeMessageCompression3(LargeMessageCompressTest.java:171)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

49 matches

Mail list logo