[GitHub] [activemq-artemis] wy96f commented on a change in pull request #2832: ARTEMIS-2482 Large messages could leak native ByteBuffers
wy96f commented on a change in pull request #2832: ARTEMIS-2482 Large messages
could leak native ByteBuffers
URL: https://github.com/apache/activemq-artemis/pull/2832#discussion_r325571766
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/LargeServerMessageImpl.java
##
@@ -511,6 +512,44 @@ protected void closeFile() throws Exception {
}
}
+ private static int read(final SequentialFile file, final ByteBuffer
bufferRead) throws Exception {
Review comment:
Do you mean readFileBuffer in Page? In most cases, we sequentially read
messages from page and readFileBuffer might contains several messages(depends
on message size) in buffer. This way we can just read following messages from
buffer(no need bothering file system). If we use netty pools, we need to read
the file every time we read a message, right? Will this somewhat affect the
perf?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [activemq-artemis] franz1981 commented on a change in pull request #2832: ARTEMIS-2482 Large messages could leak native ByteBuffers
franz1981 commented on a change in pull request #2832: ARTEMIS-2482 Large
messages could leak native ByteBuffers
URL: https://github.com/apache/activemq-artemis/pull/2832#discussion_r325575480
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/LargeServerMessageImpl.java
##
@@ -511,6 +512,44 @@ protected void closeFile() throws Exception {
}
}
+ private static int read(final SequentialFile file, final ByteBuffer
bufferRead) throws Exception {
Review comment:
> If we use netty pools, we need to read the file every time we read a
message, right? Will this somewhat affect the perf?
We need to find a proper solution, but is a leak and we use a heap buffer
too there so it means a leak on IOUtil too...
I see the same on decoding side for replicated files and on encoding side
for non netty connections...basically everytime we performa FileChannel ops
with heap ByteBuffers.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [activemq-artemis] wy96f commented on a change in pull request #2839: [ARTEMIS-2490] Prevent NumberFormatExc when reading large message
wy96f commented on a change in pull request #2839: [ARTEMIS-2490] Prevent
NumberFormatExc when reading large message
URL: https://github.com/apache/activemq-artemis/pull/2839#discussion_r325585266
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/JournalStorageManager.java
##
@@ -369,12 +369,11 @@ protected SequentialFile createFileForLargeMessage(final
long messageID, final b
LargeMessagePersister.getInstance().decode(buff, largeMessage);
if (largeMessage.containsProperty(Message.HDR_ORIG_MESSAGE_ID)) {
- // for compatibility: couple with old behaviour, copying the old file
to avoid message loss
- long originalMessageID =
largeMessage.getLongProperty(Message.HDR_ORIG_MESSAGE_ID);
Review comment:
Hi, I wonder in which case NumberFormatExc would be thrown because of the
fact that message has already contained property HDR_ORIG_MESSAGE_ID.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [activemq-artemis] franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842#issuecomment-532666729 Don't merge it yet, still testing it :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] franz1981 commented on a change in pull request #2832: ARTEMIS-2482 Large messages could leak native ByteBuffers
franz1981 commented on a change in pull request #2832: ARTEMIS-2482 Large
messages could leak native ByteBuffers
URL: https://github.com/apache/activemq-artemis/pull/2832#discussion_r325575480
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/LargeServerMessageImpl.java
##
@@ -511,6 +512,44 @@ protected void closeFile() throws Exception {
}
}
+ private static int read(final SequentialFile file, final ByteBuffer
bufferRead) throws Exception {
Review comment:
> If we use netty pools, we need to read the file every time we read a
message, right? Will this somewhat affect the perf?
We need to find a proper solution, but is a leak and we use a heap buffer
too there so it means a leak on IOUtil too...
I see the same on decoding side for replicated files and on encoding side
for non netty connections...basically everytime we performa FileChannel ops
with heap ByteBuffers.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [activemq-artemis] ecki commented on a change in pull request #2839: [ARTEMIS-2490] Prevent NumberFormatExc when reading large message
ecki commented on a change in pull request #2839: [ARTEMIS-2490] Prevent
NumberFormatExc when reading large message
URL: https://github.com/apache/activemq-artemis/pull/2839#discussion_r325603833
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/JournalStorageManager.java
##
@@ -369,12 +369,11 @@ protected SequentialFile createFileForLargeMessage(final
long messageID, final b
LargeMessagePersister.getInstance().decode(buff, largeMessage);
if (largeMessage.containsProperty(Message.HDR_ORIG_MESSAGE_ID)) {
- // for compatibility: couple with old behaviour, copying the old file
to avoid message loss
- long originalMessageID =
largeMessage.getLongProperty(Message.HDR_ORIG_MESSAGE_ID);
Review comment:
i answered in jira, or do you prefer to discuss here (short form, reason for
jms id in orig core property is unclear)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [activemq-artemis] MrEasy commented on a change in pull request #2839: [ARTEMIS-2490] Prevent NumberFormatExc when reading large message
MrEasy commented on a change in pull request #2839: [ARTEMIS-2490] Prevent
NumberFormatExc when reading large message
URL: https://github.com/apache/activemq-artemis/pull/2839#discussion_r325603848
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/JournalStorageManager.java
##
@@ -369,12 +369,11 @@ protected SequentialFile createFileForLargeMessage(final
long messageID, final b
LargeMessagePersister.getInstance().decode(buff, largeMessage);
if (largeMessage.containsProperty(Message.HDR_ORIG_MESSAGE_ID)) {
- // for compatibility: couple with old behaviour, copying the old file
to avoid message loss
- long originalMessageID =
largeMessage.getLongProperty(Message.HDR_ORIG_MESSAGE_ID);
Review comment:
Problem is that you can set the Core-message-ID and feed it with a
non-number. This then fails when reading it on journal initialization.
The format "ID:" is used as JMS-message-id (in contrast to
core-message-id), so this seems to end-up sometimes as core-message-id as well.
In any case, I would like to request the retrieval to be moved inside the
compatibility block, since it is just unneeded outside.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [activemq-artemis] franz1981 opened a new pull request #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
franz1981 opened a new pull request #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] michaelandrepearce commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
michaelandrepearce commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842#issuecomment-532746924 @franz1981 i know you closed it, but a test case to reproduce and confirm any fix would be needed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] clebertsuconic commented on a change in pull request #2834: ARTEMIS-2488: Handle the case where source address is null
clebertsuconic commented on a change in pull request #2834: ARTEMIS-2488: Handle the case where source address is null URL: https://github.com/apache/activemq-artemis/pull/2834#discussion_r325705859 ## File path: artemis-protocols/artemis-amqp-protocol/src/test/java/org/apache/activemq/artemis/protocol/amqp/proton/ProtonServerSenderContextTest.java ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.activemq.artemis.protocol.amqp.proton; + +import org.apache.activemq.artemis.core.server.AddressQueryResult; +import org.apache.activemq.artemis.protocol.amqp.broker.AMQPSessionCallback; +import org.apache.activemq.artemis.protocol.amqp.exceptions.ActiveMQAMQPNotFoundException; +import org.apache.qpid.proton.amqp.messaging.Source; +import org.apache.qpid.proton.engine.Sender; +import org.junit.Test; + +import java.util.Collections; + +import static org.mockito.Mockito.*; Review comment: this will fail Checkstyle.. can you fix it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842#issuecomment-532741437 Probably the issue is more complex then this, I'm going to close this This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] franz1981 closed pull request #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
franz1981 closed pull request #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842#issuecomment-532760149 @michaelandrepearce I've sent a messag to @wy96f on the PR related to https://issues.apache.org/jira/browse/ARTEMIS-2496 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] michaelandrepearce commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i…
michaelandrepearce commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532762474 Looking at the xnio the issue is before we worked on a copy, thus if xnio mutated the buffer it was safe, with zero copy now xnio is dangerously altering the buffer. Isnt the issue in xnio that its mutating a buffer without copying it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] franz1981 commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i…
franz1981 commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532758809 @wy96f We are actually receiving (and sending) several 0s on the wire (via xnio on wildfly, that will wrap the replication connection as a HTTP one), that would break backup with a IIOBE: ``` 2019-09-18 18:08:14,889 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214013: Failed to decode packet: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1428) at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:723) at io.netty.buffer.WrappedByteBuf.readByte(WrappedByteBuf.java:521) at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readByte(ChannelBufferWrapper.java:280) at org.apache.activemq.artemis.core.protocol.ServerPacketDecoder.decode(ServerPacketDecoder.java:126) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:377) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:426) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:796) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:427) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) 2019-09-18 18:08:14,893 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214031: Failed to decode buffer, disconnect immediately.: java.lang.IllegalStateException: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:390) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) at
[GitHub] [activemq-artemis] franz1981 removed a comment on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
franz1981 removed a comment on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842#issuecomment-532756890 @wy96f We are actually receiving (and sending) several 0s on the wire (via xnio on wildfly, that will wrap the replication connection as a HTTP one), that would break backup with a IIOBE: ``` 2019-09-18 18:08:14,889 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214013: Failed to decode packet: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1428) at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:723) at io.netty.buffer.WrappedByteBuf.readByte(WrappedByteBuf.java:521) at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readByte(ChannelBufferWrapper.java:280) at org.apache.activemq.artemis.core.protocol.ServerPacketDecoder.decode(ServerPacketDecoder.java:126) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:377) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:426) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:796) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:427) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) 2019-09-18 18:08:14,893 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214031: Failed to decode buffer, disconnect immediately.: java.lang.IllegalStateException: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:390) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) at
[GitHub] [activemq-artemis] franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
franz1981 commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842#issuecomment-532756890 @wy96f We are actually receiving (and sending) several 0s on the wire (via xnio on wildfly, that will wrap the replication connection as a HTTP one), that would break backup with a IIOBE: ``` 2019-09-18 18:08:14,889 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214013: Failed to decode packet: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1428) at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:723) at io.netty.buffer.WrappedByteBuf.readByte(WrappedByteBuf.java:521) at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readByte(ChannelBufferWrapper.java:280) at org.apache.activemq.artemis.core.protocol.ServerPacketDecoder.decode(ServerPacketDecoder.java:126) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:377) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:426) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:796) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:427) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) 2019-09-18 18:08:14,893 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214031: Failed to decode buffer, disconnect immediately.: java.lang.IllegalStateException: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:390) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) at
[GitHub] [activemq-artemis] michaelandrepearce edited a comment on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i…
michaelandrepearce edited a comment on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532762474 Looking at the xnio the issue is before we worked on a copy, thus if xnio mutated the buffer it was safe/protected from xnio being naughty and altering a buffer, with zero copy now xnio is dangerously altering the buffer. Isnt the issue in xnio that its mutating a buffer without copying it? E.g. when wrapped it should make a copy. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] clebertsuconic commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i…
clebertsuconic commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532782837 @michaelandrepearce xnio has a wrapper using Netty libraries... it's not using any of our APIs. but it seems the encoding is slightly off in a way that explodes in xnio, but it silently fails for us. as we test this we will probably revert this temporarily until we figure out what's going on. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] clebertsuconic commented on a change in pull request #2839: [ARTEMIS-2490] Prevent NumberFormatExc when reading large message
clebertsuconic commented on a change in pull request #2839: [ARTEMIS-2490]
Prevent NumberFormatExc when reading large message
URL: https://github.com/apache/activemq-artemis/pull/2839#discussion_r325761313
##
File path:
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/JournalStorageManager.java
##
@@ -369,12 +369,11 @@ protected SequentialFile createFileForLargeMessage(final
long messageID, final b
LargeMessagePersister.getInstance().decode(buff, largeMessage);
if (largeMessage.containsProperty(Message.HDR_ORIG_MESSAGE_ID)) {
- // for compatibility: couple with old behaviour, copying the old file
to avoid message loss
- long originalMessageID =
largeMessage.getLongProperty(Message.HDR_ORIG_MESSAGE_ID);
Review comment:
test?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [activemq-artemis] michaelandrepearce commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded
michaelandrepearce commented on issue #2842: ARTEMIS-2495 ReplicationSyncFileMessage last packet should be encoded URL: https://github.com/apache/activemq-artemis/pull/2842#issuecomment-532759528 Good to see we are reaching out to @wy96f, It be great if we have a test to reproduce obviously our existing tests didnt catch, but also to avoid such in future catching us. I guess wildfly caught it via their testing This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] michaelandrepearce commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i…
michaelandrepearce commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532763801 What im saying is because it was working before was by luck that we had made a copy. The issue is really we have exposed this unsafe code in xnio thats used by wildfly This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-nms-amqp] Havret opened a new pull request #33: [WIP] Update docs
Havret opened a new pull request #33: [WIP] Update docs URL: https://github.com/apache/activemq-nms-amqp/pull/33 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-nms-amqp] Havret commented on issue #28: NO-JIRA: Extend logging
Havret commented on issue #28: NO-JIRA: Extend logging URL: https://github.com/apache/activemq-nms-amqp/pull/28#issuecomment-532892930 @cjwmorgan-sol @michaelandrepearce Where are we with this one? Are we coming to any conclusions? I don't know I am to make any adjustments or what. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] asfgit closed pull request #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i…
asfgit closed pull request #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] asfgit merged pull request #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i…
asfgit merged pull request #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-artemis] asfgit closed pull request #2834: ARTEMIS-2488: Handle the case where source address is null
asfgit closed pull request #2834: ARTEMIS-2488: Handle the case where source address is null URL: https://github.com/apache/activemq-artemis/pull/2834 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-nms-amqp] Havret edited a comment on issue #28: NO-JIRA: Extend logging
Havret edited a comment on issue #28: NO-JIRA: Extend logging URL: https://github.com/apache/activemq-nms-amqp/pull/28#issuecomment-532892930 @cjwmorgan-sol @michaelandrepearce Where are we with this one? Are we coming to any conclusions? I don't know if I am to make any adjustments or what. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [activemq-nms-amqp] michaelandrepearce commented on a change in pull request #33: [WIP] Update docs
michaelandrepearce commented on a change in pull request #33: [WIP] Update docs URL: https://github.com/apache/activemq-nms-amqp/pull/33#discussion_r325963096 ## File path: docs/configuration.md ## @@ -0,0 +1,21 @@ +# Client configuration +This file details various configuration options for the client, the syntax for its related configuration, and various uri options that can be set when defining a ConnectionFactory. + +## Connection Uri +The basic format of the clients Connection URI is as follows: + +``` +amqp[s]://hostname:port[?option=value[=value...]] +``` + Review comment: Failover and other settings need to be here This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
