[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=315643=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-315643 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 20/Sep/19 12:52 Start Date: 20/Sep/19 12:52 Worklog Time Spent: 10m Work Description: clebertsuconic commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-533541903 @wy96f is this really a feature you really would like to have? - as we moved through this task, we realized the RegionFile is not actually doing zero-copy. it's using a ByteBuf just like before. I bet there's an issue within Wildfly where the execution happens twice, and the actual move would happen on a secondary buffer or something...there's a loop within xnio capturing packets from HTTP into Netty (Wildfly has a single port for everything). We had a few people working on it this week, this feature is getting lot of attention for little benefit. If you really think this is a nice feature to have.. perhaps you can keep going... but as it stands I would say to keep it out! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 315643) Time Spent: 2h 20m (was: 2h 10m) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Attachments: runTillFails.sh > > Time Spent: 2h 20m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=315441=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-315441 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 20/Sep/19 03:18 Start Date: 20/Sep/19 03:18 Worklog Time Spent: 10m Work Description: wy96f commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-533388136 > @wy96f I've just run a naive test replacing FileRegion with ChunkedNioFile and it should work OOTB...but seems that netty is not reading data from it...I need to dig better into it @franz1981 I just notice `ChunkedWriteHandler` needs to be added in pipeline by using `ChunkedFile`, https://github.com/netty/netty/blob/ff7a9fa091a8bf2e10020f83fc4df1c44098/example/src/main/java/io/netty/example/file/FileServer.java#L77 It was my bad i missed this before revert. When using `ChunkedFile`(in the ssl case), exception would be thrown, https://github.com/netty/netty/blob/ff7a9fa091a8bf2e10020f83fc4df1c44098/transport/src/main/java/io/netty/channel/nio/AbstractNioByteChannel.java#L245 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 315441) Time Spent: 2h 10m (was: 2h) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Attachments: runTillFails.sh > > Time Spent: 2h 10m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314944=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314944 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 19/Sep/19 10:09 Start Date: 19/Sep/19 10:09 Worklog Time Spent: 10m Work Description: franz1981 commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-533063407 @wy96f I've just run a naive test replacing FileRegion with ChunkedNioFile and it should work OOTB...but seems that netty is not reading data from it...I need to dig better into it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314944) Time Spent: 2h (was: 1h 50m) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 2h > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314921 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 19/Sep/19 09:36 Start Date: 19/Sep/19 09:36 Worklog Time Spent: 10m Work Description: wy96f commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-533051423 > Another thing I've noticed: before the revert, `ChunkedFile` wasn't working for me...do we have tests to verify it? @franz1981 There is no test. I copied the code https://github.com/netty/netty/blob/ff7a9fa091a8bf2e10020f83fc4df1c44098/example/src/main/java/io/netty/example/file/FileServerHandler.java#L52 `ChunkedFile` will read file into bytebuf which would then be written into socket channel, https://github.com/netty/netty/blob/ff7a9fa091a8bf2e10020f83fc4df1c44098/handler/src/main/java/io/netty/handler/stream/ChunkedWriteHandler.java#L242 It should work as the operations are general. What problem did you encounter with `ChunkedFile`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314921) Time Spent: 1h 50m (was: 1h 40m) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314908=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314908 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 19/Sep/19 08:43 Start Date: 19/Sep/19 08:43 Worklog Time Spent: 10m Work Description: franz1981 commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-533031566 @wy96f Thanks to reach out. Xnio is using https://github.com/xnio/netty-xnio-transport/blob/0.1/src/main/java/org/xnio/netty/transport/AbstractXnioSocketChannel.java#L149 that transparently would receive the flushed data into Netty ie no custom child of `Connection` Another thing I've noticed: before the revert, `ChunkedFile` wasn't working for me...do we have tests to verify it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314908) Time Spent: 1h 40m (was: 1.5h) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[
https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314903=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314903
]
ASF GitHub Bot logged work on ARTEMIS-2496:
---
Author: ASF GitHub Bot
Created on: 19/Sep/19 08:32
Start Date: 19/Sep/19 08:32
Worklog Time Spent: 10m
Work Description: wy96f commented on issue #2843: ARTEMIS-2496 Revert
catch up with zero-copy, as it's causing issues i…
URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-533027655
@clebertsuconic @franz1981 Hi, I didn't use wildfly/xnio. Will xnio use a
HttpConnection which implements Connection like InVMConnection/NettyConnection?
```
if (connection != null && connection.getTransportConnection()
instanceof NettyConnection) {
bufferSize -= dataSize;
isNetty = true;
}
buffer = createPacket(connection, bufferSize);
encodeHeader(buffer);
encodeRest(buffer, connection);
if (!isNetty) {
if (buffer.byteBuf() != null &&
buffer.byteBuf().nioBufferCount() == 1 && buffer.byteBuf().isDirect()) {
final ByteBuffer byteBuffer =
buffer.byteBuf().internalNioBuffer(buffer.writerIndex(),
buffer.writableBytes());
readFile(byteBuffer);
} else {
final ByteBuf byteBuffer =
PooledByteBufAllocator.DEFAULT.directBuffer(buffer.writableBytes(),
buffer.writableBytes());
try {
final ByteBuffer nioBuffer =
byteBuffer.internalNioBuffer(0, buffer.writableBytes());
final int readBytes = readFile(nioBuffer);
if (readBytes > 0) {
//still use byteBuf to copy data
buffer.writeBytes(byteBuffer, 0, readBytes);
}
} finally {
byteBuffer.release();
}
}
buffer.writerIndex(buffer.capacity());
}
encodeSize(buffer, encodedSize);
return buffer;
```
If not NettyConnection, file data will be read into buffer.
Then in ChannelImpl::send
```
connection.getTransportConnection().write(buffer);
connection.getTransportConnection().write(raf, fileChannel, offset,
dataSize, callback == null ? null : (ChannelFutureListener) future ->
callback.done(future == null || future.isSuccess()));
```
Both buffer and file will be written. For InVMConnection, actually no file
data is transferred:
```
@Override
public void write(RandomAccessFile raf,
FileChannel fileChannel,
long offset,
int dataSize,
final ChannelFutureListener futureListener) {
if (futureListener == null) {
return;
}
try {
executor.execute(() -> {
try {
futureListener.operationComplete(null);
} catch (Exception e) {
throw new IllegalStateException(e);
}
});
} catch (RejectedExecutionException e) {
}
}
```
But if xnio implements a connection and transfers file data one more time in
file send method, the mechanism is broken. Not sure if it is caused by this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 314903)
Time Spent: 1.5h (was: 1h 20m)
> Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration
> with artemis
>
>
> Key: ARTEMIS-2496
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2496
> Project: ActiveMQ Artemis
> Issue Type: Bug
>Affects Versions: 2.10.0
>Reporter: clebert suconic
>Assignee: clebert suconic
>Priority: Major
> Fix For: 2.11.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> This is of course an issue on Wildfly integration, but it seems something on
> our recent changes is breaking replication on Wildfly.
> My biggest concern is that it seems that paging catch up is silently failing
> in our testsuite and some other issues are currently hidden.
> Wildfly has an extra layer on top of Netty:
> https://github.com/xnio/netty-xnio-transport/tree/0.1
> But the main thing here, is that it seems that are other
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314743=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314743 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 19/Sep/19 00:00 Start Date: 19/Sep/19 00:00 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314743) Time Spent: 1h 20m (was: 1h 10m) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314742=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314742 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 19/Sep/19 00:00 Start Date: 19/Sep/19 00:00 Worklog Time Spent: 10m Work Description: asfgit commented on pull request #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314742) Time Spent: 1h 10m (was: 1h) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314501=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314501 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 18/Sep/19 17:21 Start Date: 18/Sep/19 17:21 Worklog Time Spent: 10m Work Description: clebertsuconic commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532782837 @michaelandrepearce xnio has a wrapper using Netty libraries... it's not using any of our APIs. but it seems the encoding is slightly off in a way that explodes in xnio, but it silently fails for us. as we test this we will probably revert this temporarily until we figure out what's going on. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314501) Time Spent: 1h (was: 50m) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 1h > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314449=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314449 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 18/Sep/19 16:31 Start Date: 18/Sep/19 16:31 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532763801 What im saying is because it was working before was by luck that we had made a copy. The issue is really we have exposed this unsafe code in xnio thats used by wildfly This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314449) Time Spent: 50m (was: 40m) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 50m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314445=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314445 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 18/Sep/19 16:29 Start Date: 18/Sep/19 16:29 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532762474 Looking at the xnio the issue is before we worked on a copy, thus if xnio mutated the buffer it was safe/protected from xnio being naughty and altering a buffer, with zero copy now xnio is dangerously altering the buffer. Isnt the issue in xnio that its mutating a buffer without copying it? E.g. when wrapped it should make a copy. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314445) Time Spent: 40m (was: 0.5h) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 40m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314443=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314443 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 18/Sep/19 16:28 Start Date: 18/Sep/19 16:28 Worklog Time Spent: 10m Work Description: michaelandrepearce commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532762474 Looking at the xnio the issue is before we worked on a copy, thus if xnio mutated the buffer it was safe, with zero copy now xnio is dangerously altering the buffer. Isnt the issue in xnio that its mutating a buffer without copying it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314443) Time Spent: 0.5h (was: 20m) > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314436=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314436 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 18/Sep/19 16:18 Start Date: 18/Sep/19 16:18 Worklog Time Spent: 10m Work Description: franz1981 commented on issue #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843#issuecomment-532758809 @wy96f We are actually receiving (and sending) several 0s on the wire (via xnio on wildfly, that will wrap the replication connection as a HTTP one), that would break backup with a IIOBE: ``` 2019-09-18 18:08:14,889 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214013: Failed to decode packet: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1428) at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:723) at io.netty.buffer.WrappedByteBuf.readByte(WrappedByteBuf.java:521) at org.apache.activemq.artemis.core.buffers.impl.ChannelBufferWrapper.readByte(ChannelBufferWrapper.java:280) at org.apache.activemq.artemis.core.protocol.ServerPacketDecoder.decode(ServerPacketDecoder.java:126) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:377) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:426) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:278) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:337) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1408) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:345) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:796) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:427) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:328) at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) 2019-09-18 18:08:14,893 ERROR [org.apache.activemq.artemis.core.client] (Thread-5 (ActiveMQ-client-netty-threads)) AMQ214031: Failed to decode buffer, disconnect immediately.: java.lang.IllegalStateException: java.lang.IndexOutOfBoundsException: readerIndex(4) + length(1) exceeds writerIndex(4): PooledUnsafeDirectByteBuf(ridx: 4, widx: 4, cap: 4) at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:390) at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1198) at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359) at
[jira] [Work logged] (ARTEMIS-2496) Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration with artemis
[ https://issues.apache.org/jira/browse/ARTEMIS-2496?focusedWorklogId=314422=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-314422 ] ASF GitHub Bot logged work on ARTEMIS-2496: --- Author: ASF GitHub Bot Created on: 18/Sep/19 15:56 Start Date: 18/Sep/19 15:56 Worklog Time Spent: 10m Work Description: clebertsuconic commented on pull request #2843: ARTEMIS-2496 Revert catch up with zero-copy, as it's causing issues i… URL: https://github.com/apache/activemq-artemis/pull/2843 …nto some integration usage Revert "ARTEMIS-2336 Use zero copy to replicate journal/page/large message file" This reverts commit 85b93f0883bc06a2dfe2de9d560805a59d626d38. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 314422) Remaining Estimate: 0h Time Spent: 10m > Use of Netty FileRegion on ReplicationCatch is breaking wildfly integration > with artemis > > > Key: ARTEMIS-2496 > URL: https://issues.apache.org/jira/browse/ARTEMIS-2496 > Project: ActiveMQ Artemis > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: clebert suconic >Assignee: clebert suconic >Priority: Major > Fix For: 2.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > This is of course an issue on Wildfly integration, but it seems something on > our recent changes is breaking replication on Wildfly. > My biggest concern is that it seems that paging catch up is silently failing > in our testsuite and some other issues are currently hidden. > Wildfly has an extra layer on top of Netty: > https://github.com/xnio/netty-xnio-transport/tree/0.1 > But the main thing here, is that it seems that are other issues within > Artemis. > For now I'm reverting the change from ARTEMIS-2336 > And we need more investigation to bring it back -- This message was sent by Atlassian Jira (v8.3.4#803005)
