[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378513#comment-16378513
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user asfgit closed the pull request at:

https://github.com/apache/activemq-artemis/pull/1849


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-27 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378512#comment-16378512
 ] 

ASF subversion and git services commented on ARTEMIS-1656:
--

Commit b5fa5ed3b743dabf1e40cfcaa97f1ca05cdbdef5 in activemq-artemis's branch 
refs/heads/master from [~nigro@gmail.com]
[ https://git-wip-us.apache.org/repos/asf?p=activemq-artemis.git;h=b5fa5ed ]

ARTEMIS-1656 OpenWire scalability improvements

OpenWireFormat instances are shared between OpenWire 
connections/sessions/consumers, preventing the clients to scale due to the 
synchronized marshal/unmarshal on it.

It includes:
- direct transport buffer pooling
- groupId SimpleString pooling
- clientId SimpleString pooling
- reduced ActiveMQDestination[] and AtomicLong allocations on AMQSession:send
- reduced ActiveMQDestination allocations
- refactored shouldBlockProducer path of AMQPSession::send to reduce method size
- exclusive OpenWireFormat per session and connection (in/out) to avoid 
contention
- refactored trace log to favour inlining
- changed lastSent volatile set into lazy set to avoid full barrier cost on x86
- stateless OpenWireMessageConverter
- send's lock removal thanks to thread-safe NettyConnection


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378508#comment-16378508
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user gaohoward commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
@franz1981 @michaelandrepearce I think it's good and am going to merge it.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-27 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378246#comment-16378246
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
@michaelandrepearce Yep, it has passed the same tests as master with no 
regressions, hence it can be merged :+1: 


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-26 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377407#comment-16377407
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
@franz1981 you noted you were going to merge this once you ran all openwire 
suite. 
Also you said "I'm fixing the last few bits and it is getting in the right 
shape to be merged". Did you get a chance to do this?




> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-21 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372027#comment-16372027
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
@franz1981 can you rebase this please?


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-16 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16367745#comment-16367745
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
I have no further comments. I haven’t had a chance to run test suites but 
as long as you run them and they pass I’m happy 


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-16 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366727#comment-16366727
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
I'm going to merge this provided that is not causing any regressions on 
OpenWire tests/compatibility tests :+1: 


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-12 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360736#comment-16360736
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
@michaelandrepearce 
> FYI gains were with non persistence

Yep, I need to specify it on the JIRA thanks!
I'm fixing the last few bits and it is getting in the right shape to be 
merged :+1: 


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353578#comment-16353578
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
FYI gains were with non persistence, still need to sort native support of 
open wire format for persistent.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16353577#comment-16353577
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
Btw I ran a perf test using my POC physical server setup, I see similar 
gains, so +1 from me (once comments are addressed)


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352563#comment-16352563
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r166011829
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -248,8 +255,13 @@ private ConnectionInfo getConnectionInfo() {
 
//tells the connection that
//some bytes just sent
-   public void bufferSent() {
-  lastSent = System.currentTimeMillis();
+   private void bufferSent() {
+  //much cheaper than a volatile set if contended, but less precise 
(ie allows stale loads)
+  LAST_SENT_UPDATER.lazySet(this, System.currentTimeMillis());
+   }
+
+   private static void traceBufferReceived(Object connectionID, Command 
command) {
--- End diff --

Its not generic, its still specific, its just following the conventions 
that specific logging lives in the Loggers, such as ActiveMQServerLogger (in 
this case maybe a new logger OpenWireLogger is needed/wanted)


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352525#comment-16352525
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r166001786
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -248,8 +255,13 @@ private ConnectionInfo getConnectionInfo() {
 
//tells the connection that
//some bytes just sent
-   public void bufferSent() {
-  lastSent = System.currentTimeMillis();
+   private void bufferSent() {
+  //much cheaper than a volatile set if contended, but less precise 
(ie allows stale loads)
+  LAST_SENT_UPDATER.lazySet(this, System.currentTimeMillis());
+   }
+
+   private static void traceBufferReceived(Object connectionID, Command 
command) {
--- End diff --

can't say if it has the same reason and could be addressed in that way: I 
do not need specific/common handling of those trace messages, but instead a way 
to packed them in separate methods leaving the callers smaller and able to be 
eligible for inlining.
Making them more generic just for perf reasons could be even worst wdyt?


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352523#comment-16352523
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r166001057
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -713,12 +730,16 @@ public void reconnect(AMQConnectionContext 
existingContext, ConnectionInfo info)
   context.incRefCount();
}
 
+   private static void traceSendCommand(Command command) {
+  ActiveMQServerLogger.LOGGER.trace("sending " + command);
+   }
+
/**
 * This will answer with commands to the client
 */
public boolean sendCommand(final Command command) {
-  if (ActiveMQServerLogger.LOGGER.isTraceEnabled()) {
- ActiveMQServerLogger.LOGGER.trace("sending " + command);
+  if (ACTIVE_MQ_SERVER_LOGGER_TRACE_ENABLED) {
--- End diff --

You're quite right, probably I will revert it but it is actually an issue 
and probably could be turned into a lock-free way: actually it really create a 
point of contentions for any send of messages, affecting responses and 
consumers all together...not good!


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352414#comment-16352414
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165979719
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -248,8 +255,13 @@ private ConnectionInfo getConnectionInfo() {
 
//tells the connection that
//some bytes just sent
-   public void bufferSent() {
-  lastSent = System.currentTimeMillis();
+   private void bufferSent() {
+  //much cheaper than a volatile set if contended, but less precise 
(ie allows stale loads)
+  LAST_SENT_UPDATER.lazySet(this, System.currentTimeMillis());
+   }
+
+   private static void traceBufferReceived(Object connectionID, Command 
command) {
--- End diff --

Still same can be done by using the logger.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352412#comment-16352412
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165979537
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -713,12 +730,16 @@ public void reconnect(AMQConnectionContext 
existingContext, ConnectionInfo info)
   context.incRefCount();
}
 
+   private static void traceSendCommand(Command command) {
+  ActiveMQServerLogger.LOGGER.trace("sending " + command);
+   }
+
/**
 * This will answer with commands to the client
 */
public boolean sendCommand(final Command command) {
-  if (ActiveMQServerLogger.LOGGER.isTraceEnabled()) {
- ActiveMQServerLogger.LOGGER.trace("sending " + command);
+  if (ACTIVE_MQ_SERVER_LOGGER_TRACE_ENABLED) {
--- End diff --

Why not fix it at the route in jboss logger? Surely then it be global fix. 
Im much more happy to fix things at root source.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352270#comment-16352270
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165944739
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -713,12 +730,16 @@ public void reconnect(AMQConnectionContext 
existingContext, ConnectionInfo info)
   context.incRefCount();
}
 
+   private static void traceSendCommand(Command command) {
+  ActiveMQServerLogger.LOGGER.trace("sending " + command);
+   }
+
/**
 * This will answer with commands to the client
 */
public boolean sendCommand(final Command command) {
-  if (ActiveMQServerLogger.LOGGER.isTraceEnabled()) {
- ActiveMQServerLogger.LOGGER.trace("sending " + command);
+  if (ACTIVE_MQ_SERVER_LOGGER_TRACE_ENABLED) {
--- End diff --

I know and it seems an issue with this logger, because it creates a point 
of contention between all the threads (all in the broker!) because it is locked 
(I will provide soon a trace of it): I don't think could exist a clean way to 
fix it other than on the jboss logger and it isn't feasible...


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352267#comment-16352267
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165944067
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -248,8 +255,13 @@ private ConnectionInfo getConnectionInfo() {
 
//tells the connection that
//some bytes just sent
-   public void bufferSent() {
-  lastSent = System.currentTimeMillis();
+   private void bufferSent() {
+  //much cheaper than a volatile set if contended, but less precise 
(ie allows stale loads)
+  LAST_SENT_UPDATER.lazySet(this, System.currentTimeMillis());
+   }
+
+   private static void traceBufferReceived(Object connectionID, Command 
command) {
--- End diff --

I need to add a comment here: it is a trick to allow the caller to be 
inlined reducing bytecode size moving away uncommon paths...hence it doesn't 
have functional purposes.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-04 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352116#comment-16352116
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165900265
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -430,24 +442,29 @@ private void callFailureListeners(final 
ActiveMQException me) {
 
// send a WireFormatInfo to the peer
public void sendHandshake() {
-  WireFormatInfo info = wireFormat.getPreferedWireFormatInfo();
+  WireFormatInfo info = inWireFormat.getPreferedWireFormatInfo();
   sendCommand(info);
}
 
public ConnectionState getState() {
   return state;
}
 
+   private static void tracePhysicalSend(Connection transportConnection, 
Command command) {
+  logger.trace("connectionID: " + (transportConnection == null ? "" : 
transportConnection.getID()) + " SENDING: " + (command == null ? "NULL" : 
command));
+   }
+
public void physicalSend(Command command) throws IOException {
 
   if (logger.isTraceEnabled()) {
- logger.trace("connectionID: " + (getTransportConnection() == null 
? "" : getTransportConnection().getID())
- + " SENDING: " + (command == null ? "NULL" : 
command));
+ tracePhysicalSend(transportConnection, command);
   }
 
   try {
- ByteSequence bytes = wireFormat.marshal(command);
- ActiveMQBuffer buffer = OpenWireUtil.toActiveMQBuffer(bytes);
+ final ByteSequence bytes = outWireFormat.marshal(command);
+ final int bufferSize = bytes.length;
+ final ActiveMQBuffer buffer = 
transportConnection.createTransportBuffer(bufferSize);
--- End diff --

should this be getTransportConnection, like the existing code below, or 
should the existing code below use the field directly also. Just a consistency 
POV. 


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-04 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352106#comment-16352106
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165899293
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -248,8 +255,13 @@ private ConnectionInfo getConnectionInfo() {
 
//tells the connection that
//some bytes just sent
-   public void bufferSent() {
-  lastSent = System.currentTimeMillis();
+   private void bufferSent() {
+  //much cheaper than a volatile set if contended, but less precise 
(ie allows stale loads)
+  LAST_SENT_UPDATER.lazySet(this, System.currentTimeMillis());
+   }
+
+   private static void traceBufferReceived(Object connectionID, Command 
command) {
--- End diff --

dedicated log message method, please add to the logger.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-04 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352105#comment-16352105
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165899196
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -430,24 +442,29 @@ private void callFailureListeners(final 
ActiveMQException me) {
 
// send a WireFormatInfo to the peer
public void sendHandshake() {
-  WireFormatInfo info = wireFormat.getPreferedWireFormatInfo();
+  WireFormatInfo info = inWireFormat.getPreferedWireFormatInfo();
   sendCommand(info);
}
 
public ConnectionState getState() {
   return state;
}
 
+   private static void tracePhysicalSend(Connection transportConnection, 
Command command) {
--- End diff --

dedicated log message please add to the logger class


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-04 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352103#comment-16352103
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165898998
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -713,12 +730,16 @@ public void reconnect(AMQConnectionContext 
existingContext, ConnectionInfo info)
   context.incRefCount();
}
 
+   private static void traceSendCommand(Command command) {
--- End diff --

If you want a dedicated log message please add it to the Logger


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-04 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16352101#comment-16352101
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1849#discussion_r165898852
  
--- Diff: 
artemis-protocols/artemis-openwire-protocol/src/main/java/org/apache/activemq/artemis/core/protocol/openwire/OpenWireConnection.java
 ---
@@ -713,12 +730,16 @@ public void reconnect(AMQConnectionContext 
existingContext, ConnectionInfo info)
   context.incRefCount();
}
 
+   private static void traceSendCommand(Command command) {
+  ActiveMQServerLogger.LOGGER.trace("sending " + command);
+   }
+
/**
 * This will answer with commands to the client
 */
public boolean sendCommand(final Command command) {
-  if (ActiveMQServerLogger.LOGGER.isTraceEnabled()) {
- ActiveMQServerLogger.LOGGER.trace("sending " + command);
+  if (ACTIVE_MQ_SERVER_LOGGER_TRACE_ENABLED) {
--- End diff --

Was this really needed? From a code consistency POV, this is an odd one 
out, either all should follow in the code or not.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-03 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351347#comment-16351347
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on the issue:

https://github.com/apache/activemq-artemis/pull/1849
  
I'm running the some sanity tests on it.
perf-wire now OpenWire is just 10/20% less than Core protocol for non 
durable messages (depending on message size), it has a much better scalability 
(like Core) and memory footprint thanks to transport buffer pooling.


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-03 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16351346#comment-16351346
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

GitHub user franz1981 opened a pull request:

https://github.com/apache/activemq-artemis/pull/1849

ARTEMIS-1656 OpenWire scalability improvements

It includes:
- direct transport buffer pooling
- groupId SimpleString pooling
- exclusive OpenWireFormat per session and connection (in/out) to avoid 
contention
- cached trace check on JBoss server logger to avoid contention
- changed lastSent volatile set into lazy set to avoid full barrier cost on 
x86
- stateless OpenWireMessageConverter

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/franz1981/activemq-artemis open_wire_tests

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/activemq-artemis/pull/1849.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1849


commit 75564032def0c1aa79de824a51e53eab0c2a8791
Author: Francesco Nigro 
Date:   2018-02-03T07:03:36Z

ARTEMIS-1656 OpenWire scalability improvements

It includes:
- direct transport buffer pooling
- groupId SimpleString pooling
- exclusive OpenWireFormat per session and connection (in/out) to avoid 
contention
- cached trace check on JBoss server logger to avoid contention
- changed lastSent volatile set into lazy set to avoid full barrier cost on 
x86
- stateless OpenWireMessageConverter




> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-02 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350778#comment-16350778
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 closed the pull request at:

https://github.com/apache/activemq-artemis/pull/1842


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-02 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350777#comment-16350777
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on the issue:

https://github.com/apache/activemq-artemis/pull/1842
  
It is broking some test.. I will close it for now :+1: 


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-02 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350746#comment-16350746
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

GitHub user franz1981 opened a pull request:

https://github.com/apache/activemq-artemis/pull/1842

ARTEMIS-1656 OpenWire scalability improvements

The improvements included are:
- exclusive OpenWireFormat instances for input/output message paths
- direct ByteBuf pooling for messages sent across connection
- groupId pooling
- stateless OpenWire converter

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/franz1981/activemq-artemis 
open_wire_scalability

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/activemq-artemis/pull/1842.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1842


commit c2a9f884edead8634cea0e872b6252924db1a420
Author: Francesco Nigro 
Date:   2018-02-02T11:59:16Z

ARTEMIS-1656 OpenWire scalability improvements

The improvements included are:
- exclusive OpenWireFormat instances for input/output message paths
- direct ByteBuf pooling for messages sent across connection
- groupId pooling
- stateless OpenWire converter




> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1656) OpenWire scalability improvements

2018-02-02 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16350747#comment-16350747
 ] 

ASF GitHub Bot commented on ARTEMIS-1656:
-

Github user franz1981 commented on the issue:

https://github.com/apache/activemq-artemis/pull/1842
  
Do not merge it yet: need to run the OpenWire compatibility tests 


> OpenWire scalability improvements
> -
>
> Key: ARTEMIS-1656
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1656
> Project: ActiveMQ Artemis
>  Issue Type: Improvement
>Reporter: Francesco Nigro
>Assignee: Francesco Nigro
>Priority: Minor
>
> OpenWire is using the synchronized OpenWireFormat instance belonging to 
> OpenWireProtocolManager: it won't allow to scale with the number of 
> connections.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)