[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-08-23 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16590180#comment-16590180
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on the issue:

https://github.com/apache/activemq-artemis/pull/2262
  
@cshannon we have an issue where we have reported message count that never 
get dispatched, it was thought it was metric issue only, e.g. difference 
between whats in the messageferences and the counter (e.g. the messages aren't 
really there), but it looks like a more serious issue with the messages not 
being delivered for one reason or another and the messages really are in 
messageferences. They fun of a not very easy to replicate issue.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
> Fix For: 2.5.0
>
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-08-23 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1659#comment-1659
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/2262
  
@michaelandrepearce - Do you have an actual use case or example where it's 
not reliable?  The new metric gets used for size stuff as well so I don't think 
it should be changed for consistency and should be fixed if there is an issue

However I also see you closed this out so is there not actually an issue 
here?


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
> Fix For: 2.5.0
>
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-08-23 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589861#comment-16589861
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce closed the pull request at:

https://github.com/apache/activemq-artemis/pull/2262


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
> Fix For: 2.5.0
>
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-08-23 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589860#comment-16589860
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

GitHub user michaelandrepearce opened a pull request:

https://github.com/apache/activemq-artemis/pull/2262

ARTEMIS-1663 fix disparity on messagecount

Reverting a change done in 1663.

It seems the new messagecount != the original messageReferences.size all 
the time, it is more reliable to use the original messageReferences size as 
this is the actual collection's size.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/michaelandrepearce/activemq-artemis 
AREMIS1663-FIX

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/activemq-artemis/pull/2262.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2262


commit 2f9e30cde211950bf00cd9a00c05e09ae4dc6fea
Author: Michael André Pearce 
Date:   2018-08-23T07:50:34Z

ARTEMIS-1663 fix disparity on messagecount

Reverting a change done in 1663.

It seems the new messagecount != the original messageReferences.size all 
the time, it is more reliable to use the original messageReferences size as 
this is the actual collection's size.




> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
> Fix For: 2.5.0
>
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-14 Thread Christopher L. Shannon (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364199#comment-16364199
 ] 

Christopher L. Shannon commented on ARTEMIS-1663:
-

[~clebertsuconic] - good catch, I had increased that encodeSize because I 
initially was storing a size value there in an early version of this patch but 
then took it out and apparently forgot to revert back the encodeSize to the 
original 

> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
> Fix For: 2.5.0
>
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-14 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16364189#comment-16364189
 ] 

ASF subversion and git services commented on ARTEMIS-1663:
--

Commit 9b104930c25beeba646d92ab38c6038a11f1e227 in activemq-artemis's branch 
refs/heads/master from Clebert Suconic
[ https://git-wip-us.apache.org/repos/asf?p=activemq-artemis.git;h=9b10493 ]

ARTEMIS-1663 Fixing Encoding on PageCountPendingImpl

The PageCountPendingImpl was increasing the encode size without using its full 
allocation.
This was causing issues on replication as the encode is also used to determine 
the size of the packets.
however the packets were not receive the full allocated data causing missing 
packets on the replication
and test failures.

This is fixing the issue


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
> Fix For: 2.5.0
>
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-08 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357231#comment-16357231
 ] 

ASF subversion and git services commented on ARTEMIS-1663:
--

Commit ea70af15a3fbfc1b6ac86589e1e2e04a79ca3e23 in activemq-artemis's branch 
refs/heads/master from [~cshannon]
[ https://git-wip-us.apache.org/repos/asf?p=activemq-artemis.git;h=ea70af1 ]

ARTEMIS-1663 - Add new message count and size metrics

Adding new metrics for tracking message counts and sizes on a Queue.
This includes tracking metrics for pending, delivering and scheduled
messages.  The paging store also tracks message size now.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-08 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357233#comment-16357233
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user asfgit closed the pull request at:

https://github.com/apache/activemq-artemis/pull/1853


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-08 Thread ASF subversion and git services (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357232#comment-16357232
 ] 

ASF subversion and git services commented on ARTEMIS-1663:
--

Commit 58c4660426eceabecfaecaad42716562d5e2fd92 in activemq-artemis's branch 
refs/heads/master from Clebert Suconic
[ https://git-wip-us.apache.org/repos/asf?p=activemq-artemis.git;h=58c4660 ]

ARTEMIS-1663 Tweak on Paging test

Making sure it JournalCompatibilityTest/paging is actually paging


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355830#comment-16355830
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
Had to update the PR again as something got messed up when rebasing


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355808#comment-16355808
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166700570
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionImpl.java
 ---
@@ -439,7 +453,7 @@ public void confirmPosition(final Transaction tx, final 
PagePosition position) t
public void ackTx(final Transaction tx, final PagedReference reference) 
throws Exception {
   confirmPosition(tx, reference.getPosition());
 
-  counter.increment(tx, -1);
+  counter.increment(tx, -1, -getPersistentSize(reference));
--- End diff --

@michaelandrepearce  - So because of this line here we can't really default 
the persistentSize to -1, it needs to be 0.   This call here will store a new 
PageCountRecordInc() record to the journal with a negative count and negative 
size on message ack to decrement the counters. Because of this a negative can 
actually be a valid value and really 0 should mean not set (we should never 
have a message size of 0)


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355801#comment-16355801
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@clebertsuconic - I did some cleanup work and I added a couple of basic 
tests to the compatibility test.  It doesn't do much but show that the current 
snapshot can load a 2.4 journal successfully (and it can load the paging store 
as well).  We can expand the tests with more if you want.

Also, I removed the persistentSize from the PageCountPendingImpl class as 
the size (and the count apparently) aren't actually used and not needed.  On 
recovery we can recover the size from the paged reference.  

Do you think this is ready to merge?


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355535#comment-16355535
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166636796
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -62,21 +67,41 @@ public long getPageID() {
   return pageID;
}
 
+   /**
+* @return the persistentSize
+*/
+   @Override
+   public long getPersistentSize() {
+  return persistentSize;
+   }
+
+   /**
+* @param persistentSize the persistentSize to set
+*/
+   public void setPersistentSize(long persistentSize) {
+  this.persistentSize = persistentSize;
+   }
+
@Override
public int getEncodeSize() {
-  return DataConstants.SIZE_LONG * 2;
+  return DataConstants.SIZE_LONG * 3;
}
 
@Override
public void encode(ActiveMQBuffer buffer) {
   buffer.writeLong(queueID);
   buffer.writeLong(pageID);
+  buffer.writeLong(persistentSize);
--- End diff --

JournalCompatibilityTest under compatibility-tests.


I don't think the test is using paging. we should just extend the test to 
use paging.


I can help you with that if you have questions.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355515#comment-16355515
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166632761
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -62,21 +67,41 @@ public long getPageID() {
   return pageID;
}
 
+   /**
+* @return the persistentSize
+*/
+   @Override
+   public long getPersistentSize() {
+  return persistentSize;
+   }
+
+   /**
+* @param persistentSize the persistentSize to set
+*/
+   public void setPersistentSize(long persistentSize) {
+  this.persistentSize = persistentSize;
+   }
+
@Override
public int getEncodeSize() {
-  return DataConstants.SIZE_LONG * 2;
+  return DataConstants.SIZE_LONG * 3;
}
 
@Override
public void encode(ActiveMQBuffer buffer) {
   buffer.writeLong(queueID);
   buffer.writeLong(pageID);
+  buffer.writeLong(persistentSize);
--- End diff --

I can add a compatibility test. The value will just be a default 
(@michaelandrepearce wants me to use -1 instead of 0 and i guess we just would 
ignore it if not set for the count).

Where's the journal test located in the test suite?


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355503#comment-16355503
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166630869
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -62,21 +67,41 @@ public long getPageID() {
   return pageID;
}
 
+   /**
+* @return the persistentSize
+*/
+   @Override
+   public long getPersistentSize() {
+  return persistentSize;
+   }
+
+   /**
+* @param persistentSize the persistentSize to set
+*/
+   public void setPersistentSize(long persistentSize) {
+  this.persistentSize = persistentSize;
+   }
+
@Override
public int getEncodeSize() {
-  return DataConstants.SIZE_LONG * 2;
+  return DataConstants.SIZE_LONG * 3;
}
 
@Override
public void encode(ActiveMQBuffer buffer) {
   buffer.writeLong(queueID);
   buffer.writeLong(pageID);
+  buffer.writeLong(persistentSize);
--- End diff --

OIC.. you already implemented it..


I would like to have a compatibility test validating that.. there's a 
journal test on the new compatibility testsuite already... but not one with 
paging.. and validating the counters...

If you have a way to do it.. it would be great.. otherwise let me know and 
I will do it.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355500#comment-16355500
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166630379
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -62,21 +67,41 @@ public long getPageID() {
   return pageID;
}
 
+   /**
+* @return the persistentSize
+*/
+   @Override
+   public long getPersistentSize() {
+  return persistentSize;
+   }
+
+   /**
+* @param persistentSize the persistentSize to set
+*/
+   public void setPersistentSize(long persistentSize) {
+  this.persistentSize = persistentSize;
+   }
+
@Override
public int getEncodeSize() {
-  return DataConstants.SIZE_LONG * 2;
+  return DataConstants.SIZE_LONG * 3;
}
 
@Override
public void encode(ActiveMQBuffer buffer) {
   buffer.writeLong(queueID);
   buffer.writeLong(pageID);
+  buffer.writeLong(persistentSize);
--- End diff --

@clebertsuconic - here is an example of where the size is now persisted


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355496#comment-16355496
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@clebertsuconic - The metric in the page counters is persisted along with 
the page counters now.  When the page counters are written to the journal that 
metric is encoded to disk and reloaded on restart.

 Older versions of the journal are compatible, they will just not have that 
size value loaded as it won't have been saved previously.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355475#comment-16355475
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@cshannon although looking at the PR now you added persistent size into 
page counters... but as I read it, it is not persistent...


We should go all the way in.. i.e... persist the new size metric along with 
the page counters... or all the way back.. i.e... only give user's paging 
metrics through page size... 

Implementing the counter on the journal is no big deal.. but I would like 
to keep it compatible... older versions of the journal should still be 
compatible.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355408#comment-16355408
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
Oh ok I get it now.  Yeah you are right that the PagingStore interface has 
a method to get the number of pages and the page size which can be used to get 
an idea of consumption.  This value is exposed through AddressControl so users 
can see it.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355388#comment-16355388
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
I was just saying paging metrics should be exposed. Even if the files are 
to be GCed but if something happened and they are not the user should have the 
metrics exposed about pages.  I think we do have them but i don’t remember now. 
I’m away from a computer and I can’t check now. 


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355384#comment-16355384
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@clebertsuconic - I don't think we need to re-calculate anything if the 
values are missing for paging.  The  point of this metric isn't to give the 
user information on storage consumption as that data already exists.  They can 
find out how many page files are being used and disk usage already, etc.

The point of these metrics is to know exactly how much data on a queue is 
left to consume.  So there might be a 5 megabyte page file but only 2 messages 
left to consume and the rest of the file is just waiting to be GC'd.  This 
metric only includes the 2 messages to be consumed and not the rest of the file.



> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355380#comment-16355380
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
I think paging should be measured just in number of pages (files) on the 
address. That would give users an idea about storage consumption.  It wouldn’t 
indeed be practical to load it to calculate sizes. 


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355376#comment-16355376
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166601621
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountRecordInc.java
 ---
@@ -26,17 +26,20 @@
 
private int value;
 
+   private long persistentSize;
--- End diff --

@michaelandrepearce  - That is fine I will make those values -1, but I 
don't think we would try and re-calculate a correct value.  When loading the 
journal you'd have to re-load the messages to calculate from the paging store 
which we don't want to do.  I think the proper behavior is to just ignore it 
and not use the value in the calculations.  Then over time as new records come 
in the values would be filled in and correct.  

If I change the value to -1 on the record, are you thinking we should just 
have the getter do the negative check and return 0 if it's -1 or were you 
thinking to just return -1 from the getter and then check for negative 
everywhere else the value is used?




> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-07 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355123#comment-16355123
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166545869
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountRecordInc.java
 ---
@@ -26,17 +26,20 @@
 
private int value;
 
+   private long persistentSize;
--- End diff --

Seems my comment is removed due to update. Should be -1.

Reason for this is that if it isn't set e.g. have read up an old journal as 
such decode wont set it, and you know you need to calculate it to get a correct 
value.




> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354536#comment-16354536
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
I updated the PR with getPersistentSize() everywhere along with comments to 
try and make it more clear what the value represented.  I can tweak it some 
more if people still think it's not clear.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354492#comment-16354492
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
Ok I can just use getPersistentSize() then for everything, I will update 
the PR shortly.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354489#comment-16354489
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
I would include the sum you mentioned for large message.  The large message 
file size plus the large message encode size.  That translates fine as 
getPersjstentsjze


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354477#comment-16354477
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@clebertsuconic  - hmm so I'm starting to like you original idea of just 
using getEncodeSize() more as the the message might be non-durable and in that 
case the encode size is still used but it isn't on disk so that would be a 
little misleading.

However there is still the issue of what to do aboutt large messages?  
Maybe we need a name like fullEncodeSize() or something so that it is implies 
it is the full message including the body?


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354413#comment-16354413
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
getPersistentSize() is fine with me as it specifies that the size 
represents the entire persisted size of the message


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354353#comment-16354353
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@cshannon you may not like getEncodeSize().. but doing 
message.getMessageSize() doesn't make it any clearer..we need to find a better 
name? maybe getPersistentSize()?


message.get *Message* size to me has the same meaning of message.getSize().


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354349#comment-16354349
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@clebertsuconic - Fair enough, but I think I should call it 
getMessageSize() instead (i already did this in a couple places)  The reason 
why I would use that name is because the value is slightly different than 
getEncodeSize(). For the normal message case, getMessageSize() and 
geEncodeSize() are equal.  But for a large message getMessageSize() will be 
getEncodeSize() + the body size.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354341#comment-16354341
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
i would rename getSize() as getEncodeSize() on the interfaces then. getSize 
could be dubious and users may get the wrong expectation of the meaning.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354319#comment-16354319
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166401530
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -43,6 +44,8 @@ public PageCountPendingImpl(long queueID, long pageID, 
int inc) {
 
long pageID;
 
+   long size;
--- End diff --

@michaelandrepearce - did you have in mind having the getter in the class 
handle the -1 (ie have the getSize() method check for -1 and return 0 if 
negative) or having the getSize() method return the -1 as is and then have the 
code using the value explicitly do the check for negative?


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354235#comment-16354235
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166388367
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -43,6 +44,8 @@ public PageCountPendingImpl(long queueID, long pageID, 
int inc) {
 
long pageID;
 
+   long size;
--- End diff --

The issue is backwards compatibility.  These values are now being stored to 
the journal and old records won't have a size.  So by defaulting to a 0 value 
for when an old record is it makes everything simple.  Yes I could make it -1 
but then I have to add checks for -1 versus just using 0 as a default and then 
the logic is clean as we don't need to handle negative edge cases.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354223#comment-16354223
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166387776
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -43,6 +44,8 @@ public PageCountPendingImpl(long queueID, long pageID, 
int inc) {
 
long pageID;
 
+   long size;
--- End diff --

Eg it’s typical for -1 to be used to represent null vs 0 as no size , 
slight semantic difference but important none the less


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354222#comment-16354222
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166387479
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -43,6 +44,8 @@ public PageCountPendingImpl(long queueID, long pageID, 
int inc) {
 
long pageID;
 
+   long size;
--- End diff --

There’s a difference between no size (0) and not being set (-1) the logic 
calculating should be aware of this as -1 means it would need to be calculated


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354219#comment-16354219
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166386744
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -43,6 +44,8 @@ public PageCountPendingImpl(long queueID, long pageID, 
int inc) {
 
long pageID;
 
+   long size;
--- End diff --

Why would we use -1 and not 0? These values default to 0 because we don't 
want the counter to change if the size is not set.  If we default to -1 then 
the counters will go negative where as if they are 0 when not set then nothing 
changes with the counters which is what we want.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354213#comment-16354213
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166385486
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountRecord.java
 ---
@@ -26,18 +26,21 @@
 
private long value;
 
+   private long size;
--- End diff --

Default to -1


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354214#comment-16354214
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166385388
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountPendingImpl.java
 ---
@@ -43,6 +44,8 @@ public PageCountPendingImpl(long queueID, long pageID, 
int inc) {
 
long pageID;
 
+   long size;
--- End diff --

Should default be -1


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354212#comment-16354212
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166385592
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountRecordInc.java
 ---
@@ -26,17 +26,20 @@
 
private int value;
 
+   private long size;
--- End diff --

Default to -1


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354198#comment-16354198
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@clebertsuconic - No, I'm not trying to measure the size on the JVM.  I 
want to measure the size of the messages in general (including how much space 
is on disk) as a metric to know how much data is left to consume on the Queue.  
The most accurate way is to use the encode size as this is the amount of space 
taken up on disk.  For large messages this also includes the body size.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354131#comment-16354131
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
You are getting the getEncodeSize() as the size... what is what you want to 
measure? how much memory it uses on the JVM?

We have some calculations we do on paging.. perhaps that's what you want to 
expose?

perhaps we should also review those calculations.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354104#comment-16354104
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@michaelandrepearce - ok made those 2 changes so it is ready for another 
look


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354043#comment-16354043
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166346352
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/PagedReferenceImpl.java
 ---
@@ -53,6 +53,8 @@
 
private Object protocolData;
 
+   private Long messageSize;
--- End diff --

Yep, I'll set this to -1 to mean not set.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354042#comment-16354042
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166346269
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionImpl.java
 ---
@@ -1309,4 +1336,68 @@ public void remove() {
   public void close() {
   }
}
+
+
+   private static class PagePositionSize {
--- End diff --

I will fix this and extend PagePosition as if the position is incremented 
that is fine as the size will just go back to 0 unless set.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353964#comment-16353964
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166328137
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/PagedReferenceImpl.java
 ---
@@ -53,6 +53,8 @@
 
private Object protocolData;
 
+   private Long messageSize;
--- End diff --

Could this be primitive? and either -1 or 0 to denote empty or not set in 
logic, just objects = GC, and we're putting effort in to remove them on the hot 
path as much as possible to keep GC pressure as low as possible.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353957#comment-16353957
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166325405
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionCounterImpl.java
 ---
@@ -374,8 +403,43 @@ private ItemOper(PageSubscriptionCounterImpl counter, 
long id, int add) {
   @Override
   public void afterCommit(Transaction tx) {
  for (ItemOper oper : operations) {
-oper.counter.incrementProcessed(oper.id, oper.amount);
+oper.counter.incrementProcessed(oper.id, oper.amount, 
oper.size);
  }
   }
}
+
+   private static class PendingCounter {
+  final long id;
+  final AtomicInteger count;
--- End diff --

Yes this i what i meant , ill have a look at the updated PR in a 
bit/tomorrow.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353955#comment-16353955
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166325203
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionImpl.java
 ---
@@ -1309,4 +1336,68 @@ public void remove() {
   public void close() {
   }
}
+
+
+   private static class PagePositionSize {
--- End diff --

But this new object still holds PagePosition so its not been removed.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353835#comment-16353835
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
@michaelandrepearce - I updated my PR, take a look and let me know what you 
think.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353816#comment-16353816
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166283882
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionCounterImpl.java
 ---
@@ -374,8 +403,43 @@ private ItemOper(PageSubscriptionCounterImpl counter, 
long id, int add) {
   @Override
   public void afterCommit(Transaction tx) {
  for (ItemOper oper : operations) {
-oper.counter.incrementProcessed(oper.id, oper.amount);
+oper.counter.incrementProcessed(oper.id, oper.amount, 
oper.size);
  }
   }
}
+
+   private static class PendingCounter {
+  final long id;
+  final AtomicInteger count;
--- End diff --

I'm about to push an updated PR using the field updater classes.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353768#comment-16353768
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166272476
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionImpl.java
 ---
@@ -1309,4 +1336,68 @@ public void remove() {
   public void close() {
   }
}
+
+
+   private static class PagePositionSize {
--- End diff --

The reason I didn't extend is because PagePosition has methods like 
nextMessage() which increment the message number, etc.  If that is called then 
the stored size doesn't really make sense anymore as it won't match which is 
why I just created a new object.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353767#comment-16353767
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166272177
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountRecordIncV2.java
 ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.persistence.impl.journal.codec;
+
+import org.apache.activemq.artemis.api.core.ActiveMQBuffer;
+import org.apache.activemq.artemis.utils.DataConstants;
+
+public class PageCountRecordIncV2 extends PageCountRecordInc {
--- End diff --

Also a good point, I can get rid of the extra objects and just do a check 
on the buffer for readableBytes()  is greater than 0. Thanks.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353766#comment-16353766
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166271726
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/PageSubscription.java
 ---
@@ -161,4 +165,9 @@
 * @throws Exception
 */
void onDeletePage(Page deletedPage) throws Exception;
+
+   AtomicLong getDeliveredCount();
--- End diff --

Good point, I will make that change.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-06 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353764#comment-16353764
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166271561
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionCounterImpl.java
 ---
@@ -374,8 +403,43 @@ private ItemOper(PageSubscriptionCounterImpl counter, 
long id, int add) {
   @Override
   public void afterCommit(Transaction tx) {
  for (ItemOper oper : operations) {
-oper.counter.incrementProcessed(oper.id, oper.amount);
+oper.counter.incrementProcessed(oper.id, oper.amount, 
oper.size);
  }
   }
}
+
+   private static class PendingCounter {
+  final long id;
+  final AtomicInteger count;
--- End diff --

What do you mean by atomic updater, do you mean using something like 
AtomicIntegerFieldUpdater ?


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353312#comment-16353312
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166179369
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/PageSubscription.java
 ---
@@ -161,4 +165,9 @@
 * @throws Exception
 */
void onDeletePage(Page deletedPage) throws Exception;
+
+   AtomicLong getDeliveredCount();
--- End diff --

not sure would want to expose an atomic outside the object as it allows 
other code to update it unexpectedly, should probably be  primitive and the 
object should return get on the atomic.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353309#comment-16353309
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166179104
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/server/impl/QueuePendingMessageMetrics.java
 ---
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.server.impl;
+
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import com.google.common.base.Preconditions;
+
+import org.apache.activemq.artemis.core.server.ActiveMQServerLogger;
+import org.apache.activemq.artemis.core.server.MessageReference;
+import org.apache.activemq.artemis.core.server.Queue;
+
+public class QueuePendingMessageMetrics {
+
+   private final AtomicInteger messageCount = new AtomicInteger(0);
--- End diff --

Could all these atomics be primitives with static atomic updates, just to 
reduce object size.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353304#comment-16353304
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166178773
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountRecordV2.java
 ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.persistence.impl.journal.codec;
+
+import org.apache.activemq.artemis.api.core.ActiveMQBuffer;
+import org.apache.activemq.artemis.utils.DataConstants;
+
+public class PageCountRecordV2 extends PageCountRecord {
--- End diff --

Ditto as per other comment if v2 is needed.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353303#comment-16353303
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166178732
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/persistence/impl/journal/codec/PageCountRecordIncV2.java
 ---
@@ -0,0 +1,72 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.activemq.artemis.core.persistence.impl.journal.codec;
+
+import org.apache.activemq.artemis.api.core.ActiveMQBuffer;
+import org.apache.activemq.artemis.utils.DataConstants;
+
+public class PageCountRecordIncV2 extends PageCountRecordInc {
--- End diff --

Is a new V2 needed? could have added size to PageCountRecord default to , 
and checked for readable bytes before read the long, to be back compatible.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353301#comment-16353301
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166178431
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionImpl.java
 ---
@@ -1309,4 +1336,68 @@ public void remove() {
   public void close() {
   }
}
+
+
+   private static class PagePositionSize {
--- End diff --

Why not just extend PagePosition (saves on objects)


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16353300#comment-16353300
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user michaelandrepearce commented on a diff in the pull request:

https://github.com/apache/activemq-artemis/pull/1853#discussion_r166178175
  
--- Diff: 
artemis-server/src/main/java/org/apache/activemq/artemis/core/paging/cursor/impl/PageSubscriptionCounterImpl.java
 ---
@@ -374,8 +403,43 @@ private ItemOper(PageSubscriptionCounterImpl counter, 
long id, int add) {
   @Override
   public void afterCommit(Transaction tx) {
  for (ItemOper oper : operations) {
-oper.counter.incrementProcessed(oper.id, oper.amount);
+oper.counter.incrementProcessed(oper.id, oper.amount, 
oper.size);
  }
   }
}
+
+   private static class PendingCounter {
+  final long id;
+  final AtomicInteger count;
--- End diff --

could this be an int, with a static atomic updater (ditto with the atomic 
below also)


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352728#comment-16352728
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

Github user cshannon commented on the issue:

https://github.com/apache/activemq-artemis/pull/1853
  
This PR is quite large so I would appreciate if multiple people take a look 
to make sure it looks ok.


> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (ARTEMIS-1663) Add enhanced message count and size metrics for Queues

2018-02-05 Thread ASF GitHub Bot (JIRA)

[ 
https://issues.apache.org/jira/browse/ARTEMIS-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352726#comment-16352726
 ] 

ASF GitHub Bot commented on ARTEMIS-1663:
-

GitHub user cshannon opened a pull request:

https://github.com/apache/activemq-artemis/pull/1853

ARTEMIS-1663 - Add new message count and size metrics

Adding new metrics for tracking message counts and sizes on a Queue.
This includes tracking metrics for pending, delivering and scheduled
messages.  The paging store also tracks message size now.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cshannon/activemq-artemis ARTEMIS-1663

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/activemq-artemis/pull/1853.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1853


commit ce0b980b4e90a5c95ed88ac697d87cac0605789c
Author: Christopher L. Shannon (cshannon) 
Date:   2018-02-05T18:24:31Z

ARTEMIS-1663 - Add new message count and size metrics

Adding new metrics for tracking message counts and sizes on a Queue.
This includes tracking metrics for pending, delivering and scheduled
messages.  The paging store also tracks message size now.




> Add enhanced message count and size metrics for Queues
> --
>
> Key: ARTEMIS-1663
> URL: https://issues.apache.org/jira/browse/ARTEMIS-1663
> Project: ActiveMQ Artemis
>  Issue Type: New Feature
>  Components: Broker
>Affects Versions: 2.4.0
>Reporter: Christopher L. Shannon
>Assignee: Christopher L. Shannon
>Priority: Major
>
> The purpose of this Jira is to enhance the message count metrics and to add 
> size metrics for Queues.   By size metrics I mean the encoded size of all the 
> messages (for large messages this will include body as well).  Right now we 
> track memory usage but it is also very useful to know exactly how much data 
> is pending on a Queue to be consumed.  
> The scope of this will be support for tracking the count/size of pending 
> messages, count/size of delivering messages, and count/size of scheduled 
> messages.  For each of these categories durable size metrics will be tracked 
> as well. The paging store will also support tracking of size metrics as well.
> These new metrics will be exposed through JMX to allow users to view the 
> statistics.  Also by tracking these new values it will allow for enhanced 
> features in the future (ie new flow control options)
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)