[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread graben
Github user graben commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@andytaylor: Have a look into 
http://activemq.apache.org/how-do-i-use-jms-efficiently.html 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread mattrpav
Github user mattrpav commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@graben agreed.

Queue Consumer use cases:

# Connection(s) | # Queue(s) | Option |  Desired Behavior
--- |  | --- | -
 1 | 1 | | Single threaded, ordered message consumption
 1 | 1..n | | Single threaded, ordered consumption of multiple queues 
(comma separated today)
1..x | 1..y | Using connection pool | multithreaded, ordered consumption 
using shared connection pool (same clientId). Each thread getting ordered 
messages for their specific queues. 
2 | 1 | | Round-robin dispatch of message consumption 
2 | 1 | exclusiveConsumer | Single threaded, order consumption w/ HA 
standby consumer
2 | 1 | exclusiveConsumer+JMSXGroupId | Multi-threaded, order consumption 
w/ active-active consumers separated by header value




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@graben It sounds different, what is it a pool of, connections?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread graben
Github user graben commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@andytaylor: I mean the usage of one single pool which is used to 
send/receive concurrently to multiple topics and virtual topic consumers. Such 
are often used in integration infrastructures like e.g. Apache ServiceMix. You 
only have to configure one pool which addresses every purpose since you do not 
have to register clientId to the jms connections when consuming from queues 
only. I think that's different to pooled subscribed users in JMS 2.0.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@graben Im not sure what you mean by shared pooling but if you mean a pool 
of consumers subscribed to the same topic then thats what JMS 2.0 gives you. 
Maybe I'm misunderstanding what you mean by pooled connections.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread graben
Github user graben commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@andytaylor: One big advantage of queue semantics vs topic subscriptions is 
the usage of shared pooling. No need for clientId@JMSComnection. Every single 
pool connection can be used for every topic (sending) or every queue (receiving 
or sending). Maximum flexibility.  As far as I know this is still not addressed 
in JMS 2.0 API. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread rlodge
Github user rlodge commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@mtaylor What I'm hearing is that basically, following changes in 780, a 
client would be able to send to an address "RecordUpdateNotifications", other 
clients could subscribe to 
"RecordUpdateNotifications::SystemARecordUpdateNotifications" and it would act 
essentially like virtual topics do now; all "RecordUpdateNotifications::*" 
subscriptions would read from queues coming from the single address?

If so, do OpenWire clients need to use a specific queue / address naming 
scheme, or would it happen automatically somehow (and could it be mapped from 
the VirtualTopic.* naming scheme)?  It's not a terribly large problem for us to 
go through and change our queues from VirtualTopic.* to a slightly different 
naming scheme, though we'd rather change nothing about the clients; it would be 
more difficult to somehow change the API's we were calling...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread mtaylor
Github user mtaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@rlodge

So one thing we are working on right now is adding is the ability to 
directly address a queue from a consumer.  All of this is upcoming in 
Artemis-780 jIRA (which I will post more info on very soon).

But the key points are:

1. You'll be able to specify routing semantics on addresses.  e.g.



Essentially what this means is that any queue created with an address 
"myTopic" will get **all** messages sent to myTopic.  To create a subscription 
all we do internally in Artemis is create a new queue with address "myTopic".

To do Point to point or "queue" sematnics, you simply configure the address 
to have routing type anycast.



In Artemis if a consumer requests consuming from myQueue we look to see if 
a queue already exists, if so we add a consumer, other wise we create one and 
add a consumer.

As part of the work we'll also allow clients to specically reference a 
queue directly.  Using the pattern:

address::queueName

This means I want to consume from a specific queue called, "queueName" with 
address "address".  If the queue doesn't exist it will create it (providing you 
have autoCreateQueues enabled).  This allows you to have several clients, 
creating subscriptions to the same queue.

Whilst ARTEMIS-780 is meant to cover the changes to the CORE addressing 
model, to enable all this extra stuff, there's also additional tasks to offer 
this support over the various protocols.  To get ActiveMQ JMS 1.1 clients 
working, we'll also need to update the OpenWire protocol.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@rlodge cheers, makes sense. All that is implemented now, It just needs 
exposing in the protocol layers when an Openwire JMS client connects to a 
virtual topic. Like I mentioned before the addressing changes will make it 
easier to expose this as well.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread rlodge
Github user rlodge commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@andytaylor We have a single topic, like 
"VirtualTopic.RecordUpdateNotifications".  The system doesn't have a list of 
who's receiving it; downstream systems subscribe to Queues with the right name 
pattern and receive the updates.  To them, it's like any other Queue.  The 
upstream system and the broker configuration aren't modified when a new 
consumer comes on line.  The key is no server or application configuration to 
make it work, and also all queue features (especially message groups) being 
available.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@mattrpav thanks for that, for point 1, everything supported on a queue is 
supported on a subscription as thats actually what it is. Maybe Artemis is just 
not exposing some of these. Again with the new addressing changes this will be 
easy.  Im not sure 3 is applicable since Artemis routes to addresses and 
consumes from queues. 3 I am not sure I understand.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@rlodge so I think all that is doable without having to add the notion of a 
virtual topic. With the addressing changes this should be even easier to add to 
the Openwire protocol manager.

one thing you could explain is does "we want to be able to broadcast a 
message to an unknown number of different subscribers" mean to different 
addresses or just one address like myTopic?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread mattrpav
Github user mattrpav commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@andytaylor its a couple things beyond just shard topic subscriptions (JMS 
2.0). #3 being super critical and highly valuable.

1. Using queues instead of durable topic subscribers allows for all the 
add'l queue features-- exclusive consumers, message groups, etc. 

2. It separates the producer and the consumers into separate destinations 
in an automatic way, without requiring server-side configuration

3. #2 allows you to separate producing message flow from consuming message 
flow between brokers across network connectors. We implement both scenarios all 
the time:  replicate topic production across brokers, but separate consumers.. 
and the inverse.. share consumers across brokers, but not producers.

Competitor Note: IBM MQ supports the topic -> queue separation. The 
advantage ActiveMQ's Virtual Topics have is that it doesn't require server-side 
configuration for every topic -> queue setup and every new subscriber.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread rlodge
Github user rlodge commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
A discussion on the dev list would at the very least leave those of us who 
don't subscribe out of the discussion . . .

Our use case for virtual topics in ActiveMQ is fairly straightforward:  we 
want to be able to broadcast a message to an unknown number of different 
subscribers without doing something like camel multiplex on the application 
end.  The consumers must be able to do concurrent processing (multiple 
consumers) with queue semantics.  We need the consumers to be able to create 
and delete their own subscriptions without restarting the broker or doing any 
application configuration on our end; the consumers must be able to go offline 
and come back and find their messages waiting for them; waiting messages can't 
expire or be thrown away.  This probably has to be done with the latest 
ActiveMQ client library and without code changes.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread mtaylor
Github user mtaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@clebertsuconic +1

@mattrpav and @rlodge if you could provide us some more information about 
your use cases, we can see if current features or those coming in ARTEIMS-780 
can help out.  If not we can try to address them.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread mtaylor
Github user mtaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
With regards to the general Virtual Topics use case.  We're making some 
improvements to the Artemis addressing model.  See 
https://issues.apache.org/jira/browse/ARTEMIS-780.  I think these changes might 
make it easier to use equivilent "shared topic" behaviour.  

It will depend on the protocol you are using as to how this would behave.  
But, you should be able to either get direct access to a pre-configured queue 
and add another consumer on it.  The abilty to reference queues directly is 
something we'll be exposing.

Alternatively, you'll be able to subscribe to the same "durable" 
subscription as another client, by using the same client id (again protocol 
specific here).  For AMQP this would be the container-id, for JMS and STOMP 
client id.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread clebertsuconic
Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
IMHO we should move this discussion to the dev-list, with the following:

i - use cases
ii - required changes.
iii - how it's being implemented on the PR.

when I read these changes I don't have a clue if the 3 points here are 
being covered, and what performance implications (or even stability) to other 
regular queue cases.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread rlodge
Github user rlodge commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@andytaylor I'm not sure what you mean by "implements topic style 
addressing using queues"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread clebertsuconic
Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@rlodge  you can have as many consumers you like on a QueueSubscription 
with Artemis. 

You can limite the client-id if you like, but you can also bypass that.


The only use case I can see for artemis with Virtual topics is migration 
case... users migrating to artemis without changing code.. but I want to make 
sure I understand the requirements. What is needed? Route the virtual queue to 
another existing core queue? if that's the case it's pretty simple 
implementation. (I guess you would have most of what is needed on this PR)?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread mtaylor
Github user mtaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
Firstly we are making some major improvements to the Artemis addressing 
model that might help here.

@mattrpav 

>  selectorAware: This allows an interested subscriber to only receive 
messages in their queue that match a given selector. This broker-side 
pre-filtering is an optimization which reduces the on-disk requirements for 
really busy Virtual 

This is how normal Artemis subscriptions currently work.  The Artemis 
addressing/routing model is different from ActiveMQ.  Messages are routed 
straight to subscription queues, and filtering happens on the broker at routing 
(send) time.

> 2) transactedSend: This is a (imho & experience) critical reliability 
requirement. In use cases where reliability is a critical requirement: when a 
producer sends a message to the broker, even during non-transacted sessions, 
the broker should transact the dispatch to the consumer queues before ack'n the 
producer. This ensures no messages are lost in the event of unplanned outage 
while a broker is dispatching messages to the consumer queues.

We also already have this behaviour, but it works slightly differently.  We 
don't create a transaction that puts the message on all consumer queues then 
returns an ack.  Instead, we persist the message and return an ack (depending 
on the protocol behaviour).  In the event of the broker going down just before 
the message is persisted, but before routing, it will retry routing on restart. 
 Which gives you the same behaviour.

> 3) concurrentSend: This is a critical performance enhancement for Virtual 
Topics where there are a large number of consumer queues. The behavior is that 
the broker processes queue dispatch in parallel to consumer queues in order to 
improve throughput and lower latency.

I don't think I fully follow what you are saying here, but, in Artemis 
there's only consumer queues.  We route messages direct to consumer queues.  
All consumption of messages form those queues and dispatching to the consumer 
is done in parrallel using a shared thread pool.

> iii. While outside the scope of this PR, I thought I'd mention it here as 
it is related-- there has long been a desire to enhance ActiveMQ to support 
single-message store. That is, for all subscriptions, only store a single copy 
of a message. This becomes really apparent in use cases with a large number of 
Virtual Consumers.

We already do this.  A message is only persisted once, we then use 
reference counting to delete it once it's been consumed from all queues.





---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@rlodge In the core Artemis Broker there is no concept of Topics, they 
don't exist. Its just a facade on the client and a small amount of config and 
management. The core Broker only understands Addresses and queues, so when a 
'JMS topic' is created nothing actually exists. In reality when the client 
connects it creates a queue that acts as the subscription. That being the case 
adding topic functionality to the core might be the wrong approach, So I am 
trying to understand what tthe actual use case is and how it might fit better 
into the current core broker concept.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@rlodge Thats what I am asking :), Artemis supports JMS 2.0 and shared 
consumers and implements topic style addressing using queues anyway. I'm asking 
for an explanation as to what else is actually needed and how best this could 
be added without introducing the notion of a topic at the broker which 
currently doesn't exist?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread rlodge
Github user rlodge commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@andytaylor How does Artemis provide the same functionality by default 
(without configuration for each "virtual topic / queue" being added to the 
broker's configuration)?  This is one of the only (maybe the only) feature 
keeping us from attempting to migrate to Artemis from ActiveMQ.  I've looked 
through the documentation and haven't been able to find a similar feature other 
than diverts, which seem to need broker configuration and broker restart, which 
would be a dealbreaker, since our environment requires clients to to be able to 
dynamically create the queue (not just queue-like) destinations on the fly 
without broker reconfiguration or restart.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread andytaylor
Github user andytaylor commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@graben Could you explain why virtual topics need to be added. As far as I 
was aware in ActiveMQ its just to provide shared consumers and queue like 
semantics on a topic which Artemis does by default?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-04 Thread mattrpav
Github user mattrpav commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@graben 

re: Transacted: if the PostOffice is transacted by default, that's great. 
The option in 5.x felt more like a "shore-up" to get to transacted. I don't see 
a compelling reason to support non-transacted broker-side message movement.  A 
note in the migration guide would suffice.

re: Concurrency: I see this having value, and agree it should be a separate 
JIRA/PR.

re: Selector/Filter: that's great that you have it on your radar. You 
mentioned filter, so I feel obligated to clarify-- this selector is determined 
by the _consumer_ using the standard JMS selector available in the API. The 
broker then store these to properly filter messages if the consumer goes 
offline.  Does Artemis' filter support this behavior, or is filtering 
broker-side configuration?

Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-03 Thread graben
Github user graben commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
Well, I'm not sure at all but IMHO the actual implementation of PostOffice 
is transacted by default no way to deactivate. Concurrency seems a bit 
difficult to me to implement in this PR. Might implemented in general? The 
selector or filter (Artemis way) feature is on my todo list after the basis is 
approved.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-03 Thread mattrpav
Github user mattrpav commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
I'll detail out the use cases:

1) selectorAware:  This allows an interested subscriber to only receive 
messages in their queue that match a given selector. This broker pre-filtering 
is a server-side optimization which reduces the on-disk requirements for really 
busy Virtual Topics.

2) transactedSend: This is a (imho & experience) critical reliability 
requirement. In use cases where reliability is a crticial requirement: when a 
producer sends a message to the broker, even during non-transacted sessions, 
the broker should transact the dispatch to the consumer queues before ack'n the 
producer. This ensures no messages are lost in the event of unplanned outage 
while a broker is dispatching messages to the consumer queues.

3) concurrentSend: This is a critical performance enhancement for Virtual 
Topics where there are a large number of consumer queues. The behavior is that 
the broker processes queue dispatch in parallel to consumer queues in order to 
improve throughput and lower latency.  

[Notes]

i. transactedSend and concurrentSend are often used in conjunction

ii. selectorAware requires some sort of persistence store so selectors are 
maintained across restarts (specifically useful during maintenance and failover 
testing)

iii. While outside the scope of this PR, I thought I'd mention it here as 
it is related-- there has long been a desire to enhance ActiveMQ to support 
single-message store. That is, for all subscriptions, only store a single copy 
of a message. This becomes really apparent in use cases with a large number of 
Virtual Consumers.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-03 Thread graben
Github user graben commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
@mattrpav: I'm not sure these options can be implemented the way the 
virtual topic solution is integrated. Artemis works a bit different than 
ActiveMQ. But maybe someone with a deeper view into core code can answer this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-02 Thread clebertsuconic
Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
I am doing a release soon (this week, was supposed to be last week), so I 
won't merge this until 1.5.0 is out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-11-02 Thread mattrpav
Github user mattrpav commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
Has there been mention of the transactedSend and concurrentSend options? 
These are super useful for improving reliability and scalability when using 
Virtual Topics.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-10-30 Thread graben
Github user graben commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
Add some initial tests for better review


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---


[GitHub] activemq-artemis issue #875: ARTEMIS-550 Add virtual topic support

2016-10-29 Thread clebertsuconic
Github user clebertsuconic commented on the issue:

https://github.com/apache/activemq-artemis/pull/875
  
can you check this, @mtaylor .. also in reference to the addressing branch


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---