Justin and Martyn have cleaned up the branch, and everything is ready to be pushed upstream...
so, what I will be doing now: i - push the cleaned branch as master upstream. this will include a commit to bump the poms to 2.0.0-SNAPSHOT ii - I will remove the temporary branch. iii - I will fork the current tip of master as 1.x And... BTW: everybody need to check on this.. this is a massive amount of work.. Awesome work guys... It is really nice to see this coming in shape this way. On Wed, Dec 7, 2016 at 4:11 PM, Christopher Shannon <[email protected]> wrote: > +1 for merging the branch into master after the cleanup is done and bumping > to 2.0 since it is a major architecture change. > > > On Wed, Dec 7, 2016 at 3:31 PM, Justin Bertram <[email protected]> wrote: > >> Essential feature parity with 5.x (where it makes sense) has been a goal >> all along, but I think waiting until such parity exists before the next >> major release means the community will be waiting quite a bit longer than >> they already have. Meanwhile, new functionality that could benefit the >> community will remain unavailable. In any event, "feature parity" is a bit >> vague. If there is something specific with regards to 5.x parity that >> you're looking for then I think you should make that explicit so it can be >> evaluated. >> >> I'm in favor of merging the addressing changes onto master, hardening >> things up a bit, and then releasing. >> >> >> Justin >> >> ----- Original Message ----- >> From: "Matt Pavlovich" <[email protected]> >> To: [email protected] >> Sent: Wednesday, December 7, 2016 2:04:13 PM >> Subject: Re: [DISCUSS] Artemis addressing improvements, JMS component >> removal and potential 2.0.0 >> >> IMHO, I think it would be good to kick up a thread on what it means to >> be 2.0. It sounds like the addressing changes definitely warrant it on >> its own, but I'm thinking having ActiveMQ 5.x feature parity would be a >> good goal for the 2.0 release. My $0.02 >> >> On 12/7/16 2:56 PM, Clebert Suconic wrote: >> > +1000 >> > >> > >> > It needs one final cleanup before it can be done though.. these commit >> > messages need meaninful descriptions. >> > >> > if Justin or Martyn could come up with that since they did most of the >> > work on the branch. >> > >> > This will really require bumping the release to 2.0.0 (there's a >> > 2.0.snapshot commit on it already). I would merge this into master, >> > and fork the current master as 1.x. >> > >> > >> > >> > >> > On Wed, Dec 7, 2016 at 1:52 PM, Timothy Bish <[email protected]> >> wrote: >> >> This would be a good time to move to master, would allow other to more >> >> easily get onboard >> >> >> >> >> >> On 12/07/2016 01:25 PM, Clebert Suconic wrote: >> >>> I have rebased ARTEMIS-780 in top of master. There was a lot of >> >>> conflicts... >> >>> >> >>> I have aggregated/squashed most of the commits by chronological order >> >>> almost. So if Martyn had 10 commits in series I had squashed all of >> >>> them, since they were small comments anyways. The good thing about >> >>> this is that nobody would lose authorship of these commits. >> >>> >> >>> We will need to come up with more meaningful messages for these >> >>> commits before we can merge into master. But this is getting into a >> >>> very good shape. I'm impressed by the amount of work I see done on >> >>> this branch. Very well done guys! I mean it! >> >>> >> >>> Also, I have saved the old branch before I pushed -f into my fork as >> >>> old-ARTEMIS-780 in case I broke anything on the process. Please check >> >>> everything and let me know if I did. >> >>> >> >>> >> >>> And please rebase more often on this branch unless you merge it soon. >> >>> >> >>> >> >>> On Mon, Nov 28, 2016 at 2:36 PM, Clebert Suconic >> >>> <[email protected]> wrote: >> >>>> If / when we do the 2.0 bump, I would like to move a few classes. >> >>>> Mainly under server.impl... I would like to move activations under a >> >>>> package for activation, replicationendpoints for a package for >> >>>> replications... some small stuff like that just to reorganize >> >>>> little things like this a bit. >> >>>> >> >>>> We can't do that now as that would break API and compatibility, but if >> >>>> we do the bump, I would like to make that simple move. >> >>>> >> >>>> On Thu, Nov 24, 2016 at 4:41 AM, Martyn Taylor <[email protected]> >> >>>> wrote: >> >>>>> Hi Matt, >> >>>>> >> >>>>> Comments inline. >> >>>>> >> >>>>> On Mon, Nov 21, 2016 at 7:02 PM, Matt Pavlovich <[email protected]> >> >>>>> wrote: >> >>>>> >> >>>>>> Martyn- >> >>>>>> >> >>>>>> I think you nailed it here-- well done =) >> >>>>>> >> >>>>>> My notes in-line-- >> >>>>>> >> >>>>>> On 11/21/16 10:45 AM, Martyn Taylor wrote: >> >>>>>> >> >>>>>>> 1. Ability to route messages to queues with the same address, but >> >>>>>>> different >> >>>>>>> routing semantics. >> >>>>>>> >> >>>>>>> The proposal outlined in ARTEMIS-780 outlines a new model that >> >>>>>>> introduces >> >>>>>>> an address object at the configuration and management layer. In the >> >>>>>>> proposal it is not possible to create 2 addresses with different >> >>>>>>> routing >> >>>>>>> types. This causes a problem with existing clients (JMS, STOMP and >> for >> >>>>>>> compatability with other vendors). >> >>>>>>> >> >>>>>>> Potential Modification: Addresses can have multiple routing type >> >>>>>>> “endpoints”, either “multicast” only, “anycast” only or both. The >> >>>>>>> example >> >>>>>>> below would be used to represent a JMS Topic called “foo”, with a >> >>>>>>> single >> >>>>>>> subscription queue and a JMS Queue called “foo”. N.B. The actual >> XML >> >>>>>>> is >> >>>>>>> just an example, there are multiple ways this could be represented >> >>>>>>> that we >> >>>>>>> can define later. >> >>>>>>> >> >>>>>>> <*addresses*> <*address* *name**="foo"*> <*anycast*> >> >>>>>>> <*queues*> <*queue* *name**="**foo”* /> >> </*queues*> >> >>>>>>> </*anycast*> <*mulcast*> <*queues*> >> >>>>>>> <*queue* *name**="my.topic.subscription" */> </*queues*> >> >>>>>>> </*multicast*> </*address*></*addresses*> >> >>>>>>> >> >>>>>> I think this solves it. The crux of the issues (for me) boils down >> to >> >>>>>> auto-creation of destinations across protocols. Having this show up >> in >> >>>>>> the >> >>>>>> configs would give developers and admins more information to >> >>>>>> troubleshoot >> >>>>>> the mixed address type+protocol scenario. >> >>>>>> >> >>>>>> 2. Sending to “multicast”, “anycast” or “all” >> >>>>>>> As mentioned earlier JMS (and other clients such as STOMP via >> >>>>>>> prefixing) >> >>>>>>> allow the producer to identify the type of end point it would like >> to >> >>>>>>> send >> >>>>>>> to. >> >>>>>>> >> >>>>>>> If a JMS client creates a producer and passes in a topic with >> address >> >>>>>>> “foo”. Then only the queues associated with the “multicast” >> section of >> >>>>>>> the >> >>>>>>> address. A similar thing happens when the JMS producer sends to a >> >>>>>>> “queue” >> >>>>>>> messages should be distributed amongst the queues associated with >> the >> >>>>>>> “anycast” section of the address. >> >>>>>>> >> >>>>>>> There may also be a case when a producer does not identify the >> >>>>>>> endpoint >> >>>>>>> type, and simply sends to “foo”. AMQP or MQTT may want to do this. >> In >> >>>>>>> this >> >>>>>>> scenario both should happen. All the queues under the multicast >> >>>>>>> section >> >>>>>>> get >> >>>>>>> a copy of the message, and one queue under the anycast section gets >> >>>>>>> the >> >>>>>>> message. >> >>>>>>> >> >>>>>>> Modification: None Needed. Internal APIs would need to be updated >> to >> >>>>>>> allow >> >>>>>>> this functionality. >> >>>>>>> >> >>>>>> I think the "deliver to all" scenario should be fine. This seems >> >>>>>> analogous >> >>>>>> to a CompositeDestination in ActiveMQ 5.x. I'll map through some >> >>>>>> scenarios >> >>>>>> and report back any gotchas. >> >>>>>> >> >>>>>> 3. Support for prefixes to identify endpoint types >> >>>>>>> Many clients, ActiveMQ 5.x, STOMP and potential clients from >> alternate >> >>>>>>> vendors, identify the endpoint type (in producer and consumer) >> using a >> >>>>>>> prefix notation. >> >>>>>>> >> >>>>>>> e.g. queue:///foo >> >>>>>>> >> >>>>>>> Which would identify: >> >>>>>>> >> >>>>>>> <*addresses*> <*address* *name**="foo"*> <*anycast*> >> >>>>>>> <*queues*> <*queue* *name**="my.foo.queue" */> >> >>>>>>> </*queues*> </*anycast*> </*address*></*addresses*> >> >>>>>>> >> >>>>>>> Modifications Needed: None to the model. An additional parameter to >> >>>>>>> the >> >>>>>>> acceptors should be added to identify the prefix. >> >>>>>>> >> >>>>>> Just as a check point in the syntax+naming convention in your >> provided >> >>>>>> example... would the name actually be: >> >>>>>> >> >>>>>> <*queue* *name**="foo" .. vs "my.foo.queue" ? >> >>>>>> >> >>>>> The queue name can be anything. It's the address that is used by >> >>>>> consumer/producer. The protocol handler / broker will decided which >> >>>>> queue >> >>>>> to connect to. >> >>>>> >> >>>>>> 4. Multiple endpoints are defined, but client does not specify >> >>>>>> “endpoint >> >>>>>>> routing type” when consuming >> >>>>>>> >> >>>>>>> Handling cases where consumers does not pass enough information in >> >>>>>>> their >> >>>>>>> address or via protocol specific mechanisms to identify an >> endpoint. >> >>>>>>> Let’s >> >>>>>>> say an AMQP client, requests to subscribe to the address “foo”, but >> >>>>>>> passes >> >>>>>>> no extra information. In the cases where there are only a single >> >>>>>>> endpoint >> >>>>>>> type defined, the consumer would associated with that endpoint >> type. >> >>>>>>> However, when both endpoint types are defined, the protocol handler >> >>>>>>> does >> >>>>>>> not know whether to associate this consumer with a queue under the >> >>>>>>> “anycast” section, or whether to create a new queue under the >> >>>>>>> “multicast” >> >>>>>>> section. e.g. >> >>>>>>> >> >>>>>>> Consume: “foo” >> >>>>>>> >> >>>>>>> <*addresses*> >> >>>>>>> >> >>>>>>> <*address* *name**="foo"*> <*anycast*> >> <*queues*> >> >>>>>>> <*queue* *name**="**foo”* /> </*queues*> >> >>>>>>> </*anycast*> <*multicast*> <*queues*> >> <*queue* >> >>>>>>> *name**="my.topic.subscription" */> </*queues*> >> >>>>>>> </*multicast*> </*address*></*addresses*> >> >>>>>>> >> >>>>>>> In this scenario, we can make the default configurable on the >> >>>>>>> protocol/acceptor. Possible options for this could be: >> >>>>>>> >> >>>>>>> “multicast”: Defaults multicast >> >>>>>>> >> >>>>>>> “anycast”: Defaults to anycast >> >>>>>>> >> >>>>>>> “error”: Returns an error to the client >> >>>>>>> >> >>>>>>> Alternatively each protocol handler could handle this in the most >> >>>>>>> sensible >> >>>>>>> way for that protocol. MQTT might default to “multicast”, STOMP >> >>>>>>> “anycast”, >> >>>>>>> and AMQP to “error”. >> >>>>>>> >> >>>>>> Yep, this works great. I think there are two flags on the >> acceptors.. >> >>>>>> one >> >>>>>> for auto-create and one for default handling of name collision. The >> >>>>>> defaults would most likely be the same. >> >>>>>> >> >>>>>> Something along the lines of: >> >>>>>> auto-create-default = "multicast | anycast" >> >>>>>> no-prefix-default = "multicast | anycast | error" >> >>>>>> >> >>>>>> 5. Fully qualified address names >> >>>>>>> This feature allows a client to identify a particular address on a >> >>>>>>> specific >> >>>>>>> broker in a cluster. This could be achieved by the client using >> some >> >>>>>>> form >> >>>>>>> of address as: >> >>>>>>> >> >>>>>>> queue:///host/broker/address/ >> >>>>>>> >> >>>>>>> Matt could you elaborate on the drivers behind this requirement >> >>>>>>> please. >> >>>>>>> >> >>>>>>> I am of the opinion that this is out of the scope of the addressing >> >>>>>>> changes, and is more to do with redirecting in cluster scenarios. >> The >> >>>>>>> current model will support this address syntax if we want to use >> it in >> >>>>>>> the >> >>>>>>> future. >> >>>>>>> >> >>>>>> I agree that tackling the impl of this should be out-of-scope. My >> >>>>>> recommendation is to consider it in addressing now, so we can >> hopefully >> >>>>>> avoid any breakage down the road. >> >>>>>> >> >>>>>> A widely used feature in other EMS brokers (IBM MQ, Tibco EMS, etc) >> is >> >>>>>> the >> >>>>>> ability to fully address a destination using a format similar to >> this: >> >>>>>> >> >>>>>> queue://brokerB/myQueue >> >>>>>> >> >>>>> The advantage of this is to allow for scaling of the number of >> >>>>> destinations >> >>>>>> and allows for more dynamic broker networks to be created without >> >>>>>> applications having to have connection information for all brokers >> in a >> >>>>>> broker network. Think simple delivery+routing, and not horizontal >> >>>>>> scaling. >> >>>>>> It is very analogous to SMTP mail routing. >> >>>>>> >> >>>>>> Producer behavior: >> >>>>>> >> >>>>>> 1. Client X connects to brokerA and sends it a message addressed: >> >>>>>> queue://brokerB/myQueue >> >>>>>> 2. brokerA accepts the message on behalf of brokerB and handles all >> >>>>>> acknowledgement and persistence accordingly >> >>>>>> 3. brokerA would then store the message in a "queue" for brokerB. >> Note: >> >>>>>> All messages for brokerB are generally stored in one queue-- this is >> >>>>>> how it >> >>>>>> helps with destination scaling >> >>>>>> >> >>>>>> Broker to broker behavior: >> >>>>>> >> >>>>>> There are generally two scenarios: always-on or periodic-check >> >>>>>> >> >>>>>> In "always-on" >> >>>>>> 1. brokerA looks for a brokerB in its list of cluster connections >> and >> >>>>>> then >> >>>>>> sends all messages for all queues for brokerB (or brokerB pulls all >> >>>>>> messages, depending on cluster connection config) >> >>>>>> >> >>>>>> In "periodic-check" >> >>>>>> 1. brokerB connects to brokerA (or vice-versa) on a given time >> interval >> >>>>>> and then receives any messages that have arrived since last check >> >>>>>> >> >>>>>> TL;DR; >> >>>>>> >> >>>>>> It would be cool to consider remote broker delivery for messages >> while >> >>>>>> refactoring the address handling code. This would bring Artemis >> inline >> >>>>>> with >> >>>>>> the rest of the commercial EMS brokers. The impact now, hopefully, >> is >> >>>>>> minor >> >>>>>> and just thinking about default prefixes. >> >>>>>> >> >>>>> Understood, from our conversations on IRC I can see why this might be >> >>>>> useful. >> >>>>> >> >>>>>> Thanks, >> >>>>>> -Matt >> >>>>>> >> >>>>>> >> >>>>>> >> >>>> >> >>>> -- >> >>>> Clebert Suconic >> >>> >> >>> >> >> >> >> -- >> >> Tim Bish >> >> twitter: @tabish121 >> >> blog: http://timbish.blogspot.com/ >> >> >> > >> > >> >> -- Clebert Suconic
