I'm scheduling this to be on a google meeting room: https://meet.google.com/jga-zxrq-zgs
let me know if anyone would have a problem on using google meeting. (I think you must have a google account to be able to join). So, if you can't open a google account for any reason, let me know and I can arrange an alternative. that will be Wed 3PM (15:00) UK time. On Mon, Oct 12, 2020 at 9:51 AM michael.andre.pearce <[email protected]> wrote: > > > If im correct thats 3pm uk time. If so im good ibe provisionally reserved an > hour in my calendar.Thanks Clebert.Sent from my Samsung Galaxy smartphone. > -------- Original message --------From: Clebert Suconic > <[email protected]> Date: 12/10/2020 03:15 (GMT+00:00) To: > [email protected] Subject: Re: [DISCUSS] AMQP connectivity new features > Wed 10AM EST works? I can send in meeting information for whoever is > interested.On Sat, Oct 10, 2020 at 3:38 PM Michael André > Pearce<[email protected]> wrote:>> Hi Clebert>> Seems quite > some work bud! Nice...>> As This is quite a big feature. To help us > understand and go through bits can we organise a call for those interested. > It would significantly help me for sure to then be able to review and give > feedback, similar to what we did with federation.>> Best> Mike>> Sent from my > iPad>> > On 9 Oct 2020, at 15:49, Clebert Suconic <[email protected]> > wrote:> >> > I can definitely rename the interface.> >> > I did not want the > interface exposed to users outside of the context> > of the mirroring I'm > doing here. for other usages users can get what> > they want from broker > plugins.> >> >> > I will rename this as Mirror> >> >> > and then I can have a > MirrorSource and MirrorTarget> >> >> > WDYT?> >> >> On Thu, Oct 8, 2020 at > 8:21 AM Gary Tully <[email protected]> wrote:> >>> >> Hi Clebert, this is > a great piece of work.> >>> >> One thing popped out for me from the PR, the > remoteControl interface.> >>> >> I know naming is hard :-). That name to me > means something is controlling> >> from afar. But I think it is really a set > of change events that a broker> >> can emit. If I understand correctly, a > ``mirror`` AMQP outbound connection> >> is turning those events/commands into > amqp messages on an address that a> >> receiver is using to replicate a > selective state.> >> Another listener could use it for generating > notifications or audits?> >>> >> Should it be called ChangeEvents? or > ChangeActions - because there are> >> operations for each event.> >>> >> > further on that, if a mirror is making a broker replica, I could imagine a> > >> change event for a new security role or for a role deletion event. Does> > >> this sort of thing fit in with your idea of a RemoteControl?> >>> >> I > guess I am getting ahead a little, as in all events that change a broker> >> > are potentially relevant to a mirror, and if some extra plugins are in> >> > play, maybe some extra events need to get replicated.> >>> >> In other words, > this is a concrete specific implementation of a more> >> general concept, > change data capture. To make it extensible would require> >> being able to > publish events and on the receiver end, find a handler or> >> ignore events > that are unrecognised. It would need some conventions etc.> >>> >> In any > event, something like that is now possible :-)> >>> >> /gary> >>> >> On Wed, > 7 Oct 2020 at 23:14, Clebert Suconic <[email protected]>> >> wrote:> > >>> >>> sorry for the SPAM: I meant to add the link:> >>>> >>> > https://github.com/apache/activemq-artemis/pull/3294> >>>> >>> On Wed, Oct 7, > 2020 at 4:17 PM Clebert Suconic> >>> <[email protected]> wrote:> > >>>>> >>>> I have posted a Pull Request.. .(for review only at this time)> > >>>>> >>>> I would appreciate reviews.> >>>>> >>>> I'm fixing some issues, > adding docs.. but the overall is ready for> >>>> code review already.> >>>>> > >>>> Thank you> >>>>> >>>> On Thu, Oct 1, 2020 at 9:55 AM Clebert Suconic> > >>>> <[email protected]> wrote:> >>>>>> >>>>> I am working on new > features in the AMQP Protocol Handler, and I would> >>>>> like to explain > here what I am doing:> >>>>>> >>>>>> >>>>> I needed to go beyond what we do > now with AMQP, and use it connect> >>>>> brokers, and eventually with > qpid-dispatch for more elaborate> >>>>> networking on datacenters.> >>>>>> > >>>>> First part:> >>>>>> >>>>> The first issue I had while implementing > something protocol specific> >>>>> (other than Core) was to use the classes > within the protocol package.> >>>>> As the broker is agnostic to the > protocols (exception is core ATM), I> >>>>> needed the whole initialization > to take place within> >>>>> artemis-amqp-protocol.> >>>>>> >>>>> To get > around that I added what I called ProtocolServices. So when you> >>>>> > implement a protocol service the interface will be called within> >>>>> > broker start.> >>>>>> >>>>>> >>>>> Second part:> >>>>>> >>>>> As the broker > is now taking action on connecting to other brokers (or> >>>>> other AMQP > servers), I needed to change out Netty bootstrapping to> >>>>> allow outgoing > connections with a given protocol. With that I'm> >>>>> inverting everything > we had for server, and I am abusing the power of> >>>>> AMQP here. Since AMQP > is totally symmetrical I can now connect brokers> >>>>> directly. So, a > Consumer on the broker and be connected directly to a> >>>>> producer on > another broker. Acting like a bridge, but way more> >>>>> lightweight.> > >>>>>> >>>>>> >>>>> Third part:> >>>>>> >>>>> I am using that for replicas. > At the moment these replicas will not> >>>>> sync the client, but they are > working quite nicely. and allowing> >>>>> multiple replicas.> >>>>>> >>>>>> > >>>>>> >>>>> The configuration will take part as the following.> >>>>>> >>>>> > You define amqp connections with their URL and their reconnecting> >>>>> > information, for each connection you can define:> >>>>>> >>>>> sender > <matching addresses>: All addresses matching will have a> >>>>> transfer > sender. This will act like a push bridge> >>>>> receiver < matching > addresses>: All matching addresses on this broker> >>>>> will create a > consumer pulling messages. This will act like a pull> >>>>> bridge> >>>>> > peer <matching addresses>: This will create both ways, but you can> >>>>> > only use this towards a special server that knows how to handle this.> >>>>> > (e.g. qpid-dispatch). otherwise you get ping pongs on transfers> >>>>> copy > <matching addresses> : This will create a replica, without> >>>>> sending the > acks, (You will be responsible to consume the messages> >>>>> yourself at the > target broker)> >>>>> replica <matching addresses>: Just like copy, but acks > are sent and> >>>>> messages removed upong ack (or accept)> >>>>>> >>>>>> > >>>>> This is an example of the configuration:> >>>>>> >>>>> > <amqp-connections>> >>>>> <!-- the connection towards another broker -->> > >>>>> <amqp-connection uri="tcp://otherNode:5671" name="myconnection"> > >>>>> retry-interval="333" reconnect-attempts="33">> >>>>> <sender > match="TEST-SENDER" />> >>>>> <receiver match="TEST-RECEIVER" />> >>>>> > <peer match="TEST-PEER"/>> >>>>> <copy match="TEST-COPY"/>> >>>>> > <replica match="TEST-REPLICA"/>> >>>>> </amqp-connection>> >>>>> > </amqp-connections>> >>>>>> >>>>>> >>>>> so far these are being worked on a > my github fork/ but it's not really> >>>>> meant for a review at this point > as I have some cleanup to do (some> >>>>> logging messages and stuff I have > to remove).> >>>>>> >>>>> I am getting ready to send a Pull Request early > next week. but you> >>>>> have the concepts here already to review them.> > >>>>>> >>>>>> >>>>> --> >>>>> Clebert Suconic> >>>>> >>>>> >>>>> >>>> --> > >>>> Clebert Suconic> >>>> >>>> >>>> >>> --> >>> Clebert Suconic> >>>> >> >> > >> > --> > Clebert Suconic-- Clebert Suconic -- Clebert Suconic
