Wed 10AM EST works? I can send in meeting information for whoever is interested.
On Sat, Oct 10, 2020 at 3:38 PM Michael André Pearce <[email protected]> wrote: > > Hi Clebert > > Seems quite some work bud! Nice... > > As This is quite a big feature. To help us understand and go through bits can > we organise a call for those interested. It would significantly help me for > sure to then be able to review and give feedback, similar to what we did with > federation. > > Best > Mike > > Sent from my iPad > > > On 9 Oct 2020, at 15:49, Clebert Suconic <[email protected]> wrote: > > > > I can definitely rename the interface. > > > > I did not want the interface exposed to users outside of the context > > of the mirroring I'm doing here. for other usages users can get what > > they want from broker plugins. > > > > > > I will rename this as Mirror > > > > > > and then I can have a MirrorSource and MirrorTarget > > > > > > WDYT? > > > >> On Thu, Oct 8, 2020 at 8:21 AM Gary Tully <[email protected]> wrote: > >> > >> Hi Clebert, this is a great piece of work. > >> > >> One thing popped out for me from the PR, the remoteControl interface. > >> > >> I know naming is hard :-). That name to me means something is controlling > >> from afar. But I think it is really a set of change events that a broker > >> can emit. If I understand correctly, a ``mirror`` AMQP outbound connection > >> is turning those events/commands into amqp messages on an address that a > >> receiver is using to replicate a selective state. > >> Another listener could use it for generating notifications or audits? > >> > >> Should it be called ChangeEvents? or ChangeActions - because there are > >> operations for each event. > >> > >> further on that, if a mirror is making a broker replica, I could imagine a > >> change event for a new security role or for a role deletion event. Does > >> this sort of thing fit in with your idea of a RemoteControl? > >> > >> I guess I am getting ahead a little, as in all events that change a broker > >> are potentially relevant to a mirror, and if some extra plugins are in > >> play, maybe some extra events need to get replicated. > >> > >> In other words, this is a concrete specific implementation of a more > >> general concept, change data capture. To make it extensible would require > >> being able to publish events and on the receiver end, find a handler or > >> ignore events that are unrecognised. It would need some conventions etc. > >> > >> In any event, something like that is now possible :-) > >> > >> /gary > >> > >> On Wed, 7 Oct 2020 at 23:14, Clebert Suconic <[email protected]> > >> wrote: > >> > >>> sorry for the SPAM: I meant to add the link: > >>> > >>> https://github.com/apache/activemq-artemis/pull/3294 > >>> > >>> On Wed, Oct 7, 2020 at 4:17 PM Clebert Suconic > >>> <[email protected]> wrote: > >>>> > >>>> I have posted a Pull Request.. .(for review only at this time) > >>>> > >>>> I would appreciate reviews. > >>>> > >>>> I'm fixing some issues, adding docs.. but the overall is ready for > >>>> code review already. > >>>> > >>>> Thank you > >>>> > >>>> On Thu, Oct 1, 2020 at 9:55 AM Clebert Suconic > >>>> <[email protected]> wrote: > >>>>> > >>>>> I am working on new features in the AMQP Protocol Handler, and I would > >>>>> like to explain here what I am doing: > >>>>> > >>>>> > >>>>> I needed to go beyond what we do now with AMQP, and use it connect > >>>>> brokers, and eventually with qpid-dispatch for more elaborate > >>>>> networking on datacenters. > >>>>> > >>>>> First part: > >>>>> > >>>>> The first issue I had while implementing something protocol specific > >>>>> (other than Core) was to use the classes within the protocol package. > >>>>> As the broker is agnostic to the protocols (exception is core ATM), I > >>>>> needed the whole initialization to take place within > >>>>> artemis-amqp-protocol. > >>>>> > >>>>> To get around that I added what I called ProtocolServices. So when you > >>>>> implement a protocol service the interface will be called within > >>>>> broker start. > >>>>> > >>>>> > >>>>> Second part: > >>>>> > >>>>> As the broker is now taking action on connecting to other brokers (or > >>>>> other AMQP servers), I needed to change out Netty bootstrapping to > >>>>> allow outgoing connections with a given protocol. With that I'm > >>>>> inverting everything we had for server, and I am abusing the power of > >>>>> AMQP here. Since AMQP is totally symmetrical I can now connect brokers > >>>>> directly. So, a Consumer on the broker and be connected directly to a > >>>>> producer on another broker. Acting like a bridge, but way more > >>>>> lightweight. > >>>>> > >>>>> > >>>>> Third part: > >>>>> > >>>>> I am using that for replicas. At the moment these replicas will not > >>>>> sync the client, but they are working quite nicely. and allowing > >>>>> multiple replicas. > >>>>> > >>>>> > >>>>> > >>>>> The configuration will take part as the following. > >>>>> > >>>>> You define amqp connections with their URL and their reconnecting > >>>>> information, for each connection you can define: > >>>>> > >>>>> sender <matching addresses>: All addresses matching will have a > >>>>> transfer sender. This will act like a push bridge > >>>>> receiver < matching addresses>: All matching addresses on this broker > >>>>> will create a consumer pulling messages. This will act like a pull > >>>>> bridge > >>>>> peer <matching addresses>: This will create both ways, but you can > >>>>> only use this towards a special server that knows how to handle this. > >>>>> (e.g. qpid-dispatch). otherwise you get ping pongs on transfers > >>>>> copy <matching addresses> : This will create a replica, without > >>>>> sending the acks, (You will be responsible to consume the messages > >>>>> yourself at the target broker) > >>>>> replica <matching addresses>: Just like copy, but acks are sent and > >>>>> messages removed upong ack (or accept) > >>>>> > >>>>> > >>>>> This is an example of the configuration: > >>>>> > >>>>> <amqp-connections> > >>>>> <!-- the connection towards another broker --> > >>>>> <amqp-connection uri="tcp://otherNode:5671" name="myconnection" > >>>>> retry-interval="333" reconnect-attempts="33"> > >>>>> <sender match="TEST-SENDER" /> > >>>>> <receiver match="TEST-RECEIVER" /> > >>>>> <peer match="TEST-PEER"/> > >>>>> <copy match="TEST-COPY"/> > >>>>> <replica match="TEST-REPLICA"/> > >>>>> </amqp-connection> > >>>>> </amqp-connections> > >>>>> > >>>>> > >>>>> so far these are being worked on a my github fork/ but it's not really > >>>>> meant for a review at this point as I have some cleanup to do (some > >>>>> logging messages and stuff I have to remove). > >>>>> > >>>>> I am getting ready to send a Pull Request early next week. but you > >>>>> have the concepts here already to review them. > >>>>> > >>>>> > >>>>> -- > >>>>> Clebert Suconic > >>>> > >>>> > >>>> > >>>> -- > >>>> Clebert Suconic > >>> > >>> > >>> > >>> -- > >>> Clebert Suconic > >>> > > > > > > > > -- > > Clebert Suconic -- Clebert Suconic
