Hi Clebert, this is a great piece of work. One thing popped out for me from the PR, the remoteControl interface.
I know naming is hard :-). That name to me means something is controlling from afar. But I think it is really a set of change events that a broker can emit. If I understand correctly, a ``mirror`` AMQP outbound connection is turning those events/commands into amqp messages on an address that a receiver is using to replicate a selective state. Another listener could use it for generating notifications or audits? Should it be called ChangeEvents? or ChangeActions - because there are operations for each event. further on that, if a mirror is making a broker replica, I could imagine a change event for a new security role or for a role deletion event. Does this sort of thing fit in with your idea of a RemoteControl? I guess I am getting ahead a little, as in all events that change a broker are potentially relevant to a mirror, and if some extra plugins are in play, maybe some extra events need to get replicated. In other words, this is a concrete specific implementation of a more general concept, change data capture. To make it extensible would require being able to publish events and on the receiver end, find a handler or ignore events that are unrecognised. It would need some conventions etc. In any event, something like that is now possible :-) /gary On Wed, 7 Oct 2020 at 23:14, Clebert Suconic <[email protected]> wrote: > sorry for the SPAM: I meant to add the link: > > https://github.com/apache/activemq-artemis/pull/3294 > > On Wed, Oct 7, 2020 at 4:17 PM Clebert Suconic > <[email protected]> wrote: > > > > I have posted a Pull Request.. .(for review only at this time) > > > > I would appreciate reviews. > > > > I'm fixing some issues, adding docs.. but the overall is ready for > > code review already. > > > > Thank you > > > > On Thu, Oct 1, 2020 at 9:55 AM Clebert Suconic > > <[email protected]> wrote: > > > > > > I am working on new features in the AMQP Protocol Handler, and I would > > > like to explain here what I am doing: > > > > > > > > > I needed to go beyond what we do now with AMQP, and use it connect > > > brokers, and eventually with qpid-dispatch for more elaborate > > > networking on datacenters. > > > > > > First part: > > > > > > The first issue I had while implementing something protocol specific > > > (other than Core) was to use the classes within the protocol package. > > > As the broker is agnostic to the protocols (exception is core ATM), I > > > needed the whole initialization to take place within > > > artemis-amqp-protocol. > > > > > > To get around that I added what I called ProtocolServices. So when you > > > implement a protocol service the interface will be called within > > > broker start. > > > > > > > > > Second part: > > > > > > As the broker is now taking action on connecting to other brokers (or > > > other AMQP servers), I needed to change out Netty bootstrapping to > > > allow outgoing connections with a given protocol. With that I'm > > > inverting everything we had for server, and I am abusing the power of > > > AMQP here. Since AMQP is totally symmetrical I can now connect brokers > > > directly. So, a Consumer on the broker and be connected directly to a > > > producer on another broker. Acting like a bridge, but way more > > > lightweight. > > > > > > > > > Third part: > > > > > > I am using that for replicas. At the moment these replicas will not > > > sync the client, but they are working quite nicely. and allowing > > > multiple replicas. > > > > > > > > > > > > The configuration will take part as the following. > > > > > > You define amqp connections with their URL and their reconnecting > > > information, for each connection you can define: > > > > > > sender <matching addresses>: All addresses matching will have a > > > transfer sender. This will act like a push bridge > > > receiver < matching addresses>: All matching addresses on this broker > > > will create a consumer pulling messages. This will act like a pull > > > bridge > > > peer <matching addresses>: This will create both ways, but you can > > > only use this towards a special server that knows how to handle this. > > > (e.g. qpid-dispatch). otherwise you get ping pongs on transfers > > > copy <matching addresses> : This will create a replica, without > > > sending the acks, (You will be responsible to consume the messages > > > yourself at the target broker) > > > replica <matching addresses>: Just like copy, but acks are sent and > > > messages removed upong ack (or accept) > > > > > > > > > This is an example of the configuration: > > > > > > <amqp-connections> > > > <!-- the connection towards another broker --> > > > <amqp-connection uri="tcp://otherNode:5671" name="myconnection" > > > retry-interval="333" reconnect-attempts="33"> > > > <sender match="TEST-SENDER" /> > > > <receiver match="TEST-RECEIVER" /> > > > <peer match="TEST-PEER"/> > > > <copy match="TEST-COPY"/> > > > <replica match="TEST-REPLICA"/> > > > </amqp-connection> > > > </amqp-connections> > > > > > > > > > so far these are being worked on a my github fork/ but it's not really > > > meant for a review at this point as I have some cleanup to do (some > > > logging messages and stuff I have to remove). > > > > > > I am getting ready to send a Pull Request early next week. but you > > > have the concepts here already to review them. > > > > > > > > > -- > > > Clebert Suconic > > > > > > > > -- > > Clebert Suconic > > > > -- > Clebert Suconic >
