Dan, There's one issue with your configuration which doesn't affect the load balancing but will cause problems with receiving messages from the brokers. In the address.prefix, you use "foo.#". This is a pure prefix and it should simply be "foo". The wildcards are coming in the next release but are not implemented in the code you are using.
Regarding your actual question. I assume that you are testing this configuration under light load (i.e. sending one message at a time). The way that the balancing works is that it will route to the consumer (broker) with the fewest outstanding deliveries + inter-router cost. This means that it will favor the local broker over the remote one if there are no in-flight deliveries. The default (and minimum) cost for an inter-router connection is 1. You can set it to a higher value in the listener or connector. If you are sending one-at-a-time synchronous sends, they will always go to the local broker because the broker's zero outstanding deliveries will always be less than the inter-router cost of 1. If you send multiple deliveries asynchronously, you will see them being distributed to both brokers in the network. You can make the local-affinity stronger by increasing the inter-router cost. -Ted On Tue, Aug 1, 2017 at 3:47 AM, Dan Langford <[email protected]> wrote: > Last week I had a thread with lots of little questions around Dispatch > Routers. Ted Ross has been awesome to answer most of those. As a result I > feel like I have my QDR config shaping up a bit better. HOWEVER with some > more very focused questions I thought it would be best to start a new > thread. The problem I am seeing is that the routers are not distributing > the message load across other brokers on the network. Here is a little > diagram: > > > RouterA-03 ----> ArtemisBrokerA > | > | > | > \/ > RouterB-05 ----> ArtemisBrokerB > > > *NOTE: i am currently using Qpid Dispatch Router from a RHEL repo. v 0.7.0* > > > Connections from clients come in through an F5 VIP which forwards those > connections to either host L-03-A or L-05-B. Each of those hosts have a > Qpid Dispatch Router installed in front of an Artemis broker. dispatch > router on L-05-B is listening on an additional port that dispatch router > L-03-A connects to for inter-router communication. > > If I go around my F5 VIP so i know I am connecting straight to L-05-B and I > send any number of messages into the router there all of those messages end > up in ArtemisBrokerB. I was hoping that some would go to BrokerB and some > would go via RouterA over to BrokerA. Now when BrokerB is taken down > CURRENTLY the messages sent to Router L-05-B ARE routed through Router > L-03-A and then out to ArtemisBrokerB. (Currently receiving messages pull > them in from both hosts so my questions only lies in message production at > the moment.) > > Do you agree that with this configuration you would expect messages to be > load balanced between the two routes? I would like to paste in some of our > config and results from running qdstat. If you observe something that is > misconfigured and are able to highlight it i would be very appreciative. > First the config files for QDR instances. I tried my best to scrub them of > specific IPs and hostnames. > > *L-03-A qrouterd.conf* > > > *router {* > > * mode: interior* > > * id: Router.A* > > *}* > > *log {* > > * module: DEFAULT* > > * enable: debug+* > > * timestamp: yes* > > *}* > > *sslProfile {* > > * name: my-ssl* > > * certFile: /opt/org/my-ssl-info.pem* > > * keyFile: /opt/org/my-ssl-info.pem* > > * password: hellokitty42* > > *}* > > *listener {* > > * role: normal* > > * host: 0.0.0.0* > > * port: 5671* > > * authenticatePeer: no* > > * saslMechanisms: ANONYMOUS* > > * sslProfile: my-ssl* > > *}* > > *connector {* > > * name: local-artemis* > > * role: route-container* > > * host: L-03-A* > > * port: 61616* > > * saslMechanisms: ANONYMOUS* > > *}* > > *connector {* > > * name: routerb* > > * role: inter-router* > > * host: L-05-B* > > * port: 6671* > > * saslMechanisms: ANONYMOUS* > > *}* > > *address {* > > * prefix: foo.#* > > * waypoint: yes* > > * distribution: balanced* > > *}* > > *autoLink {* > > * addr: foo.bar* > > * dir: in* > > * connection: local-artemis* > > *}* > > *autoLink {* > > * addr: foo.bar* > > * dir: out* > > * connection: local-artemis* > > *}* > > > *L-05-B qrouterd.conf* > > *router {* > > * mode: interior* > > * id: Router.B* > > *}* > > *log {* > > * module: DEFAULT* > > * enable: debug+* > > * timestamp: yes* > > *}* > > *sslProfile {* > > * name: my-ssl* > > * certFile: /opt/org/my-ssl-info.pem* > > * keyFile: /opt/org/my-ssl-info.pem* > > * password: hellokitty42* > > *}* > > *listener {* > > * role: normal* > > * host: 0.0.0.0* > > * port: 5671* > > * authenticatePeer: no* > > * saslMechanisms: ANONYMOUS* > > * sslProfile: my-ssl* > > *}* > > *listener {* > > * role: inter-router* > > * host: 0.0.0.0* > > * port: 6671* > > * authenticatePeer: no* > > * saslMechanisms: ANONYMOUS* > > * sslProfile: my-ssl* > > *}* > > *connector {* > > * name: local-artemis* > > * role: route-container* > > * host: L-05-B* > > * port: 61616* > > * saslMechanisms: ANONYMOUS* > > *}* > > *address {* > > * prefix: foo.#* > > * waypoint: yes* > > * distribution: balanced* > > *}* > > *autoLink {* > > * addr: foo.bar* > > * dir: in* > > * connection: local-artemis* > > *}* > > *autoLink {* > > * addr: foo.bar* > > * dir: out* > > * connection: local-artemis* > > *}* > > > > > *L-03-A ~]$ qdstat -b:5671 -g* > > *Router Statistics* > > * attr value* > > * =====================* > > * Mode interior* > > * Area 0* > > * Router Id Router.A* > > > *L-05-B ~]$ qdstat -b:5671 -g* > > *Router Statistics* > > * attr value* > > * =====================* > > * Mode interior* > > * Area 0* > > * Router Id Router.B* > > > > > *L-03-A ~]$ qdstat -b:5671 -c* > > *Connections* > > * Id host container role > dir security authentication* > > * > ============================================================ > ======================================================* > > * 14 L-03-A:61616 L-03-A > route-container out no-security anonymous-user* > > * 107132 L-05-B:6671 Router.B > inter-router out no-security anonymous-user* > > * 107267 127.0.0.1:37916 <http://127.0.0.1:37916> > 180851ba-7ce4-44d0-a937-97de16e781c4 normal in no-security > anonymous-user* > > > *L-05-B ~]$ qdstat -b:5671 -c* > > *Connections* > > * Id host container role > dir security authentication* > > * > ============================================================ > ======================================================* > > * 1 L-05-B:61616 L-05-B > route-container out no-security anonymous-user* > > * 2 10.<ip>.32:57182 Router.A > inter-router in no-security anonymous-user* > > * 143 127.0.0.1:48610 <http://127.0.0.1:48610> > 93e6b3e3-443e-43d9-9c2c-0d648266ec19 normal in no-security > anonymous-user* > > > > > *L-03-A ~]$ qdstat -b:5671 -n* > > *Routers in the Network* > > * router-id next-hop link* > > * ===========================* > > * Router.A (self) -* > > * Router.B - 0* > > > *L-05-B ~]$ qdstat -b:5671 -n* > > *Routers in the Network* > > * router-id next-hop link* > > * ===========================* > > * Router.A - 0* > > * Router.B (self) -* > > > > > *L-03-A ~]$ qdstat -b:5671 --linkroutes* > > *Link Routes* > > > *L-05-B ~]$ qdstat -b:5671 --linkroutes* > > *Link Routes* > > > > > *L-03-A ~]$ qdstat -b:5671 --autolinks* > > *AutoLinks* > > * addr dir phs link status lastErr* > > * ==========================================* > > * foo.bar in 1 8 active* > > * foo.bar out 0 9 active* > > > *L-05-B ~]$ qdstat -b:5671 --autolinks* > > *AutoLinks* > > * addr dir phs link status lastErr* > > * ==========================================* > > * foo.bar in 1 4 active* > > * foo.bar out 0 5 active* > > > > > *L-03-A ~]$ qdstat -b:5671 -l* > > *Router Links* > > * type dir conn id id peer class addr > phs cap undel unsett del presett acc rej rel mod admin oper* > > * > ============================================================ > ============================================================ > ========================* > > * endpoint in 14 8 mobile foo.bar 1 > 250 0 0 80 0 80 0 0 0 enabled up* > > * endpoint out 14 9 mobile foo.bar 0 > 250 0 0 40 0 40 0 0 0 enabled up* > > * router-control in 107132 15 > 250 0 0 524 524 0 0 0 0 enabled up* > > * router-control out 107132 16 local qdhello > 250 0 0 527 527 0 0 0 0 enabled up* > > * inter-router in 107132 17 > 250 0 0 0 0 0 0 0 0 enabled up* > > * inter-router out 107132 18 > 250 0 0 0 0 0 0 0 0 enabled up* > > * endpoint in 107336 33 mobile $management 0 > 250 0 0 1 0 1 0 0 0 enabled up* > > * endpoint out 107336 34 local temp.liHSN39azHuUPkg > 250 0 0 0 0 0 0 0 0 enabled up* > > > *L-05-B ~]$ qdstat -b:5671 -l* > > *Router Links* > > * type dir conn id id peer class addr > phs cap undel unsett del presett acc rej rel mod admin oper* > > * > ============================================================ > ============================================================ > ========================* > > * endpoint in 1 4 mobile foo.bar 1 > 250 40 0 0 0 0 0 0 0 enabled up* > > * endpoint out 1 5 mobile foo.bar 0 > 250 0 0 0 0 0 0 0 0 enabled up* > > * router-control out 2 6 local qdhello > 250 0 0 524 524 0 0 0 0 enabled up* > > * router-control in 2 7 > 250 0 0 527 527 0 0 0 0 enabled up* > > * inter-router out 2 8 > 250 0 0 0 0 0 0 0 0 enabled up* > > * inter-router in 2 9 > 250 0 0 0 0 0 0 0 0 enabled up* > > * endpoint in 212 24 mobile $management 0 > 250 0 0 1 0 1 0 0 0 enabled up* > > * endpoint out 212 25 local temp.BJPESwMGFo+7qiy > 250 0 0 0 0 0 0 0 0 enabled up* > > > > > > *L-03-A ~]$ qdstat -b:5671 -a* > > *Router Addresses* > > * class addr phs distrib in-proc local remote > cntnr in out thru to-proc from-proc* > > * > ============================================================ > ======================================================* > > * local $_management_internal closest 1 0 0 > 0 0 0 0 6 6* > > * local $displayname closest 1 0 0 > 0 0 0 0 0 0* > > * mobile $management 0 closest 1 0 0 > 0 9 0 0 9 0* > > * local $management closest 1 0 0 > 0 0 0 0 0 0* > > * router Router.B closest 0 0 1 > 0 0 0 19 0 19* > > * mobile foo.bar 1 balanced 0 0 0 > 0 80 520 0 0 0* > > * mobile foo.bar 0 balanced 0 1 1 > 0 0 40 0 0 0* > > * local qdhello flood 1 1 0 > 0 0 0 0 269,611 269,622* > > * local qdrouter flood 1 0 0 > 0 0 0 0 12 0* > > * topo qdrouter flood 1 0 1 > 0 0 0 8,992 8,991 8,996* > > * local qdrouter.ma <http://qdrouter.ma> multicast 1 > 0 0 0 0 0 0 4 0* > > * topo qdrouter.ma <http://qdrouter.ma> multicast 1 > 0 1 0 0 0 3 0 3* > > * local temp.u6NMF1AFV0JpdBi closest 0 1 0 > 0 0 0 0 0 0* > > > *L-05-B ~]$ qdstat -b:5671 -a* > > *Router Addresses* > > * class addr phs distrib in-proc local remote > cntnr in out thru to-proc from-proc* > > * > ============================================================ > =====================================================* > > * local $_management_internal closest 1 0 0 > 0 0 0 0 6 6* > > * local $displayname closest 1 0 0 > 0 0 0 0 0 0* > > * mobile $management 0 closest 1 0 0 > 0 9 0 0 9 0* > > * local $management closest 1 0 0 > 0 0 0 0 0 0* > > * router Router.A closest 0 0 1 > 0 0 0 6 0 6* > > * mobile foo.bar 1 balanced 0 0 0 > 0 0 0 0 0 0* > > * mobile foo.bar 0 balanced 0 1 1 > 0 0 0 0 0 0* > > * local qdhello flood 1 1 0 > 0 0 0 0 570 571* > > * local qdrouter flood 1 0 0 > 0 0 0 0 4 0* > > * topo qdrouter flood 1 0 1 > 0 0 0 20 21 21* > > * local qdrouter.ma <http://qdrouter.ma> multicast 1 > 0 0 0 0 0 0 4 0* > > * topo qdrouter.ma <http://qdrouter.ma> multicast 1 > 0 1 0 0 0 0 0 1* > > * local temp.xwHdS0zzku0s3lR closest 0 1 0 > 0 0 0 0 0 0* > > > > is there something off in my configuration? or in my understand of QDR > netwokin just not up to snuff? should i care a little bit less about load > balancing because the VIP will round robin new connections out to different > codes? > > Thank you so much for your time and insight >
