On 27 July 2017 at 02:56, Dan Langford <[email protected]> wrote: > Thanks For the reply. (I realize my diagrams had a small typo. Too many Ra > nodes whoops some should be Rb) I do have queues with that name on both > brokers. I'll make sure it's set to balanced. The "sharded" behavior you > went on to described is exactly our intent. We want high availability of > the service not necessarily high availability of the messages. We are ok > communicating to our users that a node may go down and those messages may > be unavailable until we get it back up. > > That being said we want to feel confident that if a node going down is a > fatal data corruption issue like a corrupt disk we definitely don't want to > lose those messages which have been persisted. We could configure Artemis > to store messages in a backed up DB or we could snapshot our VM regularly > but the approach we are investigating first is using an Artemis cluster > with colocation replication. That way in theory if we completely destroy a > disk we can via Artemis promote the backup to be primary and they will be > served again. > > (We tried using Artemis load balancing but it cannot load balance AMQP at > the moment) > > Another question crept up about transactions: using the Qpid-JMS-client we > cannot interact with our cluster here if we have session transacted=true. > Should we be able to do transacted sessions via an autoLink? Or do we have > to use a linkRoute? I may not know the difference > > As we work further on this we will explore XATransactions. I know we will > need to use another lib like the old Qpid client for Java but should we > expect XA transactions to work through dispatch Router? >
Further to Gordon's reply on the other thread, note that the router only supports AMQP 1.0 so you can't use the older JMS client with it as that only supports the AMQP 0-x protocols. > Thanks so much for your time and thought > On Wed, Jul 26, 2017 at 1:10 PM Ted Ross <[email protected]> wrote: > >> On Fri, Jul 21, 2017 at 7:12 PM, Dan Langford <[email protected]> >> wrote: >> >> > On Thu, Jul 20, 2017 at 9:58 AM Ted Ross <[email protected]> wrote: >> > >> > > On Wed, Jul 19, 2017 at 7:36 PM, Dan Langford <[email protected]> >> > > wrote: >> > > >> > > > > - Can I configure QDR to autoLink in and out ANY/ALL addresses? >> > > > No. There is no way currently for QDR to know what queues are >> present >> > on >> > > > its connected brokers. It would not be difficult to write a program >> to >> > > > synchronize autolinks to existing queues. >> > > >> > >> > You are right it wouldn't be that difficult. Also with artemis I can turn >> > on autocreation of queues and then then use QDR as the spot to manage >> what >> > queues can exist. Not bad. What about synchronizing autoLink config >> across >> > routers in a QDR network? are messages to the $management queue broadcast >> > throughout the cluster? i could always resend the necessary messages >> > through the _topo address namespace to get it to the other routers. >> > >> > >> > > > > - Artemis doesn't support vhosts. Can I configure connections to >> > > vhost:Foo >> > > > > address:bar actually be address:Foo.bar when the message goes back >> to >> > > the >> > > > > broker? >> > > > Yes. There is a multi-tenancy feature for listeners that does >> exactly >> > > what >> > > > you are asking for. If you add the attribute "multiTenant: yes" to >> the >> > > > configuration of a listener in the qdrouterd.conf file, clients >> > connected >> > > > via that listener will have their addresses annotated as vhost/addr >> in >> > > the >> > > > router. >> > > >> > >> > ok this is going to be perfect. i am starting to feel more comfortable >> > with everything in this config file >> > >> > > > - Can I configure QDR to pass auth through to the broker and let the >> > > broker >> > > > > decide is the user is authenticated and authorized? Inversely can I >> > > > > configure QDR to be the only determinate of auth? >> > > > Presently, QDR expects to be the sole determiner of authentic >> identity. >> > >> > > There is an open request to add a SASL proxy that might be used to >> allow >> > > > the broker to do authentication on behalf of the router, but that >> > hasn't >> > > > made it into master yet. >> > > >> > >> > this is one part that has me a little stuck. QDR is the sole determiner >> of >> > auth identity. but QDR delegates to a cyrus sasl config right? and cyrus >> > sasl has some local DB options or sql or ldap or it can delegate to >> > kerberos or pam and i am just starting to feel a little lost in all my >> auth >> > option because its been a long time since i have been through all that. i >> > will figure it out well enough. i kind of wish there was a way i could >> send >> > a message in through $management to add a new user/pass to the sasldb but >> > ill figure something out. >> > >> > also, in regards to auth where is it that i specify what users have >> access >> > to what addresses? it looks like that might be in the config in >> > vhost>groups but then i see a policy area of the config. ill start in the >> > vhost>groups area and see how far i get >> > >> > >> > > > > I think depending on what I learn on these topics I will likely >> have >> > > more >> > > > > questions. Thank you to anybody who is able to give me a lead or >> > point >> > > me >> > > > > to a config that may serve as an example. I really do appreciate >> it. >> > > > Please don't hesitate to ask more questions or point out where there >> is >> > > > lack of documentation. We appreciate it as well. >> > > >> > >> > so i had another question come up in my research today. i have a single >> F5 >> > BIG IP VIP that sits in front of all my VMs that are across two different >> > geographic locations. due to the two locations i want, well, two of >> > everything in a way that i can use all the resources at my disposal but >> > still function if one location goes offline. So here are (R)outers and >> > (B)rokers in locations (a) and (b) >> > >> > in order for me to be able to produce messages into Ba and Bb i found >> that >> > each one of my Routers needed a connection to each one of my Brokers. >> > >> > Essentially: >> > Ra --> Ba >> > Ra --> Bb >> > Rb --> Ba >> > Rb --> Bb >> > >> > Graphically: >> > Ra --- Ba >> > \ / >> > / \ >> > Ra --- Bb >> > >> > it was really cool that i could send messages to Ra and see them fill up >> > both Ba and Bb. Receiving across both brokers also worked. But i was >> hoping >> > for more of a configuration where the Routers where only connected to a >> > single Broker and all the Routers knew about each other. >> > >> > Essentially: >> > Ra --> Ba >> > Rb --> Bb >> > Ra <-> Rb >> > >> > Graphically: >> > Ra --- Ba >> > || >> > || >> > Ra --- Bb >> > >> > but in this configuration messages sent to Ra only got routed to Ba and >> > when i made a consumer on Ra i could only get messages off of Ba. Do you >> > know what someone would need to do in the configuration to support this? >> > or is this architecture not ideal? the next thing i was going to try was >> > to make the Cost of Ra --> Ba = 2 so that it was equal to the Cost of Ra >> > --> Rb --> Bb and then maybe they would be considered as equal routes and >> > messages would balance between them. i dont think this explains why i >> > couldnt consume from both. ill work on that tomorrow maybe. >> > >> > any ideas you might have would be cool. thanks again. i really appreciate >> > your insight >> > >> >> Your desired topology is the one that is intended by Dispatch Router (i.e. >> the routers connected together and each connected locally to its own >> brokers). I suspect that the issue you are having is due to a >> misconfiguration of some kind. >> >> Do you have instances of the same queue on both brokers? Or do you have >> some queues on one and others on the other? >> >> If you lose a location, do you care if you lose queued messages? I.e. do >> you want availability of the service only or do you need high availability >> of the messages? >> >> Here's an example: >> >> If you have both brokers configured the same (same queues) and both routers >> configured with auto-links for the queues, and the queue addresses are >> "balanced", then the queues will be "sharded". Each produced message will >> go to exactly one of the queues, and each dequeued message will be >> delivered to exactly one consumer. If you assign the inter-router >> connection a cost that is relatively high (say 100 or 1000), then traffic >> will generally stay local (senders, brokers, and receivers) unless there is >> some failure like a failed broker or no consumers in the local location. >> >> Food for thought... >> >> -Ted >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
