Bringing my conversation with Doug back on-list... > In nova.rpc with fanout=True every consumer gets a copy of the event because > every consumer has its own queue. With fanout=False only one consumer *at > all* will get a copy of the event, since they all listen on the same queue. > The changes I made come somewhere in between that. It allows multiple > consumers to receive a given message, but it also allows several consumers to > declare that they are collaborating so that only one of the subset receives a > copy. That means that multiple types of consumers can be listening to > notifications (metering and audit logging, for example) and each type of > consumer can have a load balanced pool of workers so that messages are only > processed once for metering and once for logging. >
We can do this today with the Matchmaker. You can use a standard fanout, but make one of the hosts a DNS entry with multiple A or CNAME records for round-robin DNS, where that "host" will act as a pool of workers. It would be trivial to update the matchmaker to support nested lists to support this with IP addresses as well, doing round-robin or random-selection of hosts without a pool of workers. Unfortunately, doing this in the AMQP fashion of registering workers is difficult to do via the matchmaker. Not impossible, but it requires that the matchmakers have a (de)centralized datastore. This could be solved by having get_workers and/or create_consumer communicate to the matchmaker and update mysql, zookeeper, redis, etc. While I think this is a viable approach, I've avoided /requiring/ this paradigm as the alternatives of using hash maps and/or DNS are significantly less complex and easier to scale and keep available. We should consider to what degree dynamic vs static configuration is necessary, if dynamic is truly required, and how a method like get_workers should behave on a statically configured system. Regards, Eric Windisch
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
