On Fri, Dec 20 2013, Herndon, John Luke wrote: > Yeah, I like this idea. As far as I can tell, AMQP doesn’t support grabbing > more than a single message at a time, but we could definitely have the > broker store up the batch before sending it along. Other protocols may > support bulk consumption. My one concern with this approach is error > handling. Currently the executors treat each notification individually. So > let’s say the broker hands 100 messages at a time. When client is done > processing the messages, the broker needs to know if message 25 had an error > or not. We would somehow need to communicate back to the broker which > messages failed. I think this may take some refactoring of > executors/dispatchers. What do you think?
Yeah, it definitely needs to change the messaging API a bit to handle such a case. But in the end that will be a good thing to support such a case, it being natively supported by the broker or not. For brokers where it's not possible, it may be simple enough to have a "get_one_notification_nb()" method that would either return a notification or None if there's none to read, and would that consequently have to be _non-blocking_. So if the transport is smart we write: # Return up to max_number_of_notifications_to_read notifications = transport.get_notificatations(conf.max_number_of_notifications_to_read) storage.record(notifications) Otherwise we do: for i in range(conf.max_number_of_notifications_to_read): notification = transport.get_one_notification_nb(): if notification: notifications.append(notification) else: break storage.record(notifications) So it's just about having the right primitive in oslo.messaging, we can then build on top of that wherever that is. -- Julien Danjou /* Free Software hacker * independent consultant http://julien.danjou.info */
signature.asc
Description: PGP signature
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev