On 03/01/2010 11:45 PM, Rafael Schloming wrote:
Gordon Sim wrote:
On 02/26/2010 03:09 PM, Alan Conway wrote:
On 02/26/2010 09:47 AM, Gordon Sim wrote:
On 02/26/2010 02:29 PM, Alan Conway wrote:
Gordon/Rafi: this raises an interesting question for the new APIs. It
seems like async declare/bind/subscribe are important features for
cases
like this.
I agree, this is an interesting case. On the face of it my initial
suggestion would be a single receiver for an address using create:
always and a list of binding 20,000 long.
You mean construct a single address string with 20,000 bindings in it?
Wouldn't necessarily need to be done as a string. The
qpid::messaging::Address class can be manipulated directly. However...
That doesn't sound practical. I think we need a way to split apart the
process of constructing a receiver in this case so we can create a
receiver and then incrementally add bindings to it. That would seem like
a useful feature in any case - the set of things a receiver is
interested in may change dynamically over its lifetime so it would be
good to be able to add/remove bindings to a receiver.
...I can see that being a valuable addition for some cases, yes.
It turns out that the python client (sort of by accident) is actually
capable of doing an async query/declare/bind/subscribe.
It just so happens that if you create receivers on a disconnected
connection they get created locally, but not remotely until the
connection is connected again, and when this happens they're all created
in a big batch.
Out of curiosity I tried creating 20,000 separate receivers in this
manner with the python client and it took about 75 seconds:
c = Connection("localhost", 5672)
s = c.session()
rcvs = []
for i in range(20000):
rcvs.append(s.receiver("amq.topic/%s" % i))
print "connecting"
c.connect()
print "connected"
Unfortunately if you connect() before creating the receivers then the
receiver() call will block until the resulting receiver is fully
subscribed before returning, thereby rendering the whole process
synchronous again. This version of the code was still running after 30
minutes, although part of that is due to a linear search through the
receivers list:
c = Connection("localhost", 5672)
s = c.session()
print "connecting"
c.connect()
rcvs = []
for i in range(20000):
rcvs.append(s.receiver("amq.topic/%s" % i))
print "connected"
It would, however, be straightforward to add an option to the receiver
call to control explicitly whether or not it blocks, thereby permitting
asynchronous receiver creation on a connected connection.
Sounds like that would be a worthwhile extension before we release the API, with
a similar option on the C++ side. What about declaring queues/exchanges and
creating bindings?
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
