RE: First try using the python client
Okey, thanks again. I'm just glad that my result can be explained! We will the use the RH Messaging (or maybe all of MRG) later on so having access the storage plugin will not be a problem. Thanks, Patrik > -Original Message- > From: Gordon Sim [mailto:[EMAIL PROTECTED] > Sent: den 28 november 2008 10:18 > To: [email protected] > Subject: Re: First try using the python client > > Patrik Berglund wrote: > > I've set the reject option on the queue that we use and as a test > > the 'qpid.max_count' to 10. The sender has set the immediate flag > > and discard unroutable to false on the message. > > The result is that if the sender transfer 20 message as > fast as it can, > > the receiver gets 10 message with the correct content and 10 message > > with a body size of zero. > > > > Anyone got a suggestion what I do wrong? > > On M3 the policy type has no effect and if the queue is > declared durable > then messages that exceed the limit are freed from memory and > written to > disk. > > Unfortunately if you do not have a persistence module plugged in, the > data is then lost. This is fixed on trunk for M4. > > Is your queue declared durable? If so either don't declare it as > durable, or ensure you have a persistence module loaded[1]. > Or use the > M4 beta[2]. > > [1] e.g. for the M3 broker, > https://svn.jboss.org/repos/rhmessaging/store/branches/mrg-1.0/cpp > > [2] http://people.apache.org/~rhs/qpid-incubating-M4-beta/ >
Re: First try using the python client
Patrik Berglund wrote: I've set the reject option on the queue that we use and as a test the 'qpid.max_count' to 10. The sender has set the immediate flag and discard unroutable to false on the message. The result is that if the sender transfer 20 message as fast as it can, the receiver gets 10 message with the correct content and 10 message with a body size of zero. Anyone got a suggestion what I do wrong? On M3 the policy type has no effect and if the queue is declared durable then messages that exceed the limit are freed from memory and written to disk. Unfortunately if you do not have a persistence module plugged in, the data is then lost. This is fixed on trunk for M4. Is your queue declared durable? If so either don't declare it as durable, or ensure you have a persistence module loaded[1]. Or use the M4 beta[2]. [1] e.g. for the M3 broker, https://svn.jboss.org/repos/rhmessaging/store/branches/mrg-1.0/cpp [2] http://people.apache.org/~rhs/qpid-incubating-M4-beta/
RE: First try using the python client
Carl Trieloff wrote: > M3, has few few things you can do, M4 has a few more... > > http://cwiki.apache.org/qpid/cheat-sheet-for-configuring-queue > -options.html > > The reject and flow-to-disk options work in M3. The two ring options > have been added for M4. > > Basically you set the size policy and then a message and or > byte limit > when declaring the queue. > > > set "qpid.max_count" to value > set "qpid.max_size" to value > set "qpid.policy_type" to "reject" || "flow_to_disk" || "ring" || > "ring_strict" I've set the reject option on the queue that we use and as a test the 'qpid.max_count' to 10. The sender has set the immediate flag and discard unroutable to false on the message. The result is that if the sender transfer 20 message as fast as it can, the receiver gets 10 message with the correct content and 10 message with a body size of zero. Anyone got a suggestion what I do wrong? (It is the python client, c++ broker from M3 that we use.) Thanks, Patrik
RE: First try using the python client
> Gordon Sim wrote: > Patrik Berglund wrote: > > Q2. What happen to messages sent to an exchange, that for > some reason > > has not (yet) been declared and bound as above. In our > example the > > messages just disappear. > > If the exchange has not been created, the transfer to that > exchange will > fail with a 404, not found exception. Badly formulated of me, I really ment to refer to a missing queue declare, not the exchange at all. (We only use the amq.direct for now.) > The spec says that if the accept-mode is explicit (and > discard-unroutable is not set), the broker should issue a reject. > Unfortunately that feature is not yet implemented. > > I've created a Jira for that [1]. I'm afraid its probably a > bit late to > get that in for M4, but I'll work on getting that in as soon as the > feature freeze is lifted. Okey, I understand. We will try to work around it as you suggest! Thanks, /Patrik
Re: First try using the python client
Patrik Berglund wrote: Q2. What happen to messages sent to an exchange, that for some reason has not (yet) been declared and bound as above. In our example the messages just disappear. If the exchange has not been created, the transfer to that exchange will fail with a 404, not found exception. However if the exchange exists (as will always be the case for amq.direct) but there is no matching queue bound, the broker will currently try to route the message to any defined 'alternate-exchange' (unfortunately none is defined for amq.direct). The spec says that if the accept-mode is explicit (and discard-unroutable is not set), the broker should issue a reject. Unfortunately that feature is not yet implemented. I've created a Jira for that [1]. I'm afraid its probably a bit late to get that in for M4, but I'll work on getting that in as soon as the feature freeze is lifted. The workaround in the meantime would be to use your own exchange instance (not amq.direct) with an alternate-exchange defined, and have the publisher bind a queue to that alternate exchange and subscribe to it so that it at least gets notified of messages that don't get routed. (There is an example for this in the 0-10 tests if its of interest). [1] https://issues.apache.org/jira/browse/QPID-1490
Re: First try using the python client
Patrik Berglund wrote: Hello, I've just started to play with Qpid (M3) and the python client. The company I work for is interested in using Qpid or Red Hat MRG Messaging (instead of IBM MQ). We have read the AMQP 0-10 spec and the examples and unit tests that are part of M3, but when writing or own tests we haven't been able to figure out how to do a couple of things. The test scenario is this, we just want to send a lot of messages directly from application A to application B and never want to experience any loss of messages. Our queue is currently declared like this (python): session.queue_declare(queue="testQueue1", durable=True, auto_delete=False, exclusive=True) session.exchange_bind(exchange="amq.direct", queue="testQueue1", binding_key="testQueue1") Messages are transferred from application A like this: dprops = session.delivery_properties(routing_key="testQueue1") msg = qpid.datatypes.Message(dprops, "X" * 2**16) session.message_transfer(destination='amq.direct', message=msg) Both of our test applications are written in python and are running on the same (multi-core) machine as the broker. Now to my two newbie questions. Q1. If messages are produced faster than the consumer can handle them, how do you configure flow control? Either some form of credit or something like a "rejected message" exception thrown when the producer transfer a message that overflows the queue, would be interesting for us. M3, has few few things you can do, M4 has a few more... http://cwiki.apache.org/qpid/cheat-sheet-for-configuring-queue-options.html The reject and flow-to-disk options work in M3. The two ring options have been added for M4. Basically you set the size policy and then a message and or byte limit when declaring the queue. set "qpid.max_count" to value set "qpid.max_size" to value set "qpid.policy_type" to "reject" || "flow_to_disk" || "ring" || "ring_strict" This can be done by setting the queue properties, of via qpid-config qpid-config --help Usage: qpid-config [OPTIONS] qpid-config [OPTIONS] exchanges [filter-string] qpid-config [OPTIONS] queues[filter-string] qpid-config [OPTIONS] add exchange [AddExchangeOptions] qpid-config [OPTIONS] del exchange qpid-config [OPTIONS] add queue [AddQueueOptions] qpid-config [OPTIONS] del queue qpid-config [OPTIONS] bind [binding-key] qpid-config [OPTIONS] unbind [binding-key] Options: -b [ --bindings ] Show bindings in queue or exchange list -a [ --broker-addr ] Address (localhost) Address of qpidd broker broker-addr is in the form: [username/[EMAIL PROTECTED] hostname | ip-address [:] ex: localhost, 10.1.1.7:1, broker-host:1, guest/[EMAIL PROTECTED] Add Queue Options: --durable Queue is durable --file-count N (8) Number of files in queue's persistence journal --file-size N (24) File size in pages (64Kib/page) --max-queue-size N Maximum in-memory queue size as bytes --max-queue-count N Maximum in-memory queue size as a number of messages --policy-type TYPE Action taken when queue limit is reached (reject, flow_to_disk, ring, ring_strict) Add Exchange Options: --durable Exchange is durable Q2. What happen to messages sent to an exchange, that for some reason has not (yet) been declared and bound as above. In our example the messages just disappear. We would like to be able to setup (delivery-properties?) so that the message_transfer will throw an appropriate exception, and then the producer simply need to retry until the messages are not rejected. What is the correct way to achieve this? The above can be done, but I think a safer pattern is to text the existence of the exchange before publishing. Then to make sure via message properties a binding exists when the message is transfered. This can be achieved I believe using immediate flag on the deliever-properties of message transfer so in your code above - set arguments on queue declare with your size policy - set immediate flag in the delivery properties on message transfer - optionally test the exchange has been declared. regards, Carl.
