|
FAQ has been edited by Carl Trieloff (Jan 16, 2009). Content:
This page is a collection of FAQ and How to-s for Qpid. If you have a question, post it to the users list and we will place the answer here to build out our FAQ/ How to. FAQAbout AMQPWhat is AMQP?AMQP is a wire-level protocol and model for high performance enterprise messaging. AMQP is an Open Standard for Messaging Middleware. By complying to the AMQP standard, middleware products written for different platforms and in different languages can send messages to one another. AMQP addresses the problem of transporting value-bearing messages across and between organizations in a timely manner. AMQP enables complete interoperability for messaging middleware; both the networking protocol and the semantics of broker services are defined in AMQP. Where did AMQP come fromAMQP was born out from Frustration by John O'Hara at JPMC. He started a project internally to create commodity messaging that was easy to use. Carl Trieloff from Red Hat had started a project to build messaging for both users and for use in infrastructure, while looking around spoke to John about his work. Out of these discussion was born the AMQP working Group with 6 initial members, under an agreement that it will be eternally be licensed for everyone to use. Since then the Working Group has had many join, and has been making solid progress working on revisions of the specification. For more details see. Why use AMQP?AMQP is has been designed to be able to handle the hardest workloads, scale to the largest systems, but also deal with reduction of change and maintenance costs by doing a refresh on many aged practices. The specification is also not language specific allowing the freedom from language and platform lock in, without compromise on user experience, security, scalability and consistently excellent performance. Qpid & AMQPIs Qpid AMQP Compliant?Yes, Apache Qpid implements the latest AMQP specifications, providing transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support and a lot more. And Apache Qpid is extremely fast. Apache Qpid aims to be 100% AMQP Compliant. What Client support does Qpid have?Apache Qpid provides AMQP Client APIs for the following languages:
If you need another client, join the lists and ask or feel free to contribute one. What messaging topologies are supported by AMQP and Qpid?AMQP provides the ability to do Point-to-Point, Peer-to-Peer, Pub-Sub, and Eventing. This allows many patterns to be craeted: Point-to-point This is one of the simplest use-cases. AMQP allows for this in a few ways. One-to-many There are a few patterns that can be used. a.) AMQP provides a 'fanout' exchange which will send a message to all the queues that have been bound to it. Different domains or topics are created with the 'fanout' exchange by declaring different named fan-out exchanges. b.) A 'topic' or 'headers' exchange can also be used. in this case the pattern match is used to send the message to all the bound queues. It can be thought of as a filter allowing you to create just about any One-to-many routing patterns. Pub-Sub Topic can be created with the 'topic' or other 'direct' exchange to allow consumer to bind to into the steams of data they care about. This pattern combined with the use of reply-to and Alternate-routing is the staple of what most people use messaging for today. FAST Reliable Messaging AMQP 0-10 allows for fully reliable transfers between any two peers. This means that you can publish or subscribe to the broker fully reliable without requiring the need for transactions. This can all be done in async mode with the C++ broker allowing for high throughput while running entirely reliable. Transactional AMQP supports two types of transactions in AMQP 0-10, TX and DTX. This allows for local (1PC), and 2PC transaction and the ability to coordinate with a TM (Transaction Manager). The Java broker supports TX, the C++ broker support TX, DTX, XA, JTA for fully ACID transactions. This allows you to commit a single unit of work with may contain enqueues & dequeues either locally on the broker, or in coordination with other transactional resource like RDBMS. Transient message delivery By default messages are transient. Transient message can be sent to queues that are durable. They will not be safe stored or recovered, and will perform as any other transient message - fast! Durable message delivery There is a header on each message where the message properties are specified, one of these is durability. Messages that are marked as durable and published to a durable queue will be safe stored. Durable messages will survive restart of the broker or cluster. Federation (Hub-spoke, Trees, graphs) As AMQP 0-10 is symmetric for peer-to-peer communication all the building block are in place for creating networks of brokers. The C++ broker allows you to link the brokers together using 'qpid-route' and then create routes between the brokers either statically or with dynamic routes. This allows for a message to be published to one broker and consumed from another broker in the federated broker network. This feature is great to create data-center, or project isolation, but allow cross communication. It also allows networks to be created to scaled. For more details see And many others, including custom pattern Message Reply, Rings, Initial Value Caches, Last Value Messaging All the above cases can be constructed using the AMQP and features of Qpid. For example reply can be constructed using message browsing and setting TTL on the messages. The C++ broker also support ring queues, last value queues, initial value caches on exchanges. With a bit of throught many additional patterns can be constructed. Store-and-forward Store-and-forward can be achieved by publishing to well know durable queues, that are not marked with auto delete. Consumers will be able to 'came back' to consume then at any time, even after restarts. What AMQP and other exchanges does Qpid support?Both brokers support:
In additional the C++ broker support
Custom exchanges allow you to provide your own custom routing logic and algorithms via a plug-in. If you build an interesting exchange, please feel free to contribute it back to the Qpid project. SecurityWhat encryption does Qpid support?
What authentication does Qpid support?SASL Authentication is supported. All Clients support PLAIN, and Kerberos support if being added to all the clients. The C++ broker support Kerberbos authentication. What authorization does Qpid support?Full ACL is supported in the brokers. For details on configuring ACL see. ACL supports realms and allows for granular permission to be set on all the broker actions including management on an user or group basis. PerformanceDoes Qpid Perform (Latency/Throughput)?Yes, The Qpid C++ broker has been achieved great benchmark results in published papers by those that redistribute it. Red Hat MRG Latencies have been recored as low as 180-250us (.18ms-.3ms) for TCP round trip and 60-80us for RDMA round trip using the C++ broker. How do I measure throughput?There is a great resource supplied in the C++ broker test directory called perftest. If allows you to create load on a broker for all the exchanges, multiple queues, multiple connection, coordinate multiple publishing and consuming processes, beachmark transactions and much much more such as acquire mode, txn size, message size. For all the options ./perftest --help How do I measure latency?
Don't forget to set tune the machine and set --tcp-nodelay on both the C++ broker & client. For all the options ./latencytest --help How do I measure performance with Java clients?In Java we provide a utility called QpidBench. It allows you to test the performance of the native AMQP API in Java for 0-10 and the JMS API against both brokers. Can I run my Java client with JAVA-RT?Yes, recently a thread abstraction layer has been added to the Java client allowing it to be used with both the SUN and IBM RT JVMs. This increases the determinism of latency when using the Java client. ManagementWhat Management does Qpid supportThe Java broker supports JMX and provides an Eclipse plug-in and command line tool to manage via JMX. The C++ broker has far more extensive management support via QMF which will be added to the Java broker in a future release. The C++ Broker supports a layered management protocol over AMQP called QMF. This allows for the management of resource either in the broker or connected to the broker via the AMQP fabric. This management includes statistics, control, eventing, and reporting/updating properties. How do I manage a broker?A set of tools are provided to manage the C++ broker, they include
Then you can also access all thsi information via JMX or WS-DM (work in progress) using QMan. What logging tracing and events does Qpid support?Qpid support the ability to output events from any the broker or any managed object via QMF, or to do a variety of logging from the broker & clients. for tracing options run qpidd --help. Multiple levels of of logging are supported in the C++ broker from debug, warning, error, info, etc – all of which can be filtered. Can I get to all the management data from a client?yes, All the management data is just AMQP messages on specially named queues. An API is provided for working with the management data called QMFC What is QMFQMF is the layered Management protocol used to manage the C++ broker. For details on teh protocol see the Development pages. QMF allows you to manage any resource and provides the following infrsstructure:
What are QMF Agents, and what do they do for me?An Agent is any client (producer or consumer) that generates a QMF schema and registers itself to be management by QMF. A great use case of this is a consumer that is processing order from a queue can reference itself to that queue and for example provide a schema for the number or successful orders process and a method to suspend processing. Now it becomes possible to use qpid-tool to connect to the broker, see which order processors are on queue via the reference and the via the stats of the order processor client. It is also possible to issue a command to the client via qpid-tool to suspend processing. ACL in the broker can be applied to all these actions if desired. What is QMFC and what does it do for mr?QMFC is the API used to consume QMF data, event and issue commands to QMF agents from an AMQP client. What is QManQman is a tool that dynamically reads the QMF Schema information and creates JMX objects that consumed by any JMX console or application server to manage Qpid. QMan is also adding support for WS-DM management of QMF resources. Clustering, Federation and Disaster RecoveryDoes Qpid provide Fault Tolerance for the cluster?The C++ broker has plug-ins for Active-Active clustering which keep all the nodes of the cluster in sync. This means that any action that is performed on one of the brokers on the cluster is performed on all of them at the same time. New nodes can be added to the cluster at any time, and removed at any time with no consequences, exept for the extra multi-cast load created for the sync on joining. What does the cluster guarantee?Everything! All configuration, all messages and all actions are replicated in a cluster. This means that two consumers can be connected to different nodes in the cluster and they will behave EXACTLY the same as if they where on a single broker. Do clients get notified members joining or leaving the cluster?yes, All clients are updated with the addresses of node add/removed as supported by the AMQP 0-10 specification. This means that the client can dynamically track the nodes in the cluster and reconnect as required. Can I specify more than one host to connect initially to the cluster to avoid single point of failure?yes, the AMQP address is multi-honed and more than one IP address can be specified at the initial connection. The client will then iterate through the host until it makes a successful connection. This feature can also be used in none clustered brokers. How does Clustering work?When C++ brokers are configured into a cluster, the nodes communicate with each other over a mulitcast protocol called AIS, an open Telco multicast protocol that provides all the quorum and group services. Every action that is performed on any node of the cluster is then sequenced via totem and then performed on each node of the cluster in sync. As the cluster backbone is multicast, a separate network can be used for cluster communication and there is little impact adding additional nodes to the cluster with-in reason. What is Federation?Federation provides the ability to create networks of brokers that communicate with each other in all types of typologies. This allows a producer to publish messages to one broker and someone to consume the messages from another broke somewhere on the broker federated network. Disater recover features are in process, Q&A will be added once they are complete.How ToHow to use RDMA with QpidThe RDMA plugin uses native OFED1.3 and puts AMQP directly onto the DMA. When using the RDMA plug-in for Qpid note the following
Once you have it up and running, use latencytest to make sure it is working. You should see latencies between 50 and 80us round trip. Message TTL, auto expireI need to be able to set time for a message that I send to be removed from the queue if it is not read by my subscriber. For example: I enqueue a message and I want it to be automatically dequeued after a certain amount of time has passed.Is there a feature like this in qpid? yes, the TTL can be set in the message headers and the messages get dequeued if TTL expires E.g. from c++: Message m("Hello World!");
m.getDeliveryProperties().setTtl(500);
Sets a 500 millisecond timeout. How to install the qpid-tools for c++ booker?I see [commands]$ ./qpid-queue-stats Traceback (most recent call last): File "./qpid-queue-stats", line 29, in from qmf.console import Session, Console ImportError: No module named qmf.console This problem occurs because the PYTHONPATH environment variable does not include the location of the qpid python files. If you are running from the SVN checkout, add <path>/qpid/python to PYTHONPATH (where <path> is the location of your SVN tree). If you are installing from source, make sure you configure with the same prefix where Python is installed. This is most likely: # configure --prefix=/usr # make # make install If you are running from vendor RPMs, this should work automatically. |
Unsubscribe or edit your notifications preferences
