On Mon, 2011-01-31 at 12:01 -0300, Angel Freire wrote: > Hi, > > I need a guide or help configuring the persistence plugin [0]. > > This is my current qpidd.conf: > > log-to-syslog=yes > log-to-stderr=no > daemon=yes > auth=no > worker-threads=16 > # transport=tcp > pid-dir=/var/run/qpidd > data-dir=/var/lib/qpidd > store-dir=/var/lib/qpidd/store > num-jfiles32 > jfile-size-pgs=96 > wcache-page-size=128 > tpl-num-jfiles=32 > tpl-jfile-size-pgs=96 > tpl-wcache-page-size=128 > > I need to be able to enqueue about 2.500.000 msgs that are normally > 400 bytes long. > > Currently, with those settings, I get a lot of messages like this: > > framing-error: Enqueue capacity threshold exceeded on queue > "charges_async_request". (JournalImpl.cpp:621) > > I'm using the version 0.6 of the broker and the plugin. > > Thanks, > Angel > > [0] http://qpidcomponents.org/download.html#persistence >
Your store size is too small. Using num-jfiles=32 (I hope you have an '=' in there) and jfile-size-pgs=96 yields a total store size of 192 MiB (201,326,592 bytes). The unit for the jfile-size-pgs parameters is read cache pages of 64kiB each, so 96 such units yields 96 * 64 * 1024 = 6291456 bytes (6.000 MiB). 32 such files brings the total to 192MiB. Assuming your messages are 400 bytes each, and knowing that the store saves messages in 128 byte blocks, each message will consume 4 blocks = 512 bytes of disk space. 2.5 million such messages will need 1.19GiB. I recommend allocating twice this (but you can go as low as 1.5 times this amount), so a store of about 2.4 GiB should be enough, provided you are consuming more-or-less in the same order of production, and you are not using transactions or frequent flushing. Keeping 32 files (you can go as high as 64, but watch out for having enough file handles available), this makes each file about 76.8MiB. Since each unit of file size is 64kiB, this can be achieved using jfile-size-pgs=1229. Note that with such large journal sizes, it will take a noticeable amount of time to declare each queue - as 2.4GiB of journal files need to be created and formatted - and also to recover in the case of a broker restart. The following resources may help from the persistence section in the Qpid FAQ[1]: How do I configure the persistence store? What is a RHM_IORES_ENQCAPTHRESH error? [1] https://cwiki.apache.org/qpid/faq.html#FAQ-Persistence --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
