These are two separate features: 1) The linearstore only makes sure that all messages are stored on the disk and will be recovered when you restart the broker. But the linear store it self doesn't offload them from memory. The messages will be still kept also in RAM. "qpidd --help" should give you a list of all configuration options relevant to the store with some description. But often you don't need to change them. The store is used only when a queue is created as durable (for example using the --durable option of the qpid-config tool) and even in such case it will store only messages which were sent as persistent.
2) To offload messages from memory, you can use a feature called paging. That allows you to have more messages in the broker than RAM. It has to be activated when the queue is created. You have to create the queue with argument qpid.paging set to true. The paging feature has two additional config arguments. Argument qpid.page_factor defines how big one page is as a multiply of your platforms page size (getconf PAGESIZE ... usually 4k). The page has to be bigger than your maximum message size - i.e. for 1MB messages you would need qpid.page_factor of at least 256. Second argument is qpid.max_pages_loaded and defines how many such pages are kept in memory - the rest will be stored only on disk. You can use the --argument=<NAME>=<VALUE> option in qpid-config to specify the arguments when creating the queue. Since the paging is a separate feature, you can use this also without the linear store. For example when you want to send large message amounts, but don't need the persistency between restarts. But as a side-effect, when you use both linearstore and paging together, all messages are written to the disk twice. Once into the paging file and once into the store journals. So there is some performance penalty and it consumes double the amount of disk space. J. On Sat, May 14, 2016 at 11:12 PM, Graham Leggett <[email protected]> wrote: > Hi all, > > I am trying to create a queue larger than will fit in available RAM (1TB > in my case). The messages on the queue will be modest in size (1MB each or > so), the key thing is that I need to be able to enqueue large quantities of > data. > > I am using qpid-cpp-server as supplied by EPEL, along with > qpid-cpp-server-linearstore, however I am struggling to find any clear > documentation on how to configure qpid-cpp-server-linearstore. > > The documentation for qpid-cpp-server makes no mention of linearstore at > all: > https://qpid.apache.org/releases/qpid-cpp-0.34/cpp-broker/book/chapter-Managing-CPP-Broker.html#MgmtC-2B-2B-Usingqpidconfig > > It appears that --efp-partition-num and --efp-pool-file-size are somehow > relevant, but googling for these options finds mention of them in patches > and bugreports, but no concrete documentation. > > Can anyone confirm what I need to do to configure queues that persist on > disk? > > Regards, > Graham > — > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
