Hi Gordon, Thanks for looking at my questions.
ad 7) I was trying it even with configuration max_pages_loaded=100 and page_factor=100. The test I'm doing is following: - Create a queue with "--limit-policy=reject --max-queue-size=1024000000 --max-queue-count=0 --argument qpid.paging=true --argument qpid.max_pages_loaded=100 --argument qpid.page_factor=100 --file-count=8 --file-size=4096 --durable" parameters - Send 100.000 messages (~1kB each) into the queue - Consume the messages from the queue with a receiver capacity 1000, acknowledge sent after 1000 messages (I tried acknowledging after 10 messages as well - no impact). The consumer is quite fast, but after ~38.000 messages consumed, I always get the "Dequeuing message with null persistence Id." error which damages the queue until broker restart (strangely sometimes even beyond broker restart). Have you seen the "Dequeuing message with null persistence Id." error as well? ad 8) I agree that for non-persistent queue the only problem seems to be when max_pages_loaded=1. I'm not sure there is really a usecase for using max_pages_loaded=1 ... I was just playing with it when I discovered it. ad 9) Indeed. It seems that the selector didn't worked for me mainly because of the problem with max_pages_loaded=1. ad 10) Yes, the full path in the broker log makes sense. The error message received by the clients was what I didn't like. Thanks & Regards Jakub On Thu, Nov 7, 2013 at 11:38 AM, Gordon Sim <[email protected]> wrote: > On 11/06/2013 11:09 PM, Jakub Scholz wrote: > >> I finally managed to spent some quality time with the queue paging today. >> I >> have quite a lot of questions / comments / feedback. I hope you don't >> mind. >> > > We certainly don't mind, indeed we are very grateful! > > > If you want, I can as usually rise the JIRAs or provide more details for >> reproducing of the problems. Just let me know what is a real problem and a >> JIRA should be raised and what "works as designed" and no JIRA is needed. >> > > As always, thanks very much for taking the time to ry it out and give > your feedback Jakub! > > > >> 1) Why is the page file created as executable? >> -rwx------. 1 qpidd qpidd 110592000 Nov 6 19:07 my-test-queue-2 >> > > That's just a mistake. I'll fix it. > > > --- >> >> 2) As I understood from the previous discussion, the queue can store only >> messages which are not bigger than the page size (i.e. page_factor * >> memory >> page). Does the value of the page_factor have also some other >> implications? >> Does it have some effect on performance? >> > > It affects the number of messages held in memory. It probably has some > impact on performance, but I haven't fully explored what that might be. > > > --- >> >> 3) Can I somewhere see some statistics about how many pages are currently >> in memory or how many are off-loaded to the disk? The qpid-tool doesn't >> seem to have this information. >> > > Unfortunately not at this point. I'll have a think about ways to do this. > > > --- >> >> 4) It would be really great to have a possibility to change the >> qpid.max_pages_loaded parameter without deleting and recreating the queue. >> Even having this as an "offline" option - i.e. when the change would be >> possible only when the broker is switched ogg - would be IMHO useful. >> > > Let me have a think about what the options might be to address this. > > > --- >> >> 5) It would be great to have a broker configuration option "--paging-dir" >> to be able to specify the directory for the page files should be created. >> Right now it seems to use the "--data-dir" parameter. >> > > That should be easy enough to add. > > > --- >> >> 6) Even when using the paging, the broker memory consumption seems to >> increase all the time. Is it just some sort of "index" linking to the >> offloaded pages? If it is only the index what is kept in memory, does that >> mean that with higher qpid.page_factor I will have lower memory >> consumption? >> > > Yes to both. I found the growth to be very slow and so acceptable to begin > with at least. I suspect there are ways to avoid it or limit it (though at > a cost of some more complexity) if it is a problem in anticipated use cases. > > > --- >> >> 7) The purge QMF method doesn't seem to work. When I call it for the first >> time, it seem to always purge only the messages from the loaded pages. >> When >> I try to call it again I get the following error: >> Connection closed by peer with amqp:internal-error: Queue >> "my-test-queue-2": Dequeuing message with null persistence Id. >> (/home/jakub/qpid/qpid-trunk/cpp/src/qpid/legacystore/ >> MessageStoreImpl.cpp:1372) >> >> (the queue configuration: durable=true, max_pages_loaded=1, >> page_factor=1). >> In fact with the queue configured as above, this happens to receiver as >> well - when reading the queue for the first time it gets only the messages >> from the loaded page. When read for the second time, it creates the same >> error. >> >> I also tried to change the values of max_pages_loaded and page_factor ... >> but that doesn't seem to change much. I can read/purge more messages >> before >> seeing the error. But it still seems to occur regularly. >> > > Is it only with max_pages_loaded=1 that you see problems? > > I can reproduce the issues as described with that setting (I think the > logic can't cope with a single page in memory somehow, though I've yet to > spot why), but with max_pages_loaded=2 I seem to be able to receive and > purge with both durable and non-durable. > > > --- >> >> 8) When the queue is created as above (max_pages_loaded=1, page_factor=1), >> but as non-durable, purging and reading still doesn't work properly. It >> always reads / purges only the loaded pages. But of course I do not get >> the >> error from the persistent message store. However, with increase >> max_pages_loaded and/or page_factor (e.g. 10 and 10), the problem seems to >> be resolved (for non-persistent queues). >> >> It seems to me that when the pages are too small / only few pages are >> loaded, the broker doesn't manage to load the pages quickly enough from >> disk and that some kind of "starves" the receiver. Perhaps setting minimal >> values for the page_factor / max_pages_loaded can help here? >> > > It looks to me like it might simply be the max_pages_loaded=1 case, but I > could simply be missing the correct reproducer for wider issues. > > > --- >> >> 9) I know that the paged queues don't support LVQ functionality and >> probably also message priorities. But it seems that paged queues don't >> really support filters / selectors as well, or? The client using selector >> always receives only the messages which match the selector and are in the >> loaded pages. I guess it makes sense - since the broker doesn't keep >> complete message headers, it cannot easily match >> the selectors. But in that case, I would suggest to raise an error if a >> consumer with filter/selector is created. >> > > There is some bug here I think, that behaviour is certainly not > intentional. > > (I tried a simple test case, with max_pages_loaded=2 and batches of > messages with a different 'colour' property, and selecting by colour. I saw > slightly different results from you. At first all seemed to be ok, > selectors were pulling out the correct ranges even if separated by large > gaps in sequence. However I did then notice that some messages got 'stuck' > somehow. I'll dig more into this.) > > > --- >> >> 10) When I try to send a message bigger than the page size, I get >> following >> error: >> Link detached by peer with amqp:precondition-failed: precondition-failed: >> Message is larger than page size for queue backed by >> /home/qpidd/qpid/0000/data/broker/my-test-queue-2 >> (/home/jakub/qpid/qpid-trunk/cpp/src/qpid/broker/PagedQueue.cpp:107) >> >> Is it intentional to include the full path to the page file? I would >> definitely prefer to show just the file name or a queue name. >> > > I'll change that. I'll log the full path but only include the queue name > in the error message. Does that sound reasonable? > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
