This looks like a bug to me, and that is why I am keen to see a
reproducer if you can find one. How many queues are there? How many
producers and consumers are there for each queue? How are the consumers
working? Are they configured as listeners, or do they poll for new
messages? How frequently? How long does it take under these conditions
for the error to occur typically? If I can get some kind of idea what
the runtime conditions are, it will give me some idea where to look.
If you set the broker to use INFO+ logging (log-enable=info+), then you
should see some detail about the starting and recovery of the store when
the broker starts, which should include this info. The store settings in
the config file are global, so when you set a particular buffer
configuration, all queues will use this. It should be reported during
startup when using INFO+ level logging. Watch your log size, however, as
using this level will make the logs big.
On 12/5/18 5:06 PM, rammohan ganapavarapu wrote:
Kim,
We have set wcache-page-size=128 in qpidd.conf, restarted broker and let
client recreated the queues fresh, we still getting this error, how do we
find if queues created by client actually have this wcache-page-size=128?
2018-12-05 21:18:16 [Protocol] error Connection
qpid.<server>:5672-<client>:17769 closed by error: Queue <queue-name>:
MessageStoreImpl::store() failed: jexception 0x0803 wmgr::enqueue() threw
JERR_WMGR_ENQDISCONT: Enqueued new dtok when previous enqueue returned
partly completed (state ENQ_PART). (This data_tok: id=456535 state=NONE)
(/home/rganapavarapu/rpmbuild/BUILD/qpid-cpp-1.35.0/src/qpid/linearstore/MessageStoreImpl.cpp:1211)(501)
Thanks,
Ram
On Tue, Dec 4, 2018 at 8:18 AM rammohan ganapavarapu <
[email protected]> wrote:
Kim,
Thank you, i will play with that setting, please let me know if any other
tunings will help.
Ram
On Wed, Nov 28, 2018 at 8:04 AM Kim van der Riet <[email protected]>
wrote:
The answer to your first question depends on what is more important to
you - low latency or high throughput. Messages to be persisted will
accumulate in a buffer page until it is full or until a timer is
triggered, then it will be written to disk. It is not until this happens
that the message will be acknowledged by the broker. If low latency is
important, then having smaller but more numerous buffer pages will mean
the messages will not wait for very long before being written to disk
and acknowledged as received. However this occurs at the cost of some
efficiency, which can affect throughput. If you have large volumes of
messages and the throughput is more important, then using fewer but
larger buffer pages will help you.
Be aware, however, that the product of the size and number of pages is
the total memory that will be consumed and held by the broker for
buffering *per queue*. If you have a very large number of queues, then
you must watch out that you don't over-size your write buffers or else
you will run out of memory.
While I cannot give you specific answers, as these depend on your
performance priorities, I suggest some trial-and-error if you want to
adjust these values.
The Transaction Prepared List (TPL) is a special global queue for
persisting transaction boundaries. As this info is usually small and
relatively infrequent, the tpl-* settings apply to this queue only and
the user has the option to use different values than the regular queues.
If you don't use transactions, then this can be ignored. It is not a
queue that can be written to directly, but the store creates its own
data that is saved in this queue. Adjusting the tpl-* settings depends
only on the frequency of transactions in the user's application or
use-case.
Hope that helps,
Kim van der Riet
On 11/27/18 4:44 PM, rammohan ganapavarapu wrote:
Kim,
1. My message size is around 80kb, so what would be suggested values for
the blow properties?
wcache-page-size
wcache-num-pages
tpl-wcache-num-pages
tpl-wcache-page-size
right now i have all defaults, so i am trying to see if i can tune these
values for my messages size to avoid those AIO busy cases. I have try
to
define those properties/options in qpidd.conf file but when i run
qpid-config queues its not showing those values on my queues created by
client application, do i have to define those options when i create
queue
instead of keep them in qpidd.conf?
2. What is difference b/w tpl-wcache-page-size and wcache-page-size
Thanks,
Ram
On Fri, Nov 16, 2018 at 9:26 AM Kim van der Riet <[email protected]>
wrote:
There is little documentation on linearstore. Certainly, the Apache
docs
don't contain much. I think this is an oversight, but it won't get
fixed
anytime soon.
Kim
On 11/16/18 12:11 PM, rammohan ganapavarapu wrote:
Any one point me to the doc where i can read internals about how
linearstore works and how qpid uses it?
Thanks,
Ram
On Mon, Nov 12, 2018 at 8:43 AM rammohan ganapavarapu <
[email protected]> wrote:
Kim,
Thanks for clearing that up for me, does it support SAN storage
blocks.
Where can i read more about linearstore if i want to know the low
level
internals?
Ram
On Mon, Nov 12, 2018 at 8:32 AM Kim van der Riet <
[email protected]>
wrote:
The linearstore relies on using libaio for its async disk writes.
The
O_DIRECT flag is used, and this requires a block of aligned memory
to
serve as a memory buffer for disk write operations. To my knowledge,
this technique only works with local disks and controllers. NFS does
not
allow for DMA memory writes to disk AFAIK, and for as long as I can
remember, has been a problem for the linearstore. With some work it
might be possible to make it work using another write technique
though.
NFS has never been a "supported" medium for linearstore.
On 11/9/18 4:28 PM, rammohan ganapavarapu wrote:
But how does NFS will cause this issue, i am interested to see
because
we
are using NFS (V4 version) in some environments, so wanted to learn
tunings
when we use NFS.
Thanks,
Ram
On Fri, Nov 9, 2018 at 6:48 AM rammohan ganapavarapu <
[email protected]> wrote:
Sorry, i thought it's NFS but it's actually SAN storage volume.
Thanks,
Ram
On Fri, Nov 9, 2018, 2:10 AM Gordon Sim <[email protected] wrote:
On 08/11/18 16:56, rammohan ganapavarapu wrote:
I was wrong about the NFS for qpid journal files, looks like
they
are on
NFS, so does NFS cause this issue?
Yes, I believe it does. What version of NFS are you using?
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
