The answer to your first question depends on what is more important to
you - low latency or high throughput. Messages to be persisted will
accumulate in a buffer page until it is full or until a timer is
triggered, then it will be written to disk. It is not until this happens
that the message will be acknowledged by the broker. If low latency is
important, then having smaller but more numerous buffer pages will mean
the messages will not wait for very long before being written to disk
and acknowledged as received. However this occurs at the cost of some
efficiency, which can affect throughput. If you have large volumes of
messages and the throughput is more important, then using fewer but
larger buffer pages will help you.
Be aware, however, that the product of the size and number of pages is
the total memory that will be consumed and held by the broker for
buffering *per queue*. If you have a very large number of queues, then
you must watch out that you don't over-size your write buffers or else
you will run out of memory.
While I cannot give you specific answers, as these depend on your
performance priorities, I suggest some trial-and-error if you want to
adjust these values.
The Transaction Prepared List (TPL) is a special global queue for
persisting transaction boundaries. As this info is usually small and
relatively infrequent, the tpl-* settings apply to this queue only and
the user has the option to use different values than the regular queues.
If you don't use transactions, then this can be ignored. It is not a
queue that can be written to directly, but the store creates its own
data that is saved in this queue. Adjusting the tpl-* settings depends
only on the frequency of transactions in the user's application or use-case.
Hope that helps,
Kim van der Riet
On 11/27/18 4:44 PM, rammohan ganapavarapu wrote:
Kim,
1. My message size is around 80kb, so what would be suggested values for
the blow properties?
wcache-page-size
wcache-num-pages
tpl-wcache-num-pages
tpl-wcache-page-size
right now i have all defaults, so i am trying to see if i can tune these
values for my messages size to avoid those AIO busy cases. I have try to
define those properties/options in qpidd.conf file but when i run
qpid-config queues its not showing those values on my queues created by
client application, do i have to define those options when i create queue
instead of keep them in qpidd.conf?
2. What is difference b/w tpl-wcache-page-size and wcache-page-size
Thanks,
Ram
On Fri, Nov 16, 2018 at 9:26 AM Kim van der Riet <[email protected]>
wrote:
There is little documentation on linearstore. Certainly, the Apache docs
don't contain much. I think this is an oversight, but it won't get fixed
anytime soon.
Kim
On 11/16/18 12:11 PM, rammohan ganapavarapu wrote:
Any one point me to the doc where i can read internals about how
linearstore works and how qpid uses it?
Thanks,
Ram
On Mon, Nov 12, 2018 at 8:43 AM rammohan ganapavarapu <
[email protected]> wrote:
Kim,
Thanks for clearing that up for me, does it support SAN storage blocks.
Where can i read more about linearstore if i want to know the low level
internals?
Ram
On Mon, Nov 12, 2018 at 8:32 AM Kim van der Riet <[email protected]>
wrote:
The linearstore relies on using libaio for its async disk writes. The
O_DIRECT flag is used, and this requires a block of aligned memory to
serve as a memory buffer for disk write operations. To my knowledge,
this technique only works with local disks and controllers. NFS does
not
allow for DMA memory writes to disk AFAIK, and for as long as I can
remember, has been a problem for the linearstore. With some work it
might be possible to make it work using another write technique though.
NFS has never been a "supported" medium for linearstore.
On 11/9/18 4:28 PM, rammohan ganapavarapu wrote:
But how does NFS will cause this issue, i am interested to see because
we
are using NFS (V4 version) in some environments, so wanted to learn
tunings
when we use NFS.
Thanks,
Ram
On Fri, Nov 9, 2018 at 6:48 AM rammohan ganapavarapu <
[email protected]> wrote:
Sorry, i thought it's NFS but it's actually SAN storage volume.
Thanks,
Ram
On Fri, Nov 9, 2018, 2:10 AM Gordon Sim <[email protected] wrote:
On 08/11/18 16:56, rammohan ganapavarapu wrote:
I was wrong about the NFS for qpid journal files, looks like they
are on
NFS, so does NFS cause this issue?
Yes, I believe it does. What version of NFS are you using?
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
