Kim, Thank you, i will play with that setting, please let me know if any other tunings will help.
Ram On Wed, Nov 28, 2018 at 8:04 AM Kim van der Riet <[email protected]> wrote: > The answer to your first question depends on what is more important to > you - low latency or high throughput. Messages to be persisted will > accumulate in a buffer page until it is full or until a timer is > triggered, then it will be written to disk. It is not until this happens > that the message will be acknowledged by the broker. If low latency is > important, then having smaller but more numerous buffer pages will mean > the messages will not wait for very long before being written to disk > and acknowledged as received. However this occurs at the cost of some > efficiency, which can affect throughput. If you have large volumes of > messages and the throughput is more important, then using fewer but > larger buffer pages will help you. > > Be aware, however, that the product of the size and number of pages is > the total memory that will be consumed and held by the broker for > buffering *per queue*. If you have a very large number of queues, then > you must watch out that you don't over-size your write buffers or else > you will run out of memory. > > While I cannot give you specific answers, as these depend on your > performance priorities, I suggest some trial-and-error if you want to > adjust these values. > > The Transaction Prepared List (TPL) is a special global queue for > persisting transaction boundaries. As this info is usually small and > relatively infrequent, the tpl-* settings apply to this queue only and > the user has the option to use different values than the regular queues. > If you don't use transactions, then this can be ignored. It is not a > queue that can be written to directly, but the store creates its own > data that is saved in this queue. Adjusting the tpl-* settings depends > only on the frequency of transactions in the user's application or > use-case. > > Hope that helps, > > Kim van der Riet > > On 11/27/18 4:44 PM, rammohan ganapavarapu wrote: > > Kim, > > > > 1. My message size is around 80kb, so what would be suggested values for > > the blow properties? > > > > > > wcache-page-size > > wcache-num-pages > > tpl-wcache-num-pages > > tpl-wcache-page-size > > > > right now i have all defaults, so i am trying to see if i can tune these > > values for my messages size to avoid those AIO busy cases. I have try to > > define those properties/options in qpidd.conf file but when i run > > qpid-config queues its not showing those values on my queues created by > > client application, do i have to define those options when i create queue > > instead of keep them in qpidd.conf? > > > > 2. What is difference b/w tpl-wcache-page-size and wcache-page-size > > > > Thanks, > > Ram > > > > On Fri, Nov 16, 2018 at 9:26 AM Kim van der Riet <[email protected]> > > wrote: > > > >> There is little documentation on linearstore. Certainly, the Apache docs > >> don't contain much. I think this is an oversight, but it won't get fixed > >> anytime soon. > >> > >> Kim > >> > >> On 11/16/18 12:11 PM, rammohan ganapavarapu wrote: > >>> Any one point me to the doc where i can read internals about how > >>> linearstore works and how qpid uses it? > >>> > >>> Thanks, > >>> Ram > >>> > >>> On Mon, Nov 12, 2018 at 8:43 AM rammohan ganapavarapu < > >>> [email protected]> wrote: > >>> > >>>> Kim, > >>>> > >>>> Thanks for clearing that up for me, does it support SAN storage > blocks. > >>>> Where can i read more about linearstore if i want to know the low > level > >>>> internals? > >>>> > >>>> Ram > >>>> > >>>> On Mon, Nov 12, 2018 at 8:32 AM Kim van der Riet <[email protected] > > > >>>> wrote: > >>>> > >>>>> The linearstore relies on using libaio for its async disk writes. The > >>>>> O_DIRECT flag is used, and this requires a block of aligned memory to > >>>>> serve as a memory buffer for disk write operations. To my knowledge, > >>>>> this technique only works with local disks and controllers. NFS does > >> not > >>>>> allow for DMA memory writes to disk AFAIK, and for as long as I can > >>>>> remember, has been a problem for the linearstore. With some work it > >>>>> might be possible to make it work using another write technique > though. > >>>>> NFS has never been a "supported" medium for linearstore. > >>>>> > >>>>> On 11/9/18 4:28 PM, rammohan ganapavarapu wrote: > >>>>>> But how does NFS will cause this issue, i am interested to see > because > >>>>> we > >>>>>> are using NFS (V4 version) in some environments, so wanted to learn > >>>>> tunings > >>>>>> when we use NFS. > >>>>>> > >>>>>> Thanks, > >>>>>> Ram > >>>>>> > >>>>>> On Fri, Nov 9, 2018 at 6:48 AM rammohan ganapavarapu < > >>>>>> [email protected]> wrote: > >>>>>> > >>>>>>> Sorry, i thought it's NFS but it's actually SAN storage volume. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Ram > >>>>>>> > >>>>>>> On Fri, Nov 9, 2018, 2:10 AM Gordon Sim <[email protected] wrote: > >>>>>>> > >>>>>>>> On 08/11/18 16:56, rammohan ganapavarapu wrote: > >>>>>>>>> I was wrong about the NFS for qpid journal files, looks like they > >>>>> are on > >>>>>>>>> NFS, so does NFS cause this issue? > >>>>>>>> Yes, I believe it does. What version of NFS are you using? > >>>>>>>> > >>>>>>>> > >> --------------------------------------------------------------------- > >>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>> > >>>>>>>> > >>>>> --------------------------------------------------------------------- > >>>>> To unsubscribe, e-mail: [email protected] > >>>>> For additional commands, e-mail: [email protected] > >>>>> > >>>>> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
