[
https://issues.apache.org/jira/browse/QPID-7666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859981#comment-15859981
]
Kim van der Riet commented on QPID-7666:
----------------------------------------
h1. Release Notes
h3. Linearstore per-queue memory options
The linearstore has two options that control the default per-queue memory
allocation:
* {{\-\-wcache-page-size}} which sets the write buffer page size in kiB
(default: 16 kiB)
* {{\-\-wcache-num-pages}} which sets the number of pages (default: 16)
The total per-queue memory allocated for this buffer is the product of these
two paramters. (default: 256kiB/queue).
h3. Using per-queue memory options
If the broker uses log level _info+_ (ie {{\-\-log-enable info+}}, then it is
possible to see the following messages:
If a broker is started without setting any store options, then the default
values will be used and displayed:
{noformat}
...
2017-02-08 13:35:33 [Store] info Linear Store: > Default EFP file size: 2048
(KiB)
2017-02-08 13:35:33 [Store] info Linear Store: > Default write cache page size:
16 (KiB)
2017-02-08 13:35:33 [Store] info Linear Store: > Default number of write cache
pages: 16
...
{noformat}
If a broker is started using the options either on the command-line or in the
configuration file (located at /etc/qpid/qpidd.conf), then these values will
set the default values for new queues. For example, setting the following in
the qpidd.conf file:
{noformat}
wcache-page-size=4
wcache-num-pages=8
efp-file-size=1024
{noformat}
or using the parameters {{--wcache-page-size 4 --wcache-num-pages 8
--efp-file-size 1024}} on the command-line will result in the broker taking
these values on startup:
{noformat}
...
2017-02-09 10:59:18 [Store] info Linear Store: > Default EFP file size: 1024
(KiB)
2017-02-09 10:59:18 [Store] info Linear Store: > Default write cache page size:
4 (KiB)
2017-02-09 10:59:18 [Store] info Linear Store: > Default number of write cache
pages: 8
...
{noformat}
This sets the _default_ value for all newly created queues on that broker.
However, if {{qpid-config}} is used to create the queue, or if a client sets
the appropriate properties on the to-be-created queue, then these defaults can
be overridden or updated to new values:
{noformat}
$ qpid-config add queue test-queue-0 --durable
$ qpid-config add queue test-queue-1 --durable --wcache-page-size 64
$ qpid-config add queue test-queue-2 --durable --wcache-page-size 64
--wcache-num-pages 128 --efp-pool-file-size 512
$ qpid-config queues
Queue Name Attributes
=================================================================
7aad5f88-11d3-48ec-abad-bdbd6f1d048d:0.0 auto-del excl
test-queue-0 durable
test-queue-1 durable wcache-page-size=64
test-queue-2 durable efp-pool-file-size=512
wcache-page-size=64 wcache-num-pages=128
{noformat}
and the broker prints details of any non-default queues:
{noformat}
2017-02-09 12:01:48 [Store] info Linear Store: Journal "test-queue-0": Created
2017-02-09 12:01:52 [Store] info Linear Store: Journal "test-queue-1": Created,
parameters: qpid.wcache-page-size=16
2017-02-09 12:01:55 [Store] info Linear Store: Journal "test-queue-2": Created,
parameters: qpid.wcache-num-pages=128 qpid.wcache-page-size=16
qpid.efp_pool_file_size=512
{noformat}
h3. Journal options
The linearstore has two options that control the default per-queue memory
allocation:
* {{\-\-efp-partition}} which sets the partition to use for creating new
journals. (default: 1)
* {{\-\-efp-file-size}} which sets the size of each journal file used in kiB
(default: 2048kiB)
Empty journal files are returned to an empty file pool (EFP) for re-use within
that partition.
h3. Disk partitions in the linearstore
Disk partitions were introduced in linearstore to allow more than one disk
media to be present at the same time for queue use. For example, certain queues
with critical low-latency or high-volume traffic patterns may require expensive
solid-state drives, while others may have low volume or non-latency-critical
use-cases. By correctly mounting the high-performance media within the store
directory structure, it is possible to set up a "partition" which the store can
use for creating journals. By directing certain queues to use certain
partitions for its journals, it is possible to control which media a queues
journals will use.
h4. Setting default per-queue options
When the broker is started, using the following options will set default values
for all new queues that are created on that broker:
|| Option || Default || Range || Description ||
| {{store-dir}} | Broker {{data-dir}} option | | Location of store directory |
| {{truncate}} | false | | If true, will discard all existing records from all
queues rather than recover them |
| {{wcache-page-size}} | 16 (kiB) | Powers of 2 from 4 up | Size of the pages
in the queue write page cache in KiB|
| {{wcache-num-pages}} | 16 | >=4 | Number of pages in the queue write page
cache |
| {{tpl-wcache-page-size}} | 16 (kiB) | Powers of 2 from 4 up | Size of the
pages in the transaction prepared list (TPL) write page cache in KiB|
| {{tpl-wcache-num-pages}} | 16 | >=4 | Number of pages in the transaction
prepared list (TPL) write page cache |
| {{efp-partition}} | 1 | >=1 | Default journal partition number as set up in
the {{store-dir}} directory under {{qls/}} |
| {{efp-file-size}} | 2018 (kiB) | Multiple of 4 >=4 | Default Empty File Pool
(EFP) file size in KiB to use for journal files |
| {{overwrite-before-return}} | false | | If true, will overwrite each file
before being returned to Empty FIle Pool. Used to erase potentially sensitive
information that might persist in the journal files from old messages |
| {{journal-flush-timeout}} | 500 (ms) | 0+ | Maximum time to wait to flush
journal in milliseconds |
(!) NOTE: Default values for Empty File Pool (EFP) parameters
({{efp-partition}} and {{efp-file-size}}) _*do not affect the settings used on
existing queues*_. If a broker is stopped and the defaults are changed, then on
restart, the existing queues will be restored using their original EFP values.
*Only new queues will use the new default EFP values.*
h3. Optimizing for store performance
These values can be used to optimize the performance for the disk media and
message load. Using a small page size works well for small messages, while a
large page size works better for large messages or a burst of smaller messages.
(on) Each page is used to make an asynchronous write to disk, so more pages
means more simultaneous writes are possible during high load situations.
However, if there is a small volume of messages or they occur infrequently,
then having a large number of pages does not provide any advantages and may be
a waste of memory resources.
(on) It is recommended that the values used for .{{\-\-wcache-page-size}} and
{{\-\-wcache-num-pages}} be tested and tuned for the disk media and message
load profile to achieve optimum performance. Apart from the message load
characteristics (ie size, frequency and consistency), the disk type, controller
and networking environment (ie how many remote clients, network speed, etc.)
will all affect the optimum settings.
(on) There is no single one-rule-fits-all for optimum queue settings.
h3. Log message updates
Some of the log messages have been cleaned up:
* Missing space in some log messages fixed
* On shutdown, message is now {{2017-02-08 13:54:04 \[Store\] info Linear
Store: Journal "test-queue-0": Stopped}} (rather than {{Destroyed}} as was
previously the case).
* Removed old debug message on queue creation, which was a duplicate of an
existing message.
h3. qpid-config updates
The following options have been added to qpid-config (as described above):
* {{\-\-wcache-page-size}}, which sets the store option
{{"qpid.wcache-page-size"}} as described above
* {{\-\-wcache-num-pages}}, which sets the store option
{{"qpid.wcache-num-pages"}} as described above
In addition, if the command {{qpid-config queues}} is used, these two options
are printed out as shown above.
> [linearstore] Enhancements to the write buffer default and per-queue options
> ----------------------------------------------------------------------------
>
> Key: QPID-7666
> URL: https://issues.apache.org/jira/browse/QPID-7666
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Affects Versions: qpid-cpp-1.37.0
> Reporter: Kim van der Riet
> Assignee: Kim van der Riet
>
> In applications where large numbers of persistent queues are created on a
> single broker, control over the default and per-queue store options are
> important so that resources are not wasted or become exhausted prematurely.
> There are two resources of concern:
> # Memory allocated per queue for buffering;
> # Disk resources consumed for journal files.
> Currently, the default setting for memory allocation is 1024kiB/queue (ie 32
> pages of 32kiB). The only default setting available is the page size (using
> the {{wcache-page-size}} option), and the number of pages is set internally
> according to the following table:
> ||{{wcache-page-size}}||{{wcache-num-pages}}||total mmap||
> ||(kiB)||(set interanlly)||(kiB)||
> | 4 | 64 | 256 |
> | 8 | 64 | 512|
> | 16 | 32 | 512|
> | 32* | 32 | 1024|
> | 64 | 16 | 1024|
> |128 | 8 | 1024|
> \* default
> This gives in effect only 3 per-queue memory allocation options: 256kiB,
> 512kiB and 1024kiB.
> Currently, the values of {{wcache-page-size}} is limited to the powers of 2
> between 4 and 128.
> It is suggested that the {{wcache-num-pages}} value be set as as an option
> rather than internally, which would give the user much greater control over
> the per-queue memory footprint and the buffer geometry (which can be
> optimized for the disk hardware and queue load characteristics).
> In addition, the upper value limit on {{wcache-page-size}} should be removed
> (as it is rather arbitrary) and the default value should be revised.
> Further, it would be valuable to add these parameters to {{qpid-config}} so
> that if a queue is created using this tool, the default values for the store
> parameters can be overridden on a per-queue basis.
> Disk resource controls are currently in place for the EFP partition
> ({{efp-partition}}) and the size of the EFP files ({{efp-file-size}}), both
> as a default and in {{qpid-config}}. The default values should also be
> revised.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
