[
https://issues.apache.org/jira/browse/QPID-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kim van der Riet updated QPID-5947:
-----------------------------------
Description:
The linearstore consumes kernel resources in such a way as to limit the number
of possible queues that may be handled on a given hardware system. The current
kernel-limited resources are:
# File descriptors. Currently 1 is used per queue.
# AIO contexts.
Currently both of these have per-process and total default limits which can be
modified through configuration of the kernel, but there remains an upper limit
which cannot be exceeded. If we intend to be able to persist queues in the
millions, changes to the way these resources are used will have to be made.
In addition, a large number of queues will have a negative impact on the time
taken to recover them. While this is expected behaviour, there are improvements
(such as multi-threaded recovery) that can be made which will help overcome
these issues.
This bug is intended as a long-term container for individual changes intended
to address these and other queue scalability issues. As these issues are
identified and tackled, they should be added as a sub-task of this issue.
was:
The linearstore consumes kernel resources in such a way as to limit the number
of possible queues that may be handled on a given hardware system. The current
kernel-limited resources are:
#. File descriptors. Currently 1 is used per queue.
# AIO contexts.
Currently both of these have per-process and total default limits which can be
modified through configuration of the kernel, but there remains an upper limit
which cannot be exceeded. If we intend to be able to persist queues in the
millions, changes to the way these resources are used will have to be made.
In addition, a large number of queues will have a negative impact on the time
taken to recover them. While this is expected behaviour, there are improvements
(such as multi-threaded recovery) that can be made which will help overcome
these issues.
This bug is intended as a long-term container for individual changes intended
to address these and other queue scalability issues. As these issues are
identified and tackled, they should be added as a sub-task of this issue.
> [linearstore] Improvements to queue scalability
> -----------------------------------------------
>
> Key: QPID-5947
> URL: https://issues.apache.org/jira/browse/QPID-5947
> Project: Qpid
> Issue Type: Improvement
> Components: C++ Broker
> Reporter: Kim van der Riet
> Assignee: Kim van der Riet
>
> The linearstore consumes kernel resources in such a way as to limit the
> number of possible queues that may be handled on a given hardware system. The
> current kernel-limited resources are:
> # File descriptors. Currently 1 is used per queue.
> # AIO contexts.
> Currently both of these have per-process and total default limits which can
> be modified through configuration of the kernel, but there remains an upper
> limit which cannot be exceeded. If we intend to be able to persist queues in
> the millions, changes to the way these resources are used will have to be
> made.
> In addition, a large number of queues will have a negative impact on the time
> taken to recover them. While this is expected behaviour, there are
> improvements (such as multi-threaded recovery) that can be made which will
> help overcome these issues.
> This bug is intended as a long-term container for individual changes intended
> to address these and other queue scalability issues. As these issues are
> identified and tackled, they should be added as a sub-task of this issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
