Branch: refs/heads/staging
Home: https://github.com/qemu/qemu
Commit: 86a637e48104ae74d8be53bed6441ce32be33433
https://github.com/qemu/qemu/commit/86a637e48104ae74d8be53bed6441ce32be33433
Author: Stefan Hajnoczi <stefa...@redhat.com>
Date: 2024-03-19 (Tue, 19 Mar 2024)
Changed paths:
M util/qemu-coroutine.c
Log Message:
-----------
coroutine: cap per-thread local pool size
The coroutine pool implementation can hit the Linux vm.max_map_count
limit, causing QEMU to abort with "failed to allocate memory for stack"
or "failed to set up stack guard page" during coroutine creation.
This happens because per-thread pools can grow to tens of thousands of
coroutines. Each coroutine causes 2 virtual memory areas to be created.
Eventually vm.max_map_count is reached and memory-related syscalls fail.
The per-thread pool sizes are non-uniform and depend on past coroutine
usage in each thread, so it's possible for one thread to have a large
pool while another thread's pool is empty.
Switch to a new coroutine pool implementation with a global pool that
grows to a maximum number of coroutines and per-thread local pools that
are capped at hardcoded small number of coroutines.
This approach does not leave large numbers of coroutines pooled in a
thread that may not use them again. In order to perform well it
amortizes the cost of global pool accesses by working in batches of
coroutines instead of individual coroutines.
The global pool is a list. Threads donate batches of coroutines to when
they have too many and take batches from when they have too few:
.-----------------------------------.
| Batch 1 | Batch 2 | Batch 3 | ... | global_pool
`-----------------------------------'
Each thread has up to 2 batches of coroutines:
.-------------------.
| Batch 1 | Batch 2 | per-thread local_pool (maximum 2 batches)
`-------------------'
The goal of this change is to reduce the excessive number of pooled
coroutines that cause QEMU to abort when vm.max_map_count is reached
without losing the performance of an adequately sized coroutine pool.
Here are virtio-blk disk I/O benchmark results:
RW BLKSIZE IODEPTH OLD NEW CHANGE
randread 4k 1 113725 117451 +3.3%
randread 4k 8 192968 198510 +2.9%
randread 4k 16 207138 209429 +1.1%
randread 4k 32 212399 215145 +1.3%
randread 4k 64 218319 221277 +1.4%
randread 128k 1 17587 17535 -0.3%
randread 128k 8 17614 17616 +0.0%
randread 128k 16 17608 17609 +0.0%
randread 128k 32 17552 17553 +0.0%
randread 128k 64 17484 17484 +0.0%
See files/{fio.sh,test.xml.j2} for the benchmark configuration:
https://gitlab.com/stefanha/virt-playbooks/-/tree/coroutine-pool-fix-sizing
Buglink: https://issues.redhat.com/browse/RHEL-28947
Reported-by: Sanjay Rao <s...@redhat.com>
Reported-by: Boaz Ben Shabat <bbens...@redhat.com>
Reported-by: Joe Mario <jma...@redhat.com>
Reviewed-by: Kevin Wolf <kw...@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefa...@redhat.com>
Message-ID: <20240318183429.1039340-1-stefa...@redhat.com>
Commit: 4992306b65a09f2632f935dabc598ecdcde50121
https://github.com/qemu/qemu/commit/4992306b65a09f2632f935dabc598ecdcde50121
Author: Peter Maydell <peter.mayd...@linaro.org>
Date: 2024-03-19 (Tue, 19 Mar 2024)
Changed paths:
M util/qemu-coroutine.c
Log Message:
-----------
Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into
staging
Pull request
This fix solves the "failed to set up stack guard page" error that has been
reported on Linux hosts where the QEMU coroutine pool exceeds the
vm.max_map_count limit.
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAmX5qq0ACgkQnKSrs4Gr
# c8ginQf8DRKzA7K8OivEegKpf0TgGcAcw9/xKc6zJH3X0/GXi1my61tzz+XUkbNy
# /R9HRrjBUb4MhSmJzP9kxuPFcBD5fZeipg4eTqtJCdi+DQ57+YypShVpsDrD7eNv
# X5dxeeONdWwP+k9JiOj9NtSOMmTKExn/Q/w45G2eeBlJh4yRA+56XN/dDXTFlidm
# NEpOGrKbyFKuAf/ZwYmeBr4aqIGTN3UgOVco/rqkGPYPTYpKlCoE5rSTEnQrbR7/
# C9KojlrGawJXlKjxfu/6i7yGHrv0eJ2N1VauvR/DHhQvdRhojVVt3NFGG/WJi+cL
# CMbxNyYeQJLNFtfPWzokjKEudxkshg==
# =lznr
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 19 Mar 2024 15:09:33 GMT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefa...@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefa...@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8
* tag 'block-pull-request' of https://gitlab.com/stefanha/qemu:
coroutine: cap per-thread local pool size
Signed-off-by: Peter Maydell <peter.mayd...@linaro.org>
Compare: https://github.com/qemu/qemu/compare/7e9595e41502...4992306b65a0
To unsubscribe from these emails, change your notification settings at
https://github.com/qemu/qemu/settings/notifications
