------- Comment From [email protected] 2018-07-24 00:03 EDT------- There are a couple of points about the CMA and HPT allocations:
1. The HPT is probably getting sized according to the maximum memory of the guest, not the initial amount of memory. Since we haven't been given any specifics about the configuration of the guests, I can't tell whether they are configured with maxmem greater than current/initial memory. If the guests are configured with a very large maxmem, their HPTs will be much larger than the 128MB mentioned in previous comments, and that will obviously make it much more likely to have problems allocating the HPT. (With a sufficiently recent host kernel, guest kernel and QEMU, the HPT can be resized while the guest is running, and in that case, QEMU determines the size of the HPT from the current memory rather than the maximum memory. However, HPT resizing went into the kernel later than 4.4, and I don't believe it has been backported into the Ubuntu version of 4.4, hence the "probably" in the previous paragraph.) 2. The memory in the CMA zone is not locked away in the way that previous comments imply. Memory in the CMA zone is still available for movable allocations, which includes page cache and anonymous pages for user processes, as well as memory for KVM guests. It is not available for kernel allocations (including things like network packet buffers). Thus it is worth while trying a larger kvm_cma_resv_ratio value in situations like this. When fragmentation occurs, the parts of the CMA zone that are too fragmented to use for HPTs can still be used for running user processes and backing KVM guests. 3. Other relevant factors are whether the guest has any real PCI devices passed through to it, and whether the guest is backed with large pages. If the guest has any PCI devices passed through, then when the guest sets up the DDW (dynamic DMA windows) TCE (iommu) table at boot time, that will have the effect of pinning all the guest memory. Some of the guest memory may have been allocated from the CMA zone. Balbir's patch (which is in the Ubuntu 4.4 kernel now) will try to migrate any pages that are in the CMA zone to somewhere else before they get pinned, but if memory is in short supply, it may not succeed in moving the page out of the CMA zone, and also the patch doesn't cope with large (16M) pages, whether THP or explicit large pages. Thus it would be worth disabling THP in the host. I'm not sure whether explicit large pages could ever come from the CMA zone. If the guests were backed by large pages, it would be worth trying without large-page backing. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1781038 Title: KVM guest hash page table failed to allocate contiguous memory (CMA) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1781038/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
