On 15.07.20 19:38, David Hildenbrand wrote: > On 15.07.20 18:14, Heiko Carstens wrote: >> On Wed, Jul 15, 2020 at 01:42:02PM +0200, David Hildenbrand wrote: >>>> So, are you saying that even at IPL time there might already be memory >>>> devices attached to the system? And the kernel should _not_ treat them >>>> as normal memory? >>> >>> Sorry if that was unclear. Yes, we can have such devices (including >>> memory areas) on a cold boot/reboot/kexec. In addition, they might pop >>> up at runtime (e.g., hotplugging a virtio-mem device). The device is in >>> charge of exposing that area and deciding what to do with it. >>> >>> The kernel should never treat them as normal memory (IOW, system RAM). >>> Not during a cold boot, not during a reboot. The device driver is >>> responsible for deciding how to use that memory (e.g., add it as system >>> RAM), and which parts of that memory are actually valid to be used (even >>> if a tprot might succeed it might not be valid to use just yet - I guess >>> somewhat similar to doing a tport on a dcss area - AFAIK, you also don't >>> want to use it like normal memory). >>> >>> E.g., on x86-64, memory exposed via virtio-mem or virtio-pmem is never >>> exposed via the e820 map. The only trace that there might be *something* >>> now/in the future is indicated via ACPI SRAT tables. This takes >>> currently care of indicating the maximum possible PFN. >> >> Ok, but all of this needa to be documented somewhere. This raises a >> couple of questions to me: > > I assume this mostly targets virtio-mem, because the semantics of > virtio-mem provided memory are extra-weird (in contrast to rather static > virtio-pmem, which is essentially just an emulated NVDIMM - a disk > mapped into physical memory). > > Regarding documentation (some linked in the cover letter), so far I have > (generic/x86-64) > > 1. https://virtio-mem.gitlab.io/ > 2. virtio spec proposal [1] > 3. QEMU 910b25766b33 ("virtio-mem: Paravirtualized memory hot(un)plug") > 4. Linux 5f1f79bbc9 ("virtio-mem: Paravirtualized memory hotplug") > 5. Linux cover letter [2] > 6. KVM forum talk [3] [4] > > As your questions go quite into technical detail, and I don't feel like > rewriting the doc here :) , I suggest looking at [2], 1, and 5.
Sorry, I suggest looking at [3] (not [2]) first. Includes pictures and a comparison to memory ballooning (and DIMM-based memory hotplug). > [3] > https://events19.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Paravirtualized-Memory-David-Hildenbrand-Red-Hat-1.pdf > [4] https://www.youtube.com/watch?v=H65FDUDPu9s > -- Thanks, David / dhildenb
