On 3/30/21 5:35 PM, Anthony Yznaga wrote:
> This patchset implements preserved-over-kexec memory storage or PKRAM as a
> method for saving memory pages of the currently executing kernel so that
> they may be restored after kexec into a new kernel. The patches are adapted
> from an RFC patchset sent out in 2013 by Vladimir Davydov [1]. They
> introduce the PKRAM kernel API and implement its use within tmpfs, allowing
> tmpfs files to be preserved across kexec.
>
> One use case for PKRAM is preserving guest memory and/or auxillary supporting
> data (e.g. iommu data) across kexec in support of VMM Fast Restart[2].
> VMM Fast Restart is currently using PKRAM to support preserving "Keep Alive
> State" across reboot[3]. PKRAM provides a flexible way for doing this
> without requiring that the amount of memory used by a fixed size created
> a priori. Another use case is for databases to preserve their block caches
> in shared memory across reboot.
Hi Anthony,
I have several concerns about preserving arbitrary not prereserved segments
across reboot.
1. PKRAM does not work across firmware reboots
With emulated persistent memory it is possible to do reboot through firmware
and not loose the preserved-memory. The firmware can be modified to mark the
required ranges pages as PRAM, and Linux will treat them as such. The benefit
of this is that it works for both cases kexec and reboot through firmware. The
disadvantage is that you have to know in advance how much memory needs to be
preserved. However, with the ability to hot-plug/hot-remove the PMEM, the
second point becomes moot as it is possible to mark a large chunk of memory as
PMEM if needed. I have designed something like this for one of our projects,
and it is already been used in the fleet. Reboot through firmware, allows us to
service firmware in addition to kernel.
2. Boot failures due to memory fragmentation
We also considered using PRAM instead of PMEM. PRAM was one of the previous
attempts to do the persistent memory thing via tmpfs flag: mount -t tmpfs -o
pram=mytmpfs none /mnt/crdump"; that project was never upstreamed. However, we
gave up with that idea because in addition to loosing possibility to reboot
through the firmware, it also adds memory fragmentation. For example, if the
new kernel require larger contiguous memory chunks to be allocated during boot
than the previous kernel (i.e. the next kernel has new drivers, or some debug
feature enabled), the boot might simply fail because of the extra memory ranges
being reserved.
3. New intra-kernel dependencies
Kexec reboot is when one Linux kernel works as a bootloader for the next one.
Currently, there is very little information that is passed from the old kernel
to the next kernel. Adding more information that two independent kernels must
know about each other is not a very good thing from architectural point of
view. It limits the flexibility of kexec.
However, we do need PKRAM and ability to preserve kernel memory across reboot
for fast hypervisor updates or such. User pages can already be preserved across
reboot on emulated or real persistent memory. The easiest way is via DAXFS
placed on that memory.
Kernel cannot preserve its memory on PMEM across the reboot. However,
functionality can be extended so kernel memory can be preserved on both
emulated persistent memory or on real persistent memory. PKRAM could provide an
interface to save kernel data to a file, and that file could be placed on any
filesystem including DAXFS. When placed on DAXFS, that file can be used as
iommu data, as it is actually located in physical memory and not moving
anywhere. It is preserved across firmware/kexec reboot with having the devices
survive the reboot state intact. During boot, have the device drivers that use
PKRAM preserve functionality map saved files from DAXFS in order to have IOMMU
functionality working again.
Thank you,
Pasha
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
