Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On 2026-06-03 15:03 +, Pasha Tatashin wrote: > On 06-03 17:34, Mike Rapoport wrote: > > On 2026-06-03 14:11 +, Pasha Tatashin wrote: > > > On 06-03 16:59, Mike Rapoport wrote: > > > > On Wed, Jun 03, 2026 at 12:05:04PM +, Pasha Tatashin wrote: > > > > > On 06-03 09:49, Mike Rapoport wrote: > > > > > > On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin > > > > > > wrote: > > > > > > > diff --git a/include/linux/kho/abi/block.h > > > > > > > b/include/linux/kho/abi/block.h > > > > > > > new file mode 100644 > > > > > > > index ..8641c20b379b > > > > > > > --- /dev/null > > > > > > > +++ b/include/linux/kho/abi/block.h > > > > > > > @@ -0,0 +1,56 @@ > > > > > > > [ ... skip 25 lines ... ] > > > > > > > +#define _LINUX_KHO_ABI_BLOCK_H > > > > > > > + > > > > > > > +#include > > > > > > > +#include > > > > > > > + > > > > > > > +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1" > > > > > > > > > > > > It's never used by block set and after looking at the following > > > > > > patches I > > > > > > found that it's appended to LUO compatible string. > > > > > > > > > > > > While this works for LUO, I think it should be > > > > > > kho_block_set_restore() > > > > > > responsibility to verify the compatibility. > > > > > > > > > > It should work for any component that relies on kho_block. My > > > > > proposal > > > > > is to use this method for other common KHO data structures (e.g., > > > > > kho > > > > > vmalloc, kho radix, future kho xarray). There is no need for them to > > > > > carry the compatibility string in their metadata, as whoever uses > > > > > them > > > > > will include their compatibility string. > > > > > > > > So if, say, memfd_luo uses kho vmalloc, xarray and blocks it'll have > > > > five > > > > compatibility strings glued together? > > > > > > That is correct, but it will be in only one place: the header of the > > > client's KHO subtree. Since it is dynamically sized and 8-byte aligned, > > > it should be safe to include in any struct. > > > > This is safe, you are right. > > But I have more usability concerns from one side and the duplication it > > causes from the other. > > > > I can see the downside of putting the version information in the data > > structure itself as it either requires a different header for the first > > element or needlessly increases all the headers. > > > > But > > > > #define LUO_ABI_COMPATIBLE LUO_COMPAT_BASE "-" KHO_BLOCK_ABI_COMPATIBLE > > "-" KHO_VMALLOC_ABI_COMPATIBLE "-" KHO_RADIX_COMPATIBLE > > > > is not really digestible too. And it forces KHO users to potentially > > track KHO internal changes. > > These are compatibilities; I think they are quite digestible, both to > write and also when the LUO_ABI_COMPATIBLE string is printed out for > debugging/info purposes. I agree to disagree :) It's KHO property, not it's users. > > We still don't promise any compatibility between different kernel > > versions so to avoid blocking this series on the decision what is the > > best way to convey KHO data structures compatibility I suggest to bump > > kho ABI version in v6.2* of the patch that adds KHO blocks and postpone > > this discussion to after rc1 when we'll have plenty of time. > > Let's keep this patch as is for now. We will have a broader discussion > when we convert other participants to this new scheme. If we decide not > to pursue this approach, we will change this code to use an independent > compatibility string. However, having this in place as a template will > help us convert other components correctly, ensuring proper alignment > and that correct string helpers like strncmp / strscpy are used—which I > have already ensured is the case in LUO. Pasha, this sounds like salami approach :) We didn't agree yet to convert other components and even to use this scheme globally. Changing this during -rc does not seem a good practice. So whatever new versioning scheme we'll come up with, it'll have to wait until v7.3. Let's bump kho and LUO ABI versions and drop the concatenation for now. It's a small change to the patches, so I don't see it as a blocker for merging them in v7.2. > > * sending a new version of a single file does same email traffic, but it > > confuses b4 and quite possibly other tools, so I think v7 is better. > > Agreed, I also prefer re-sending the whole series... > > Pasha > > > > Pasha > > > > > > > > > > > > For now, reviewers will have to make sure that if the ABI header > > > > > content > > > > > is changed, the compatibility string is updated. > > > > > > > -- > > > > Sincerely yours, > > > > Mike. > > > > > > > >
Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On 06-03 17:34, Mike Rapoport wrote: > On 2026-06-03 14:11 +, Pasha Tatashin wrote: > > On 06-03 16:59, Mike Rapoport wrote: > > > On Wed, Jun 03, 2026 at 12:05:04PM +, Pasha Tatashin wrote: > > > > On 06-03 09:49, Mike Rapoport wrote: > > > > > On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin > > > > > wrote: > > > > > > diff --git a/include/linux/kho/abi/block.h > > > > > > b/include/linux/kho/abi/block.h > > > > > > new file mode 100644 > > > > > > index ..8641c20b379b > > > > > > --- /dev/null > > > > > > +++ b/include/linux/kho/abi/block.h > > > > > > @@ -0,0 +1,56 @@ > > > > > > [ ... skip 25 lines ... ] > > > > > > +#define _LINUX_KHO_ABI_BLOCK_H > > > > > > + > > > > > > +#include > > > > > > +#include > > > > > > + > > > > > > +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1" > > > > > > > > > > It's never used by block set and after looking at the following > > > > > patches I > > > > > found that it's appended to LUO compatible string. > > > > > > > > > > While this works for LUO, I think it should be kho_block_set_restore() > > > > > responsibility to verify the compatibility. > > > > > > > > It should work for any component that relies on kho_block. My proposal > > > > is to use this method for other common KHO data structures (e.g., kho > > > > vmalloc, kho radix, future kho xarray). There is no need for them to > > > > carry the compatibility string in their metadata, as whoever uses them > > > > will include their compatibility string. > > > > > > So if, say, memfd_luo uses kho vmalloc, xarray and blocks it'll have five > > > compatibility strings glued together? > > > > That is correct, but it will be in only one place: the header of the > > client's KHO subtree. Since it is dynamically sized and 8-byte aligned, > > it should be safe to include in any struct. > > This is safe, you are right. > But I have more usability concerns from one side and the duplication it > causes from the other. > > I can see the downside of putting the version information in the data > structure itself as it either requires a different header for the first > element or needlessly increases all the headers. > > But > > #define LUO_ABI_COMPATIBLE LUO_COMPAT_BASE "-" KHO_BLOCK_ABI_COMPATIBLE > "-" KHO_VMALLOC_ABI_COMPATIBLE "-" KHO_RADIX_COMPATIBLE > > is not really digestible too. And it forces KHO users to potentially > track KHO internal changes. These are compatibilities; I think they are quite digestible, both to write and also when the LUO_ABI_COMPATIBLE string is printed out for debugging/info purposes. > We still don't promise any compatibility between different kernel > versions so to avoid blocking this series on the decision what is the > best way to convey KHO data structures compatibility I suggest to bump > kho ABI version in v6.2* of the patch that adds KHO blocks and postpone > this discussion to after rc1 when we'll have plenty of time. Let's keep this patch as is for now. We will have a broader discussion when we convert other participants to this new scheme. If we decide not to pursue this approach, we will change this code to use an independent compatibility string. However, having this in place as a template will help us convert other components correctly, ensuring proper alignment and that correct string helpers like strncmp / strscpy are used—which I have already ensured is the case in LUO. > * sending a new version of a single file does same email traffic, but it > confuses b4 and quite possibly other tools, so I think v7 is better. Agreed, I also prefer re-sending the whole series... Pasha > > Pasha > > > > > > > > > For now, reviewers will have to make sure that if the ABI header > > > > content > > > > is changed, the compatibility string is updated. > > > > > -- > > > Sincerely yours, > > > Mike. > > > >
Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On 2026-06-03 14:11 +, Pasha Tatashin wrote: > On 06-03 16:59, Mike Rapoport wrote: > > On Wed, Jun 03, 2026 at 12:05:04PM +, Pasha Tatashin wrote: > > > On 06-03 09:49, Mike Rapoport wrote: > > > > On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin > > > > wrote: > > > > > diff --git a/include/linux/kho/abi/block.h > > > > > b/include/linux/kho/abi/block.h > > > > > new file mode 100644 > > > > > index ..8641c20b379b > > > > > --- /dev/null > > > > > +++ b/include/linux/kho/abi/block.h > > > > > @@ -0,0 +1,56 @@ > > > > > [ ... skip 25 lines ... ] > > > > > +#define _LINUX_KHO_ABI_BLOCK_H > > > > > + > > > > > +#include > > > > > +#include > > > > > + > > > > > +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1" > > > > > > > > It's never used by block set and after looking at the following patches > > > > I > > > > found that it's appended to LUO compatible string. > > > > > > > > While this works for LUO, I think it should be kho_block_set_restore() > > > > responsibility to verify the compatibility. > > > > > > It should work for any component that relies on kho_block. My proposal > > > is to use this method for other common KHO data structures (e.g., kho > > > vmalloc, kho radix, future kho xarray). There is no need for them to > > > carry the compatibility string in their metadata, as whoever uses them > > > will include their compatibility string. > > > > So if, say, memfd_luo uses kho vmalloc, xarray and blocks it'll have five > > compatibility strings glued together? > > That is correct, but it will be in only one place: the header of the > client's KHO subtree. Since it is dynamically sized and 8-byte aligned, > it should be safe to include in any struct. This is safe, you are right. But I have more usability concerns from one side and the duplication it causes from the other. I can see the downside of putting the version information in the data structure itself as it either requires a different header for the first element or needlessly increases all the headers. But #define LUO_ABI_COMPATIBLE LUO_COMPAT_BASE "-" KHO_BLOCK_ABI_COMPATIBLE "-" KHO_VMALLOC_ABI_COMPATIBLE "-" KHO_RADIX_COMPATIBLE is not really digestible too. And it forces KHO users to potentially track KHO internal changes. We still don't promise any compatibility between different kernel versions so to avoid blocking this series on the decision what is the best way to convey KHO data structures compatibility I suggest to bump kho ABI version in v6.2* of the patch that adds KHO blocks and postpone this discussion to after rc1 when we'll have plenty of time. * sending a new version of a single file does same email traffic, but it confuses b4 and quite possibly other tools, so I think v7 is better. > Pasha > > > > > > For now, reviewers will have to make sure that if the ABI header content > > > is changed, the compatibility string is updated. > > > -- > > Sincerely yours, > > Mike. >
Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On 06-03 16:59, Mike Rapoport wrote: > On Wed, Jun 03, 2026 at 12:05:04PM +, Pasha Tatashin wrote: > > On 06-03 09:49, Mike Rapoport wrote: > > > On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin > > > wrote: > > > > diff --git a/include/linux/kho/abi/block.h > > > > b/include/linux/kho/abi/block.h > > > > new file mode 100644 > > > > index ..8641c20b379b > > > > --- /dev/null > > > > +++ b/include/linux/kho/abi/block.h > > > > @@ -0,0 +1,56 @@ > > > > [ ... skip 25 lines ... ] > > > > +#define _LINUX_KHO_ABI_BLOCK_H > > > > + > > > > +#include > > > > +#include > > > > + > > > > +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1" > > > > > > It's never used by block set and after looking at the following patches I > > > found that it's appended to LUO compatible string. > > > > > > While this works for LUO, I think it should be kho_block_set_restore() > > > responsibility to verify the compatibility. > > > > It should work for any component that relies on kho_block. My proposal > > is to use this method for other common KHO data structures (e.g., kho > > vmalloc, kho radix, future kho xarray). There is no need for them to > > carry the compatibility string in their metadata, as whoever uses them > > will include their compatibility string. > > So if, say, memfd_luo uses kho vmalloc, xarray and blocks it'll have five > compatibility strings glued together? That is correct, but it will be in only one place: the header of the client's KHO subtree. Since it is dynamically sized and 8-byte aligned, it should be safe to include in any struct. Pasha > > > For now, reviewers will have to make sure that if the ABI header content > > is changed, the compatibility string is updated. > > -- > Sincerely yours, > Mike.
Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On Wed, Jun 03, 2026 at 12:05:04PM +, Pasha Tatashin wrote: > On 06-03 09:49, Mike Rapoport wrote: > > On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin > > wrote: > > > diff --git a/include/linux/kho/abi/block.h b/include/linux/kho/abi/block.h > > > new file mode 100644 > > > index ..8641c20b379b > > > --- /dev/null > > > +++ b/include/linux/kho/abi/block.h > > > @@ -0,0 +1,56 @@ > > > [ ... skip 25 lines ... ] > > > +#define _LINUX_KHO_ABI_BLOCK_H > > > + > > > +#include > > > +#include > > > + > > > +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1" > > > > It's never used by block set and after looking at the following patches I > > found that it's appended to LUO compatible string. > > > > While this works for LUO, I think it should be kho_block_set_restore() > > responsibility to verify the compatibility. > > It should work for any component that relies on kho_block. My proposal > is to use this method for other common KHO data structures (e.g., kho > vmalloc, kho radix, future kho xarray). There is no need for them to > carry the compatibility string in their metadata, as whoever uses them > will include their compatibility string. So if, say, memfd_luo uses kho vmalloc, xarray and blocks it'll have five compatibility strings glued together? > For now, reviewers will have to make sure that if the ABI header content > is changed, the compatibility string is updated. -- Sincerely yours, Mike.
Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On 06-03 12:05, Pasha Tatashin wrote:
> On 06-03 09:49, Mike Rapoport wrote:
> > On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin
> > wrote:
> > > diff --git a/include/linux/kho/abi/block.h b/include/linux/kho/abi/block.h
> > > new file mode 100644
> > > index ..8641c20b379b
> > > --- /dev/null
> > > +++ b/include/linux/kho/abi/block.h
> > > @@ -0,0 +1,56 @@
> > > [ ... skip 25 lines ... ]
> > > +#define _LINUX_KHO_ABI_BLOCK_H
> > > +
> > > +#include
> > > +#include
> > > +
> > > +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1"
> >
> > It's never used by block set and after looking at the following patches I
> > found that it's appended to LUO compatible string.
> >
> > While this works for LUO, I think it should be kho_block_set_restore()
> > responsibility to verify the compatibility.
>
> It should work for any component that relies on kho_block. My proposal
> is to use this method for other common KHO data structures (e.g., kho
> vmalloc, kho radix, future kho xarray). There is no need for them to
> carry the compatibility string in their metadata, as whoever uses them
> will include their compatibility string.
>
> For now, reviewers will have to make sure that if the ABI header content
> is changed, the compatibility string is updated.
>
> > > diff --git a/kernel/liveupdate/kho_block.c b/kernel/liveupdate/kho_block.c
> > > new file mode 100644
> > > index ..4f147c308e6b
> > > --- /dev/null
> > > +++ b/kernel/liveupdate/kho_block.c
> > > @@ -0,0 +1,411 @@
> > > [ ... skip 121 lines ... ]
> > > +/**
> > > + * kho_block_set_grow - Expand the block set to accommodate the target
> > > count.
> > > + * @bs:The block set.
> > > + * @count: The target number of valid entries to accommodate.
> > > + *
> > > + * Acts as a runtime notifier when new resources (such as files or
> > > sessions)
> >
> > Not sure I understand what "runtime notifier" means in this context.
>
> It came from discussion with Pratyush, but I think we are on the same
> page what they are, and I will just remove this.
>
> >
> > > [ ... skip 11 lines ... ]
> > > +
> > > + while (count > bs->nblocks * bs->count_per_block) {
> > > + int err = kho_block_set_grow_one(bs);
> > > +
> > > + if (err)
> > > + return err;
> >
> > This leaks memory if more than one block is added.
> >
> > > [ ... skip 31 lines ... ]
> > > + * unregistered, allowing the block set to release and unallocate
> > > redundant
> > > + * preserved memory blocks. Checks if the last block in the set can be
> > > removed
> > > + * because the remaining entry count is fully accommodated by the
> > > preceding blocks.
> > > + *
> > > + * Note: It is the caller's responsibility to ensure that entries are
> > > removed
> > > + * in LIFO (last-in, first-out) order (the reverse order of their
> > > insertion).
> >
> > I think "in LIFO order" is sufficient :)
>
> Oh, I keep removing those :-)
>
> > > [ ... skip 173 lines ... ]
> > > + it->i = 0;
> > > + }
> > > +
> > > + entry = kho_block_entry(it, it->i++);
> > > + it->block->ser->count = it->i;
> > > + return entry;
> >
> > This looks way better than the previous version :)
> > Thanks!
>
> Thank you. I will send a new version of this patch as a reply to this
> email to avoid cluttering the mailing list.
The patch is here:
https://lore.kernel.org/all/[email protected]/
I messed up in-reply-to field with wrong message-id.
Pasha
>
> Pasha
>
> >
> > --
> > Sincerely yours,
> > Mike.
> >
Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On 06-03 09:49, Mike Rapoport wrote:
> On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin
> wrote:
> > diff --git a/include/linux/kho/abi/block.h b/include/linux/kho/abi/block.h
> > new file mode 100644
> > index ..8641c20b379b
> > --- /dev/null
> > +++ b/include/linux/kho/abi/block.h
> > @@ -0,0 +1,56 @@
> > [ ... skip 25 lines ... ]
> > +#define _LINUX_KHO_ABI_BLOCK_H
> > +
> > +#include
> > +#include
> > +
> > +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1"
>
> It's never used by block set and after looking at the following patches I
> found that it's appended to LUO compatible string.
>
> While this works for LUO, I think it should be kho_block_set_restore()
> responsibility to verify the compatibility.
It should work for any component that relies on kho_block. My proposal
is to use this method for other common KHO data structures (e.g., kho
vmalloc, kho radix, future kho xarray). There is no need for them to
carry the compatibility string in their metadata, as whoever uses them
will include their compatibility string.
For now, reviewers will have to make sure that if the ABI header content
is changed, the compatibility string is updated.
> > diff --git a/kernel/liveupdate/kho_block.c b/kernel/liveupdate/kho_block.c
> > new file mode 100644
> > index ..4f147c308e6b
> > --- /dev/null
> > +++ b/kernel/liveupdate/kho_block.c
> > @@ -0,0 +1,411 @@
> > [ ... skip 121 lines ... ]
> > +/**
> > + * kho_block_set_grow - Expand the block set to accommodate the target
> > count.
> > + * @bs:The block set.
> > + * @count: The target number of valid entries to accommodate.
> > + *
> > + * Acts as a runtime notifier when new resources (such as files or
> > sessions)
>
> Not sure I understand what "runtime notifier" means in this context.
It came from discussion with Pratyush, but I think we are on the same
page what they are, and I will just remove this.
>
> > [ ... skip 11 lines ... ]
> > +
> > + while (count > bs->nblocks * bs->count_per_block) {
> > + int err = kho_block_set_grow_one(bs);
> > +
> > + if (err)
> > + return err;
>
> This leaks memory if more than one block is added.
>
> > [ ... skip 31 lines ... ]
> > + * unregistered, allowing the block set to release and unallocate redundant
> > + * preserved memory blocks. Checks if the last block in the set can be
> > removed
> > + * because the remaining entry count is fully accommodated by the
> > preceding blocks.
> > + *
> > + * Note: It is the caller's responsibility to ensure that entries are
> > removed
> > + * in LIFO (last-in, first-out) order (the reverse order of their
> > insertion).
>
> I think "in LIFO order" is sufficient :)
Oh, I keep removing those :-)
> > [ ... skip 173 lines ... ]
> > + it->i = 0;
> > + }
> > +
> > + entry = kho_block_entry(it, it->i++);
> > + it->block->ser->count = it->i;
> > + return entry;
>
> This looks way better than the previous version :)
> Thanks!
Thank you. I will send a new version of this patch as a reply to this
email to avoid cluttering the mailing list.
Pasha
>
> --
> Sincerely yours,
> Mike.
>
Re: [PATCH v6 07/13] kho: add support for linked-block serialization
On Wed, 03 Jun 2026 03:28:58 +, Pasha Tatashin
wrote:
> diff --git a/include/linux/kho/abi/block.h b/include/linux/kho/abi/block.h
> new file mode 100644
> index ..8641c20b379b
> --- /dev/null
> +++ b/include/linux/kho/abi/block.h
> @@ -0,0 +1,56 @@
> [ ... skip 25 lines ... ]
> +#define _LINUX_KHO_ABI_BLOCK_H
> +
> +#include
> +#include
> +
> +#define KHO_BLOCK_ABI_COMPATIBLE "kho-block-v1"
It's never used by block set and after looking at the following patches I
found that it's appended to LUO compatible string.
While this works for LUO, I think it should be kho_block_set_restore()
responsibility to verify the compatibility.
>
> diff --git a/kernel/liveupdate/kho_block.c b/kernel/liveupdate/kho_block.c
> new file mode 100644
> index ..4f147c308e6b
> --- /dev/null
> +++ b/kernel/liveupdate/kho_block.c
> @@ -0,0 +1,411 @@
> [ ... skip 121 lines ... ]
> +/**
> + * kho_block_set_grow - Expand the block set to accommodate the target count.
> + * @bs:The block set.
> + * @count: The target number of valid entries to accommodate.
> + *
> + * Acts as a runtime notifier when new resources (such as files or sessions)
Not sure I understand what "runtime notifier" means in this context.
> [ ... skip 11 lines ... ]
> +
> + while (count > bs->nblocks * bs->count_per_block) {
> + int err = kho_block_set_grow_one(bs);
> +
> + if (err)
> + return err;
This leaks memory if more than one block is added.
> [ ... skip 31 lines ... ]
> + * unregistered, allowing the block set to release and unallocate redundant
> + * preserved memory blocks. Checks if the last block in the set can be
> removed
> + * because the remaining entry count is fully accommodated by the preceding
> blocks.
> + *
> + * Note: It is the caller's responsibility to ensure that entries are removed
> + * in LIFO (last-in, first-out) order (the reverse order of their insertion).
I think "in LIFO order" is sufficient :)
> [ ... skip 173 lines ... ]
> + it->i = 0;
> + }
> +
> + entry = kho_block_entry(it, it->i++);
> + it->block->ser->count = it->i;
> + return entry;
This looks way better than the previous version :)
Thanks!
--
Sincerely yours,
Mike.
