Re: [PATCH -mm -v5 RESEND] mm, swap: Fix race between swapoff and some swap operations
On Wed, Feb 21, 2018 at 7:38 AM, Andrew Morton
wrote:
> On Sun, 18 Feb 2018 09:06:47 +0800 huang ying
> wrote:
>
>> >> >> +struct swap_info_struct *get_swap_device(swp_entry_t entry)
>> >> >> +{
>> >> >> + struct swap_info_struct *si;
>> >> >> + unsigned long type, offset;
>> >> >> +
>> >> >> + if (!entry.val)
>> >> >> + goto out;
>> >> >> + type = swp_type(entry);
>> >> >> + if (type >= nr_swapfiles)
>> >> >> + goto bad_nofile;
>> >> >> + si = swap_info[type];
>> >> >> +
>> >> >> + preempt_disable();
>> >> >
>> >> > This preempt_disable() is later than I'd expect. If a well-timed race
>> >> > occurs, `si' could now be pointing at a defunct entry. If that
>> >> > well-timed race include a swapoff AND a swapon, `si' could be pointing
>> >> > at the info for a new device?
>> >>
>> >> struct swap_info_struct pointed to by swap_info[] will never be freed.
>> >> During swapoff, we only free the memory pointed to by the fields of
>> >> struct swap_info_struct. And when swapon, we will always reuse
>> >> swap_info[type] if it's not NULL. So it should be safe to dereference
>> >> swap_info[type] with preemption enabled.
>> >
>> > That's my point. If there's a race window during which there is a
>> > parallel swapoff+swapon, this swap_info_struct may now be in use for a
>> > different device?
>>
>> Yes. It's possible. And the caller of get_swap_device() can live
>> with it if the swap_info_struct has been fully initialized. For
>> example, for the race in the patch description,
>>
>> do_swap_page
>> swapin_readahead
>> __read_swap_cache_async
>> swapcache_prepare
>> __swap_duplicate
>>
>> in __swap_duplicate(), it's possible that the swap device returned by
>> get_swap_device() is different from the swap device when
>> __swap_duplicate() call get_swap_device(). But the struct_info_struct
>> has been fully initialized, so __swap_duplicate() can reference
>> si->swap_map[] safely. And we will check si->swap_map[] before any
>> further operation. Even if the swap entry is swapped out again for
>> the new swap device, we will check the page table again in
>> do_swap_page(). So there is no functionality problem.
>
> That's rather revolting. Can we tighten this up? Or at least very
> loudly document it?
TBH, I think my original fix patch which uses a reference count in
swap_info_struct is easier to be understood. But I understand it has
its own drawbacks too. Anyway, unless there are some better ideas to
resolve this, I will send out a new version with more document.
Best Regards,
Huang, Ying
Re: [PATCH -mm -v5 RESEND] mm, swap: Fix race between swapoff and some swap operations
On Sun, 18 Feb 2018 09:06:47 +0800 huang ying
wrote:
> >> >> +struct swap_info_struct *get_swap_device(swp_entry_t entry)
> >> >> +{
> >> >> + struct swap_info_struct *si;
> >> >> + unsigned long type, offset;
> >> >> +
> >> >> + if (!entry.val)
> >> >> + goto out;
> >> >> + type = swp_type(entry);
> >> >> + if (type >= nr_swapfiles)
> >> >> + goto bad_nofile;
> >> >> + si = swap_info[type];
> >> >> +
> >> >> + preempt_disable();
> >> >
> >> > This preempt_disable() is later than I'd expect. If a well-timed race
> >> > occurs, `si' could now be pointing at a defunct entry. If that
> >> > well-timed race include a swapoff AND a swapon, `si' could be pointing
> >> > at the info for a new device?
> >>
> >> struct swap_info_struct pointed to by swap_info[] will never be freed.
> >> During swapoff, we only free the memory pointed to by the fields of
> >> struct swap_info_struct. And when swapon, we will always reuse
> >> swap_info[type] if it's not NULL. So it should be safe to dereference
> >> swap_info[type] with preemption enabled.
> >
> > That's my point. If there's a race window during which there is a
> > parallel swapoff+swapon, this swap_info_struct may now be in use for a
> > different device?
>
> Yes. It's possible. And the caller of get_swap_device() can live
> with it if the swap_info_struct has been fully initialized. For
> example, for the race in the patch description,
>
> do_swap_page
> swapin_readahead
> __read_swap_cache_async
> swapcache_prepare
> __swap_duplicate
>
> in __swap_duplicate(), it's possible that the swap device returned by
> get_swap_device() is different from the swap device when
> __swap_duplicate() call get_swap_device(). But the struct_info_struct
> has been fully initialized, so __swap_duplicate() can reference
> si->swap_map[] safely. And we will check si->swap_map[] before any
> further operation. Even if the swap entry is swapped out again for
> the new swap device, we will check the page table again in
> do_swap_page(). So there is no functionality problem.
That's rather revolting. Can we tighten this up? Or at least very
loudly document it?
Re: [PATCH -mm -v5 RESEND] mm, swap: Fix race between swapoff and some swap operations
On Sat, Feb 17, 2018 at 7:38 AM, Andrew Morton
wrote:
> On Wed, 14 Feb 2018 08:38:00 +0800 "Huang\, Ying"
> wrote:
>
>> Andrew Morton writes:
>>
>> > On Tue, 13 Feb 2018 09:42:20 +0800 "Huang, Ying"
>> > wrote:
>> >
>> >> From: Huang Ying
>> >>
>> >> When the swapin is performed, after getting the swap entry information
>> >> from the page table, system will swap in the swap entry, without any
>> >> lock held to prevent the swap device from being swapoff. This may
>> >> cause the race like below,
>> >
>> > Sigh. In terms of putting all the work into the swapoff path and
>> > avoiding overheads in the hot paths, I guess this is about as good as
>> > it will get.
>> >
>> > It's a very low-priority fix so I'd prefer to keep the patch in -mm
>> > until Hugh has had an opportunity to think about it.
>> >
>> >> ...
>> >>
>> >> +/*
>> >> + * Check whether swap entry is valid in the swap device. If so,
>> >> + * return pointer to swap_info_struct, and keep the swap entry valid
>> >> + * via preventing the swap device from being swapoff, until
>> >> + * put_swap_device() is called. Otherwise return NULL.
>> >> + */
>> >> +struct swap_info_struct *get_swap_device(swp_entry_t entry)
>> >> +{
>> >> + struct swap_info_struct *si;
>> >> + unsigned long type, offset;
>> >> +
>> >> + if (!entry.val)
>> >> + goto out;
>> >> + type = swp_type(entry);
>> >> + if (type >= nr_swapfiles)
>> >> + goto bad_nofile;
>> >> + si = swap_info[type];
>> >> +
>> >> + preempt_disable();
>> >
>> > This preempt_disable() is later than I'd expect. If a well-timed race
>> > occurs, `si' could now be pointing at a defunct entry. If that
>> > well-timed race include a swapoff AND a swapon, `si' could be pointing
>> > at the info for a new device?
>>
>> struct swap_info_struct pointed to by swap_info[] will never be freed.
>> During swapoff, we only free the memory pointed to by the fields of
>> struct swap_info_struct. And when swapon, we will always reuse
>> swap_info[type] if it's not NULL. So it should be safe to dereference
>> swap_info[type] with preemption enabled.
>
> That's my point. If there's a race window during which there is a
> parallel swapoff+swapon, this swap_info_struct may now be in use for a
> different device?
Yes. It's possible. And the caller of get_swap_device() can live
with it if the swap_info_struct has been fully initialized. For
example, for the race in the patch description,
do_swap_page
swapin_readahead
__read_swap_cache_async
swapcache_prepare
__swap_duplicate
in __swap_duplicate(), it's possible that the swap device returned by
get_swap_device() is different from the swap device when
__swap_duplicate() call get_swap_device(). But the struct_info_struct
has been fully initialized, so __swap_duplicate() can reference
si->swap_map[] safely. And we will check si->swap_map[] before any
further operation. Even if the swap entry is swapped out again for
the new swap device, we will check the page table again in
do_swap_page(). So there is no functionality problem.
Best Regards,
Huang, Ying
Re: [PATCH -mm -v5 RESEND] mm, swap: Fix race between swapoff and some swap operations
On Wed, 14 Feb 2018 08:38:00 +0800 "Huang\, Ying" wrote:
> Andrew Morton writes:
>
> > On Tue, 13 Feb 2018 09:42:20 +0800 "Huang, Ying"
> > wrote:
> >
> >> From: Huang Ying
> >>
> >> When the swapin is performed, after getting the swap entry information
> >> from the page table, system will swap in the swap entry, without any
> >> lock held to prevent the swap device from being swapoff. This may
> >> cause the race like below,
> >
> > Sigh. In terms of putting all the work into the swapoff path and
> > avoiding overheads in the hot paths, I guess this is about as good as
> > it will get.
> >
> > It's a very low-priority fix so I'd prefer to keep the patch in -mm
> > until Hugh has had an opportunity to think about it.
> >
> >> ...
> >>
> >> +/*
> >> + * Check whether swap entry is valid in the swap device. If so,
> >> + * return pointer to swap_info_struct, and keep the swap entry valid
> >> + * via preventing the swap device from being swapoff, until
> >> + * put_swap_device() is called. Otherwise return NULL.
> >> + */
> >> +struct swap_info_struct *get_swap_device(swp_entry_t entry)
> >> +{
> >> + struct swap_info_struct *si;
> >> + unsigned long type, offset;
> >> +
> >> + if (!entry.val)
> >> + goto out;
> >> + type = swp_type(entry);
> >> + if (type >= nr_swapfiles)
> >> + goto bad_nofile;
> >> + si = swap_info[type];
> >> +
> >> + preempt_disable();
> >
> > This preempt_disable() is later than I'd expect. If a well-timed race
> > occurs, `si' could now be pointing at a defunct entry. If that
> > well-timed race include a swapoff AND a swapon, `si' could be pointing
> > at the info for a new device?
>
> struct swap_info_struct pointed to by swap_info[] will never be freed.
> During swapoff, we only free the memory pointed to by the fields of
> struct swap_info_struct. And when swapon, we will always reuse
> swap_info[type] if it's not NULL. So it should be safe to dereference
> swap_info[type] with preemption enabled.
That's my point. If there's a race window during which there is a
parallel swapoff+swapon, this swap_info_struct may now be in use for a
different device?
Re: [PATCH -mm -v5 RESEND] mm, swap: Fix race between swapoff and some swap operations
Andrew Morton writes:
> On Tue, 13 Feb 2018 09:42:20 +0800 "Huang, Ying" wrote:
>
>> From: Huang Ying
>>
>> When the swapin is performed, after getting the swap entry information
>> from the page table, system will swap in the swap entry, without any
>> lock held to prevent the swap device from being swapoff. This may
>> cause the race like below,
>
> Sigh. In terms of putting all the work into the swapoff path and
> avoiding overheads in the hot paths, I guess this is about as good as
> it will get.
>
> It's a very low-priority fix so I'd prefer to keep the patch in -mm
> until Hugh has had an opportunity to think about it.
>
>> ...
>>
>> +/*
>> + * Check whether swap entry is valid in the swap device. If so,
>> + * return pointer to swap_info_struct, and keep the swap entry valid
>> + * via preventing the swap device from being swapoff, until
>> + * put_swap_device() is called. Otherwise return NULL.
>> + */
>> +struct swap_info_struct *get_swap_device(swp_entry_t entry)
>> +{
>> +struct swap_info_struct *si;
>> +unsigned long type, offset;
>> +
>> +if (!entry.val)
>> +goto out;
>> +type = swp_type(entry);
>> +if (type >= nr_swapfiles)
>> +goto bad_nofile;
>> +si = swap_info[type];
>> +
>> +preempt_disable();
>
> This preempt_disable() is later than I'd expect. If a well-timed race
> occurs, `si' could now be pointing at a defunct entry. If that
> well-timed race include a swapoff AND a swapon, `si' could be pointing
> at the info for a new device?
struct swap_info_struct pointed to by swap_info[] will never be freed.
During swapoff, we only free the memory pointed to by the fields of
struct swap_info_struct. And when swapon, we will always reuse
swap_info[type] if it's not NULL. So it should be safe to dereference
swap_info[type] with preemption enabled.
Best Regards,
Huang, Ying
>> +if (!(si->flags & SWP_VALID))
>> +goto unlock_out;
>> +offset = swp_offset(entry);
>> +if (offset >= si->max)
>> +goto unlock_out;
>> +
>> +return si;
>> +bad_nofile:
>> +pr_err("%s: %s%08lx\n", __func__, Bad_file, entry.val);
>> +out:
>> +return NULL;
>> +unlock_out:
>> +preempt_enable();
>> +return NULL;
>> +}
Re: [PATCH -mm -v5 RESEND] mm, swap: Fix race between swapoff and some swap operations
On Tue, 13 Feb 2018 09:42:20 +0800 "Huang, Ying" wrote:
> From: Huang Ying
>
> When the swapin is performed, after getting the swap entry information
> from the page table, system will swap in the swap entry, without any
> lock held to prevent the swap device from being swapoff. This may
> cause the race like below,
Sigh. In terms of putting all the work into the swapoff path and
avoiding overheads in the hot paths, I guess this is about as good as
it will get.
It's a very low-priority fix so I'd prefer to keep the patch in -mm
until Hugh has had an opportunity to think about it.
> ...
>
> +/*
> + * Check whether swap entry is valid in the swap device. If so,
> + * return pointer to swap_info_struct, and keep the swap entry valid
> + * via preventing the swap device from being swapoff, until
> + * put_swap_device() is called. Otherwise return NULL.
> + */
> +struct swap_info_struct *get_swap_device(swp_entry_t entry)
> +{
> + struct swap_info_struct *si;
> + unsigned long type, offset;
> +
> + if (!entry.val)
> + goto out;
> + type = swp_type(entry);
> + if (type >= nr_swapfiles)
> + goto bad_nofile;
> + si = swap_info[type];
> +
> + preempt_disable();
This preempt_disable() is later than I'd expect. If a well-timed race
occurs, `si' could now be pointing at a defunct entry. If that
well-timed race include a swapoff AND a swapon, `si' could be pointing
at the info for a new device?
> + if (!(si->flags & SWP_VALID))
> + goto unlock_out;
> + offset = swp_offset(entry);
> + if (offset >= si->max)
> + goto unlock_out;
> +
> + return si;
> +bad_nofile:
> + pr_err("%s: %s%08lx\n", __func__, Bad_file, entry.val);
> +out:
> + return NULL;
> +unlock_out:
> + preempt_enable();
> + return NULL;
> +}
