Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Fri, Nov 09, 2018 at 08:12:39PM -0800, Omar Sandoval wrote:
> On Wed, Nov 07, 2018 at 04:28:10PM +0100, David Sterba wrote:
> > On Wed, Nov 07, 2018 at 05:07:00PM +0200, Nikolay Borisov wrote:
> > >
> > >
> > > On 7.11.18 г. 16:49 ч., David Sterba wrote:
> > > > On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote:
> > > >> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
> > > >>> From: Omar Sandoval
> > > >>> This series implements swap file support for Btrfs.
> > > >>>
> > > >>> Changes from v8 [1]:
> > > >>>
> > > >>> - Fixed a bug in btrfs_swap_activate() which would cause us to miss
> > > >>> some
> > > >>> file extents if they were merged into one extent map entry.
> > > >>> - Fixed build for !CONFIG_SWAP.
> > > >>> - Changed all error messages to KERN_WARN.
> > > >>> - Unindented long error messages.
> > > >>>
> > > >>> I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
> > > >>> ack for that one, too.
> > > >>>
> > > >>> Thanks!
> > > >>>
> > > >>> 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
> > > >>>
> > > >>> Omar Sandoval (6):
> > > >>> mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> > > >>> mm: export add_swap_extent()
> > > >>> vfs: update swap_{,de}activate documentation
> > > >>> Btrfs: prevent ioctls from interfering with a swap file
> > > >>> Btrfs: rename get_chunk_map() and make it non-static
> > > >>> Btrfs: support swap files
> > > >>
> > > >> fstest generic/472 reports an assertion failure. This is on the
> > > >> updated fstests
> > > >> git (70c4067285b0bc076), though it should not matter:
> > > >>
> > > >> [16597.002190] assertion failed: IS_ALIGNED(start,
> > > >> fs_info->sectorsize) && IS_ALIGNED(end + 1, fs_info->sectorsize),
> > > >> file: fs/btrfs/file-item.c, line: 319
> > > >
> > > > I have to revert the patch for now as it kills the testing machines.
> > >
> > > The reason is that the isize is not aligned to a sectorsize. Ie it
> > > should be:
> > >
> > > + u64 isize = ALIGN_DOWN(i_size_read(inode), fs_info->sectorsize);
> > >
> > > With this fixlet generic/472 succeeds.
> >
> > Thanks for the fix, I'll fold it in.
>
> Thanks, Nikolay, I missed that. I don't think i_size_read() is
> necessary, though, since the inode is locked.
Indeed, i_size used and patch pushed to misc-next.
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Wed, Nov 07, 2018 at 04:28:10PM +0100, David Sterba wrote:
> On Wed, Nov 07, 2018 at 05:07:00PM +0200, Nikolay Borisov wrote:
> >
> >
> > On 7.11.18 г. 16:49 ч., David Sterba wrote:
> > > On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote:
> > >> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
> > >>> From: Omar Sandoval
> > >>> This series implements swap file support for Btrfs.
> > >>>
> > >>> Changes from v8 [1]:
> > >>>
> > >>> - Fixed a bug in btrfs_swap_activate() which would cause us to miss some
> > >>> file extents if they were merged into one extent map entry.
> > >>> - Fixed build for !CONFIG_SWAP.
> > >>> - Changed all error messages to KERN_WARN.
> > >>> - Unindented long error messages.
> > >>>
> > >>> I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
> > >>> ack for that one, too.
> > >>>
> > >>> Thanks!
> > >>>
> > >>> 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
> > >>>
> > >>> Omar Sandoval (6):
> > >>> mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> > >>> mm: export add_swap_extent()
> > >>> vfs: update swap_{,de}activate documentation
> > >>> Btrfs: prevent ioctls from interfering with a swap file
> > >>> Btrfs: rename get_chunk_map() and make it non-static
> > >>> Btrfs: support swap files
> > >>
> > >> fstest generic/472 reports an assertion failure. This is on the updated
> > >> fstests
> > >> git (70c4067285b0bc076), though it should not matter:
> > >>
> > >> [16597.002190] assertion failed: IS_ALIGNED(start, fs_info->sectorsize)
> > >> && IS_ALIGNED(end + 1, fs_info->sectorsize), file: fs/btrfs/file-item.c,
> > >> line: 319
> > >
> > > I have to revert the patch for now as it kills the testing machines.
> >
> > The reason is that the isize is not aligned to a sectorsize. Ie it
> > should be:
> >
> > + u64 isize = ALIGN_DOWN(i_size_read(inode), fs_info->sectorsize);
> >
> > With this fixlet generic/472 succeeds.
>
> Thanks for the fix, I'll fold it in.
Thanks, Nikolay, I missed that. I don't think i_size_read() is
necessary, though, since the inode is locked.
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Wed, Nov 07, 2018 at 05:07:00PM +0200, Nikolay Borisov wrote:
>
>
> On 7.11.18 г. 16:49 ч., David Sterba wrote:
> > On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote:
> >> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
> >>> From: Omar Sandoval
> >>> This series implements swap file support for Btrfs.
> >>>
> >>> Changes from v8 [1]:
> >>>
> >>> - Fixed a bug in btrfs_swap_activate() which would cause us to miss some
> >>> file extents if they were merged into one extent map entry.
> >>> - Fixed build for !CONFIG_SWAP.
> >>> - Changed all error messages to KERN_WARN.
> >>> - Unindented long error messages.
> >>>
> >>> I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
> >>> ack for that one, too.
> >>>
> >>> Thanks!
> >>>
> >>> 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
> >>>
> >>> Omar Sandoval (6):
> >>> mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> >>> mm: export add_swap_extent()
> >>> vfs: update swap_{,de}activate documentation
> >>> Btrfs: prevent ioctls from interfering with a swap file
> >>> Btrfs: rename get_chunk_map() and make it non-static
> >>> Btrfs: support swap files
> >>
> >> fstest generic/472 reports an assertion failure. This is on the updated
> >> fstests
> >> git (70c4067285b0bc076), though it should not matter:
> >>
> >> [16597.002190] assertion failed: IS_ALIGNED(start, fs_info->sectorsize) &&
> >> IS_ALIGNED(end + 1, fs_info->sectorsize), file: fs/btrfs/file-item.c,
> >> line: 319
> >
> > I have to revert the patch for now as it kills the testing machines.
>
> The reason is that the isize is not aligned to a sectorsize. Ie it
> should be:
>
> + u64 isize = ALIGN_DOWN(i_size_read(inode), fs_info->sectorsize);
>
> With this fixlet generic/472 succeeds.
Thanks for the fix, I'll fold it in.
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On 7.11.18 г. 16:49 ч., David Sterba wrote:
> On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote:
>> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
>>> From: Omar Sandoval
>>> This series implements swap file support for Btrfs.
>>>
>>> Changes from v8 [1]:
>>>
>>> - Fixed a bug in btrfs_swap_activate() which would cause us to miss some
>>> file extents if they were merged into one extent map entry.
>>> - Fixed build for !CONFIG_SWAP.
>>> - Changed all error messages to KERN_WARN.
>>> - Unindented long error messages.
>>>
>>> I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
>>> ack for that one, too.
>>>
>>> Thanks!
>>>
>>> 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
>>>
>>> Omar Sandoval (6):
>>> mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
>>> mm: export add_swap_extent()
>>> vfs: update swap_{,de}activate documentation
>>> Btrfs: prevent ioctls from interfering with a swap file
>>> Btrfs: rename get_chunk_map() and make it non-static
>>> Btrfs: support swap files
>>
>> fstest generic/472 reports an assertion failure. This is on the updated
>> fstests
>> git (70c4067285b0bc076), though it should not matter:
>>
>> [16597.002190] assertion failed: IS_ALIGNED(start, fs_info->sectorsize) &&
>> IS_ALIGNED(end + 1, fs_info->sectorsize), file: fs/btrfs/file-item.c, line:
>> 319
>
> I have to revert the patch for now as it kills the testing machines.
The reason is that the isize is not aligned to a sectorsize. Ie it
should be:
+ u64 isize = ALIGN_DOWN(i_size_read(inode), fs_info->sectorsize);
With this fixlet generic/472 succeeds.
>
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Tue, Nov 06, 2018 at 10:54:51AM +0100, David Sterba wrote:
> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
> > From: Omar Sandoval
> > This series implements swap file support for Btrfs.
> >
> > Changes from v8 [1]:
> >
> > - Fixed a bug in btrfs_swap_activate() which would cause us to miss some
> > file extents if they were merged into one extent map entry.
> > - Fixed build for !CONFIG_SWAP.
> > - Changed all error messages to KERN_WARN.
> > - Unindented long error messages.
> >
> > I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
> > ack for that one, too.
> >
> > Thanks!
> >
> > 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
> >
> > Omar Sandoval (6):
> > mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> > mm: export add_swap_extent()
> > vfs: update swap_{,de}activate documentation
> > Btrfs: prevent ioctls from interfering with a swap file
> > Btrfs: rename get_chunk_map() and make it non-static
> > Btrfs: support swap files
>
> fstest generic/472 reports an assertion failure. This is on the updated
> fstests
> git (70c4067285b0bc076), though it should not matter:
>
> [16597.002190] assertion failed: IS_ALIGNED(start, fs_info->sectorsize) &&
> IS_ALIGNED(end + 1, fs_info->sectorsize), file: fs/btrfs/file-item.c, line:
> 319
I have to revert the patch for now as it kills the testing machines.
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
> This series implements swap file support for Btrfs.
>
> Changes from v8 [1]:
>
> - Fixed a bug in btrfs_swap_activate() which would cause us to miss some
> file extents if they were merged into one extent map entry.
> - Fixed build for !CONFIG_SWAP.
> - Changed all error messages to KERN_WARN.
> - Unindented long error messages.
>
> I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
> ack for that one, too.
>
> Thanks!
>
> 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
>
> Omar Sandoval (6):
> mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> mm: export add_swap_extent()
> vfs: update swap_{,de}activate documentation
> Btrfs: prevent ioctls from interfering with a swap file
> Btrfs: rename get_chunk_map() and make it non-static
> Btrfs: support swap files
fstest generic/472 reports an assertion failure. This is on the updated fstests
git (70c4067285b0bc076), though it should not matter:
[16597.002190] assertion failed: IS_ALIGNED(start, fs_info->sectorsize) &&
IS_ALIGNED(end + 1, fs_info->sectorsize), file: fs/btrfs/file-item.c, line: 319
[16597.016154] [ cut here ]
[16597.020911] kernel BUG at fs/btrfs/ctree.h:3448!
[16597.025692] invalid opcode: [#1] PREEMPT SMP
[16597.030540] CPU: 1 PID: 2216 Comm: swapon Tainted: GW
4.19.0-1.ge195904-vanilla+ #343
[16597.030542] Hardware name: empty empty/S3993, BIOS PAQEX0-3 02/24/2008
[16597.030617] RIP: 0010:btrfs_lookup_csums_range+0x3ee/0x6e0 [btrfs]
[16597.030623] RSP: 0018:959a8210fc00 EFLAGS: 00010286
[16597.030626] RAX: 008b RBX: 959a8210fc40 RCX:
[16597.030628] RDX: RSI: 89c766fd60a8 RDI: 89c766fd60a8
[16597.030630] RBP: 00f1 R08: 0001 R09:
[16597.030631] R10: R11: R12:
[16597.030633] R13: 89c74f47c000 R14: R15: 3e4f
[16597.030635] FS: 7f3d80a18700() GS:89c766e0()
knlGS:
[16597.030637] CS: 0010 DS: ES: CR0: 80050033
[16597.030639] CR2: 7f3d7fa733b0 CR3: 00021e0b5000 CR4: 06e0
[16597.030640] Call Trace:
[16597.030657] ? __mutex_unlock_slowpath+0x4b/0x2b0
[16597.142625] ? btrfs_cross_ref_exist+0x6d/0xa0 [btrfs]
[16597.142674] csum_exist_in_range+0x43/0xc2 [btrfs]
[16597.152938] can_nocow_extent+0x442/0x4a0 [btrfs]
[16597.152993] btrfs_swap_activate+0x19f/0x680 [btrfs]
[16597.162947] __do_sys_swapon+0xdf8/0x1550
[16597.162965] do_syscall_64+0x5c/0x1b0
[16597.162971] entry_SYSCALL_64_after_hwframe+0x49/0xbe
[16597.162974] RIP: 0033:0x7f3d7fb1f067
[16597.198875] RSP: 002b:7ffcf9c49808 EFLAGS: 0202 ORIG_RAX:
00a7
[16597.198877] RAX: ffda RBX: RCX: 7f3d7fb1f067
[16597.198879] RDX: 7ffcf9c49910 RSI: RDI: 7ffcf9c4a8f0
[16597.198884] RBP: R08: 7f3d7fdd8c40 R09: 7f3d8081dfc0
[16597.228520] R10: 1cc0c4fd R11: 0202 R12: 004006a0
[16597.228522] R13: 7ffcf9c498f0 R14: R15:
[16597.311530] ---[ end trace f4dc8cabefc3e36d ]---
[16597.316318] RIP: 0010:btrfs_lookup_csums_range+0x3ee/0x6e0 [btrfs]
[16597.341828] RSP: 0018:959a8210fc00 EFLAGS: 00010286
[16597.341834] RAX: 008b RBX: 959a8210fc40 RCX:
[16597.354588] RDX: RSI: 89c766fd60a8 RDI: 89c766fd60a8
[16597.354590] RBP: 00f1 R08: 0001 R09:
[16597.354592] R10: R11: R12:
[16597.354593] R13: 89c74f47c000 R14: R15: 3e4f
[16597.354595] FS: 7f3d80a18700() GS:89c766e0()
knlGS:
[16597.354596] CS: 0010 DS: ES: CR0: 80050033
[16597.354598] CR2: 7f3d7fa733b0 CR3: 00021e0b5000 CR4: 06e0
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Mon, Oct 22, 2018 at 02:13:27PM -0700, Omar Sandoval wrote:
> > > Omar Sandoval (6):
> > > mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> > > mm: export add_swap_extent()
> > > vfs: update swap_{,de}activate documentation
> > > Btrfs: prevent ioctls from interfering with a swap file
> > > Btrfs: rename get_chunk_map() and make it non-static
> > > Btrfs: support swap files
> >
> > Patches 1 and 2 now going through Andrew's tree, the btrfs part will be
> > delayed and not merged to 4.20. This is a bit unfortuante, I was busy
> > with the non-feature patches and other things, sorry.
>
> That's perfectly fine with me, than
The 3 btrfs patches are now in misc-next.
Setting up the swap file needs the extra steps to make sure it's a NOCOW
file and preallocated, that's just 2 more commands using common tools.
The surprise may come on an a multi-device filesystem when the chunks
get spread over more devices and the user has no control over that.
Reducing the file size until it fits is a workaround, tough not totally
reliable.
I've explored the error cases (balance, dev delete, adding more
swapfiles). Also a stress test (make -j on kernel). The OOM killer was
able to get the system back after each round after which I added one
more swapfile, until the system was effectively dead.
So the stability seems to be ok, we will need to document the usecase,
constraints and how to properly set up the swapfile, and that's about
it. Thanks.
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Fri, Oct 19, 2018 at 05:43:18PM +0200, David Sterba wrote:
> On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
> > From: Omar Sandoval
> > This series implements swap file support for Btrfs.
> >
> > Changes from v8 [1]:
> >
> > - Fixed a bug in btrfs_swap_activate() which would cause us to miss some
> > file extents if they were merged into one extent map entry.
> > - Fixed build for !CONFIG_SWAP.
> > - Changed all error messages to KERN_WARN.
> > - Unindented long error messages.
> >
> > I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
> > ack for that one, too.
> >
> > Thanks!
> >
> > 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
> >
> > Omar Sandoval (6):
> > mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> > mm: export add_swap_extent()
> > vfs: update swap_{,de}activate documentation
> > Btrfs: prevent ioctls from interfering with a swap file
> > Btrfs: rename get_chunk_map() and make it non-static
> > Btrfs: support swap files
>
> Patches 1 and 2 now going through Andrew's tree, the btrfs part will be
> delayed and not merged to 4.20. This is a bit unfortuante, I was busy
> with the non-feature patches and other things, sorry.
That's perfectly fine with me, thanks, Dave!
Re: [PATCH v9 0/6] Btrfs: implement swap file support
On Thu, Sep 27, 2018 at 11:17:32AM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
> This series implements swap file support for Btrfs.
>
> Changes from v8 [1]:
>
> - Fixed a bug in btrfs_swap_activate() which would cause us to miss some
> file extents if they were merged into one extent map entry.
> - Fixed build for !CONFIG_SWAP.
> - Changed all error messages to KERN_WARN.
> - Unindented long error messages.
>
> I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
> ack for that one, too.
>
> Thanks!
>
> 1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
>
> Omar Sandoval (6):
> mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
> mm: export add_swap_extent()
> vfs: update swap_{,de}activate documentation
> Btrfs: prevent ioctls from interfering with a swap file
> Btrfs: rename get_chunk_map() and make it non-static
> Btrfs: support swap files
Patches 1 and 2 now going through Andrew's tree, the btrfs part will be
delayed and not merged to 4.20. This is a bit unfortuante, I was busy
with the non-feature patches and other things, sorry.
[PATCH v9 0/6] Btrfs: implement swap file support
From: Omar Sandoval
Hi,
This series implements swap file support for Btrfs.
Changes from v8 [1]:
- Fixed a bug in btrfs_swap_activate() which would cause us to miss some
file extents if they were merged into one extent map entry.
- Fixed build for !CONFIG_SWAP.
- Changed all error messages to KERN_WARN.
- Unindented long error messages.
I've Cc'd Jon and Al on patch 3 this time, so hopefully we can get an
ack for that one, too.
Thanks!
1: https://www.spinics.net/lists/linux-btrfs/msg82267.html
Omar Sandoval (6):
mm: split SWP_FILE into SWP_ACTIVATED and SWP_FS
mm: export add_swap_extent()
vfs: update swap_{,de}activate documentation
Btrfs: prevent ioctls from interfering with a swap file
Btrfs: rename get_chunk_map() and make it non-static
Btrfs: support swap files
Documentation/filesystems/Locking | 17 +-
Documentation/filesystems/vfs.txt | 12 +-
fs/btrfs/ctree.h | 29 +++
fs/btrfs/dev-replace.c| 8 +
fs/btrfs/disk-io.c| 4 +
fs/btrfs/inode.c | 338 ++
fs/btrfs/ioctl.c | 31 ++-
fs/btrfs/relocation.c | 18 +-
fs/btrfs/volumes.c| 82 ++--
fs/btrfs/volumes.h| 2 +
include/linux/swap.h | 13 +-
mm/page_io.c | 6 +-
mm/swapfile.c | 14 +-
13 files changed, 523 insertions(+), 51 deletions(-)
--
2.19.0
