[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-27 Thread Launchpad Bug Tracker
This bug was fixed in the package zfs-linux - 0.6.5.6-0ubuntu28 --- zfs-linux (0.6.5.6-0ubuntu28) xenial; urgency=medium * Fix ZFS shrinker deadlock with xattrs (LP: #1839521) - Upstream ZFS fix 31b6111fd92a ("Kill zp->z_xattr_parent to prevent pinning") and ddae16a9cf0b

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-19 Thread Mauricio Faria de Oliveira
Verification done for linux on xenial-proposed. The inodes for file, xattr dir, and xattr child are all evicted at file removal time, not making it to any disposal list after file removal. So the window/scenario for the problem to occur is not present anymore. Log --- $ uname -rv 4.4.0-160-gene

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-14 Thread Mauricio Faria de Oliveira
Hi @mathew-hodson, This fix is needed in the Linux kernel package for Xenial as well (it duplicates the zfs-linux/dkms source). Shouldn't the 'linux' source package still be tracked? Thanks, Mauricio -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscr

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-13 Thread Mathew Hodson
** No longer affects: linux (Ubuntu) ** No longer affects: linux (Ubuntu Xenial) ** No longer affects: linux (Ubuntu Bionic) ** No longer affects: linux (Ubuntu Disco) ** No longer affects: linux (Ubuntu Eoan) ** No longer affects: zfs-linux (Ubuntu Bionic) ** No longer affects: zfs-linux (Ub

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-13 Thread Mauricio Faria de Oliveira
Verification done for zfs-linux on xenial-proposed. The inodes for file, xattr dir, and xattr child are all evicted at file removal time, not making it to any disposal list after file removal. So the window/scenario for the problem to occur is not present anymore. Log --- $ echo 'deb http://arc

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-13 Thread Brian Murray
Hello Mauricio, or anyone else affected, Accepted zfs-linux into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/zfs- linux/0.6.5.6-0ubuntu28 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-12 Thread Khaled El Mously
** Changed in: linux (Ubuntu Xenial) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1839521 Title: Xenial: ZFS deadlock in shrinker path with xattrs To ma

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-09 Thread Mauricio Faria de Oliveira
Colin, thanks for sponsoring and additional testing! Is this ZFS regression test suite available to other people? It sounds great for future work with ZFS. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bug

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-09 Thread Colin Ian King
Thanks Mauricio, I've sponsored this package, and sent the kernel related changes to the kernel-team mailing list: https://lists.ubuntu.com/archives/kernel-team/2019-August/102938.html I also sanity checked the code with the ubuntu kernel team ZFS regression tests and they all pass successfully.

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
** Description changed: [Impact] - * Xenial's ZFS can deadlock in the memory shrinker path -after removing files with extended attributes (xattr). +  * Xenial's ZFS can deadlock in the memory shrinker path +    after removing files with extended attributes (xattr). - * Extended attri

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
Fix Change Details -- (Compare with comment #3) The xattr dir/child inodes are evicted/disposed immediately on file removal, not on shrinker. $ touch /zfs/file <...>-23940 [006] d... 10808.746798: p_zfs_mknode_0: (zfs_mknode+0x0/0xe00 [zfs]) flag=0x0 dzp=0x88011a

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
debdiff for zfs-linux (zfs-linux_shrinker-deadlock-xattr.debdiff) based on ubuntu/xenial-devel branch from LP source pkg git repo [1]. [1] https://git.launchpad.net/ubuntu/+source/zfs-linux ** Patch added: "zfs-linux_shrinker-deadlock-xattr.debdiff" https://bugs.launchpad.net/ubuntu/+source/z

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
zfs-test results w/ original & modified zfs-dkms original $ uname -rv 4.4.0-159-generic #187-Ubuntu SMP Thu Aug 1 16:28:06 UTC 2019 $ modinfo zfs | head filename: /lib/modules/4.4.0-159-generic/updates/dkms/zfs.ko version:0.

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
[Original Description] One LXC user reported lots of processes stuck in D state: several threads waiting in the memory shrinker semaphore (this symptom was thought to be/fixed via LP bug 1817628). After some time, a provided crashdump revealed the issue in ZFS's evict node path running in memory

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
Problem Details: --- Helper scripts (create /zfs.img and mount it on /zfs; and setup kprobe events for debug) $ sudo ./zfs-mount.sh $ sudo ./zfs-kprobes.sh Print kprobe events to screen as we go: $ sudo cat /sys/kernel/debug/tracing/trace_pipe & Create file: - allocates normal/fil

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
** Attachment added: "zfs-mount.sh" https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1839521/+attachment/5281632/+files/zfs-mount.sh -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1839521 T

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
** Attachment added: "zfs-kprobes.sh" https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1839521/+attachment/5281633/+files/zfs-kprobes.sh ** Attachment removed: "kmod-zget.c" https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1839521/+attachment/5281631/+files/kmod-zget.c --

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
zfs-test test suite on xenial - Build/install the latest zfs-test suite from zfs.git: $ git clone https://github.com/zfsonlinux/zfs.git $ cd zfs $ git log --oneline -1 c81f179 Metaslab max_size should be persisted while unloaded $ sudo apt-get build-dep zfs-linux $ su

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
Fix Details --- Problem report (GH Issue) upstream in ZFS on Linux: https://github.com/zfsonlinux/zfs/issues/4816 ("Looping in zfs_zget()") With very similar stack trace (syscall -> shrink -> zfs_{inactive/rmnode/purgedir/zget}) https://github.com/zfsonlinux/zfs/issues/48

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
Problem Summary: --- If an xattr directory inode and its xattr child inode are on the _same_ disposal list, and the xattr directory inode is _before_ its xattr child inode in this disposal list... Then zfs_purgedir() of the xattr directory calls zfs_zget() for the xattr child inode

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
** Attachment added: "kmod-zget.c" https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1839521/+attachment/5281634/+files/kmod-zget.c -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1839521 Tit

[Bug 1839521] Re: Xenial: ZFS deadlock in shrinker path with xattrs

2019-08-08 Thread Mauricio Faria de Oliveira
Test-case for the problem (kmod-zget.c). Header comments with steps to reproduce. ** Attachment added: "kmod-zget.c" https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1839521/+attachment/5281631/+files/kmod-zget.c -- You received this bug notification because you are a member of Ubunt