[Kernel-packages] [Bug 1637619] Re: ext4dist strange timings on ubuntu 16.04
The fix for this has really been released this time. Confirmed fixed in 4.4.0-51.72. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1637619 Title: ext4dist strange timings on ubuntu 16.04 Status in Linux: New Status in linux package in Ubuntu: Fix Released Bug description: Reference: https://github.com/iovisor/bcc/issues/790 "The bug is: timekeeping: Avoid taking lock in NMI path with CONFIG_DEBUG_TIMEKEEPING The fix is: timekeeping: Fix __ktime_get_fast_ns() regression Looks like that kernel version has the bug but not the fix. http://changelogs.ubuntu.com/changelogs/pool/main/l/linux/linux_4.4.0-45.66/changelog; Apparently this is fixed by commit: commit 58bfea9532552d422bde7afa207e1a0f08dffa7d Author: John Stultz john.stu...@linaro.org Date: Tue Oct 4 19:55:48 2016 -0700 timekeeping: Fix __ktime_get_fast_ns() regression To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1637619/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1637619] Re: ext4dist strange timings on ubuntu 16.04
Actually, I believe it has not been released yet. The fix is in the changelog for 4.4.0-48.69, but the latest release of the xenial LTS kernel is 4.4.0-47.68 where the bug can still be reproduced. Specifically: * Xenial update to v4.4.25 stable release (LP: #1634153) - timekeeping: Fix __ktime_get_fast_ns() regression -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1637619 Title: ext4dist strange timings on ubuntu 16.04 Status in Linux: New Status in linux package in Ubuntu: Fix Released Bug description: Reference: https://github.com/iovisor/bcc/issues/790 "The bug is: timekeeping: Avoid taking lock in NMI path with CONFIG_DEBUG_TIMEKEEPING The fix is: timekeeping: Fix __ktime_get_fast_ns() regression Looks like that kernel version has the bug but not the fix. http://changelogs.ubuntu.com/changelogs/pool/main/l/linux/linux_4.4.0-45.66/changelog; Apparently this is fixed by commit: commit 58bfea9532552d422bde7afa207e1a0f08dffa7d Author: John Stultz john.stu...@linaro.org Date: Tue Oct 4 19:55:48 2016 -0700 timekeeping: Fix __ktime_get_fast_ns() regression To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1637619/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1637619] Re: ext4dist strange timings on ubuntu 16.04
Awesome, thanks Colin! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1637619 Title: ext4dist strange timings on ubuntu 16.04 Status in Linux: New Status in linux package in Ubuntu: Fix Released Bug description: Reference: https://github.com/iovisor/bcc/issues/790 "The bug is: timekeeping: Avoid taking lock in NMI path with CONFIG_DEBUG_TIMEKEEPING The fix is: timekeeping: Fix __ktime_get_fast_ns() regression Looks like that kernel version has the bug but not the fix. http://changelogs.ubuntu.com/changelogs/pool/main/l/linux/linux_4.4.0-45.66/changelog; Apparently this is fixed by commit: commit 58bfea9532552d422bde7afa207e1a0f08dffa7d Author: John Stultz john.stu...@linaro.org Date: Tue Oct 4 19:55:48 2016 -0700 timekeeping: Fix __ktime_get_fast_ns() regression To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1637619/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1637619] Re: ext4dist strange timings on ubuntu 16.04
It's been a few weeks - is there any update when this fix will be published? Thanks! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1637619 Title: ext4dist strange timings on ubuntu 16.04 Status in Linux: New Status in linux package in Ubuntu: Fix Committed Bug description: Reference: https://github.com/iovisor/bcc/issues/790 "The bug is: timekeeping: Avoid taking lock in NMI path with CONFIG_DEBUG_TIMEKEEPING The fix is: timekeeping: Fix __ktime_get_fast_ns() regression Looks like that kernel version has the bug but not the fix. http://changelogs.ubuntu.com/changelogs/pool/main/l/linux/linux_4.4.0-45.66/changelog; Apparently this is fixed by commit: commit 58bfea9532552d422bde7afa207e1a0f08dffa7d Author: John Stultz john.stu...@linaro.org Date: Tue Oct 4 19:55:48 2016 -0700 timekeeping: Fix __ktime_get_fast_ns() regression To manage notifications about this bug go to: https://bugs.launchpad.net/linux/+bug/1637619/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1607920] Re: zfs services fail on firstboot if zfs-utils is integrated into the deployment image
Thanks Eric, we will remove our local workaround. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1607920 Title: zfs services fail on firstboot if zfs-utils is integrated into the deployment image Status in sysvinit package in Ubuntu: Won't Fix Status in zfs-linux package in Ubuntu: Fix Released Status in sysvinit source package in Xenial: Won't Fix Status in zfs-linux source package in Xenial: Fix Released Bug description: [Impact] * zfs services fail on firstboot if zfs-utils is integrated into the deployment image. * Output from systemd - sudo systemctl --failed UNIT LOAD ACTIVE SUB DESCRIPTION ● zfs-import-scan.service loaded failed failed Import ZFS pools by device scanning ● zfs-mount.service loaded failed failed Mount ZFS filesystems * This is particularly frustrating for users who use automated monitoring as it means virtual machines must always be restarted before showing as clean. * This failure is due to zfs services starting up before /etc/mtab has a chance to be symlinked to /proc/mounts. [Test Case] 1. Grab a stock xenial image, and unpack it and add zfs-utils to it. Repack it. 2. Boot machine 3. Check systemctl --failed. [Regression Potential] * none expected, patch has been intensively tested by the upsteam zfs test script suite. * This is a upstream commit merge in 0.7.0. * A ubuntu package has been tested (including the upstream commit) by a user of the community facing this bug, and confirmed it addresses the problem (see comment #7). [Other Info] * The "reading" is redirected to /proc/self/mounts. The writing to /etc/mtab. Some distros still need that. The current hope is to replace the writing (and maybe reading) with libmount, in a second phase. * Upstream commit : https://github.com/zfsonlinux/zfs/commit/792517389fad5c495a2738b61c2e9c65dedaaa9a * Upstream bug: https://github.com/zfsonlinux/zfs/issues/4680 * Debian bug : https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=839071 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1607920/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1607920] Re: zfs services fail on firstboot if zfs-utils is integrated into the deployment image
I tested again with xenial-proposed and confirmed that everything looks good as far as systemd no longer failing ZFS units due to no /etc/mtab. ii libzfs2linux 0.6.5.6-0ubuntu14 amd64 Native OpenZFS filesystem library for Linux ii zfs-dkms 0.6.5.6-0ubuntu14 amd64 Native OpenZFS filesystem kernel modules for Linux ii zfs-doc 0.6.5.6-0ubuntu14 all Native OpenZFS filesystem documentation and examples. ii zfs-initramfs0.6.5.6-0ubuntu14 all Native OpenZFS root filesystem capabilities for Linux ii zfs-zed 0.6.5.6-0ubuntu14 amd64 OpenZFS Event Daemon (zed) ii zfsutils-linux 0.6.5.6-0ubuntu14 amd64 Native OpenZFS management utilities for Linux # systemctl --failed 0 loaded units listed. [...] -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1607920 Title: zfs services fail on firstboot if zfs-utils is integrated into the deployment image Status in sysvinit package in Ubuntu: Won't Fix Status in zfs-linux package in Ubuntu: Fix Released Status in sysvinit source package in Xenial: Won't Fix Status in zfs-linux source package in Xenial: Fix Committed Bug description: [Impact] * zfs services fail on firstboot if zfs-utils is integrated into the deployment image. * Output from systemd - sudo systemctl --failed UNIT LOAD ACTIVE SUB DESCRIPTION ● zfs-import-scan.service loaded failed failed Import ZFS pools by device scanning ● zfs-mount.service loaded failed failed Mount ZFS filesystems * This is particularly frustrating for users who use automated monitoring as it means virtual machines must always be restarted before showing as clean. * This failure is due to zfs services starting up before /etc/mtab has a chance to be symlinked to /proc/mounts. [Test Case] 1. Grab a stock xenial image, and unpack it and add zfs-utils to it. Repack it. 2. Boot machine 3. Check systemctl --failed. [Regression Potential] * none expected, patch has been intensively tested by the upsteam zfs test script suite. * This is a upstream commit merge in 0.7.0. * A ubuntu package has been tested (including the upstream commit) by a user of the community facing this bug, and confirmed it addresses the problem (see comment #7). [Other Info] * The "reading" is redirected to /proc/self/mounts. The writing to /etc/mtab. Some distros still need that. The current hope is to replace the writing (and maybe reading) with libmount, in a second phase. * Upstream commit : https://github.com/zfsonlinux/zfs/commit/792517389fad5c495a2738b61c2e9c65dedaaa9a * Upstream bug: https://github.com/zfsonlinux/zfs/issues/4680 * Debian bug : https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=839071 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1607920/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1607920] Re: zfs services fail on firstboot if zfs-utils is integrated into the deployment image
I tested this using my reproduction method in AWS EC2 with a test package/PPA provided by Eric@Ubuntu (slashd) containing the commit referenced in comment #6. All looks good now - systemd units for ZFS no longer fail. Thanks for moving this along! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1607920 Title: zfs services fail on firstboot if zfs-utils is integrated into the deployment image Status in sysvinit package in Ubuntu: Won't Fix Status in zfs-linux package in Ubuntu: In Progress Bug description: [Impact] * zfs services fail on firstboot if zfs-utils is integrated into the deployment image. * Output from systemd - sudo systemctl --failed UNIT LOAD ACTIVE SUB DESCRIPTION ● zfs-import-scan.service loaded failed failed Import ZFS pools by device scanning ● zfs-mount.service loaded failed failed Mount ZFS filesystems * This is particularly frustrating for users who use automated monitoring as it means virtual machines must always be restarted before showing as clean. * This failure is due to zfs services starting up before /etc/mtab has a chance to be symlinked to /proc/mounts. [Test Case] 1. Grab a stock xenial image, and unpack it and add zfs-utils to it. Repack it. 2. Boot machine 3. Check systemctl --failed. [Regression Potential] * [Other Info] * This can likely be resolved in the systemd init scripts, by modifying zfs-linux to depend on /proc/mounts instead, or inclusion of /lib/init/mount-functions.sh in initscripts (sysvinit). To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/sysvinit/+bug/1607920/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1574342] Re: Ship arcstat.py and arc_summary.py with zfsutils-linux
@slashd Sorry for not being more clear - I already tested from proposed (see comment #22). As far as I can tell, it hasn't changed since then but it is unclear to me if the renaming from .py from upstream will happen separately or as part of this issue. I'm fine with it either way, it just helps in knowing the plan. Thanks. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1574342 Title: Ship arcstat.py and arc_summary.py with zfsutils-linux Status in zfs-linux package in Ubuntu: Fix Released Status in zfs-linux source package in Xenial: Fix Committed Status in zfs-linux source package in Yakkety: Fix Released Bug description: [Impact] zfsutils-linux missing some important tools. Python3 compatible scripts : arc_summary.py, arcstat.py and dbufstat.py found in src_code/cmd/ are not taken into account during compilation. * arc_summary.py : Provides a summary of the statistics https://github.com/zfsonlinux/zfs/blob/master/cmd/arc_summary * arcstat.py : Print out ZFS ARC Statistics exported via kstat(1) https://github.com/zfsonlinux/zfs/tree/master/cmd/arcstat * dbufstat.py: Print out statistics for all cached dmu buffers https://github.com/zfsonlinux/zfs/blob/master/cmd/dbufstat [Test Case] # rmadison zfsutils-linux ... zfsutils-linux | 0.6.5.6-0ubuntu10 | xenial-updates ... # List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.6-0ubuntu10_amd64.deb | egrep "arcstat|arc_summary|dbufstat" # You will notice there are not present. [Regression Potential] None expected, debian package[1] has already made the change to build the python scripts. [1] Package: zfsutils-linux Source: zfs-linux Version: 0.6.5.7-1 Architecture: amd64 Maintainer: Debian ZFS on Linux maintainers# List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.7-1_amd64.deb | egrep -i "arc|dbufstat" -rwxr-xr-x root/root 40751 2016-05-31 03:40 ./bin/arc_summary.py -rwxr-xr-x root/root 13057 2016-05-31 03:40 ./bin/arcstat.py -rwxr-xr-x root/root 18718 2016-05-31 03:40 ./bin/dbufstat.py [Other Info] * This is based on the following Debian commits : Debian git commits === commit 730989397063955d60a1845003cfb7433eba1e52 Author: Aron Xu Date: Tue Apr 26 17:01:29 2016 +0800 Add dh-python to b-d -- commit 814413118fb88de3ce596a87a8a011f50c225fdd Author: Aron Xu Date: Sun Nov 22 15:44:16 2015 +0800 Add python related dependencies === * Despite what Debian did by adding theses script under /bin. I'll add them under /usr/sbin, which is more appropriate. * Unfortunately manpage for theses 3 program seems limited. There is none in Debian nor git upstream repo. The only manpage I found is for arcstat online : http://linux.die.net/man/1/arcstat [Original Description] arcstat.py and arc_summary.py are valuable tools to determine ZFS' ARC usage, it is not obvious why they are not included in zfsutils-linux. As ubuntu-minimal already depends on python3 it should be safe to assume python is available, or am I mistaken here? arcstat.py gives an iostat-like overview about ARC reads, hit rate, current and target size in regular intervals: # ./arcstat.py 1 time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c 19:22:33 0 0 0 00 00 00 2.0G 7.8G 19:22:34 3 0 0 00 00 00 2.0G 7.8G 19:22:3521 0 0 00 00 00 2.0G 7.8G ^C arc_summary.py shows a more detailed overview of the current ARC status and ZFS tunables: # ./arc_summary.py ZFS Subsystem Report Sun Apr 24 19:23:25 2016 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 46 Mutex Misses:0 Evict Skips: 0 ARC Size: 25.10% 1.95GiB Target Size: (Adaptive) 100.00% 7.78GiB Min Size (Hard Limit): 0.40% 32.00 MiB Max Size (High Water): 248:1 7.78GiB ARC Size Breakdown: Recently Used Cache Size:50.00% 3.89GiB Frequently Used Cache Size: 50.00% 3.89GiB ARC Hash Breakdown: Elements Max:32.31k Elements Current:99.78% 32.24k Collisions: 40.54k Chain Max: 3 Chains: 240 ARC Total accesses: 4.54m Cache Hit Ratio: 99.39% 4.51m Cache
[Kernel-packages] [Bug 1574342] Re: Ship arcstat.py and arc_summary.py with zfsutils-linux
OOps, per my last comment regarding tracking issue upstream - that's the Debian bug you previously mentioned, https://bugs.debian.org/cgi- bin/bugreport.cgi?bug=832938, not further upstream from ZoL. Sorry for my misunderstanding. :-/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1574342 Title: Ship arcstat.py and arc_summary.py with zfsutils-linux Status in zfs-linux package in Ubuntu: Fix Released Status in zfs-linux source package in Xenial: Fix Committed Status in zfs-linux source package in Yakkety: Fix Released Bug description: [Impact] zfsutils-linux missing some important tools. Python3 compatible scripts : arc_summary.py, arcstat.py and dbufstat.py found in src_code/cmd/ are not taken into account during compilation. * arc_summary.py : Provides a summary of the statistics https://github.com/zfsonlinux/zfs/blob/master/cmd/arc_summary * arcstat.py : Print out ZFS ARC Statistics exported via kstat(1) https://github.com/zfsonlinux/zfs/tree/master/cmd/arcstat * dbufstat.py: Print out statistics for all cached dmu buffers https://github.com/zfsonlinux/zfs/blob/master/cmd/dbufstat [Test Case] # rmadison zfsutils-linux ... zfsutils-linux | 0.6.5.6-0ubuntu10 | xenial-updates ... # List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.6-0ubuntu10_amd64.deb | egrep "arcstat|arc_summary|dbufstat" # You will notice there are not present. [Regression Potential] None expected, debian package[1] has already made the change to build the python scripts. [1] Package: zfsutils-linux Source: zfs-linux Version: 0.6.5.7-1 Architecture: amd64 Maintainer: Debian ZFS on Linux maintainers# List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.7-1_amd64.deb | egrep -i "arc|dbufstat" -rwxr-xr-x root/root 40751 2016-05-31 03:40 ./bin/arc_summary.py -rwxr-xr-x root/root 13057 2016-05-31 03:40 ./bin/arcstat.py -rwxr-xr-x root/root 18718 2016-05-31 03:40 ./bin/dbufstat.py [Other Info] * This is based on the following Debian commits : Debian git commits === commit 730989397063955d60a1845003cfb7433eba1e52 Author: Aron Xu Date: Tue Apr 26 17:01:29 2016 +0800 Add dh-python to b-d -- commit 814413118fb88de3ce596a87a8a011f50c225fdd Author: Aron Xu Date: Sun Nov 22 15:44:16 2015 +0800 Add python related dependencies === * Despite what Debian did by adding theses script under /bin. I'll add them under /usr/sbin, which is more appropriate. * Unfortunately manpage for theses 3 program seems limited. There is none in Debian nor git upstream repo. The only manpage I found is for arcstat online : http://linux.die.net/man/1/arcstat [Original Description] arcstat.py and arc_summary.py are valuable tools to determine ZFS' ARC usage, it is not obvious why they are not included in zfsutils-linux. As ubuntu-minimal already depends on python3 it should be safe to assume python is available, or am I mistaken here? arcstat.py gives an iostat-like overview about ARC reads, hit rate, current and target size in regular intervals: # ./arcstat.py 1 time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c 19:22:33 0 0 0 00 00 00 2.0G 7.8G 19:22:34 3 0 0 00 00 00 2.0G 7.8G 19:22:3521 0 0 00 00 00 2.0G 7.8G ^C arc_summary.py shows a more detailed overview of the current ARC status and ZFS tunables: # ./arc_summary.py ZFS Subsystem Report Sun Apr 24 19:23:25 2016 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 46 Mutex Misses:0 Evict Skips: 0 ARC Size: 25.10% 1.95GiB Target Size: (Adaptive) 100.00% 7.78GiB Min Size (Hard Limit): 0.40% 32.00 MiB Max Size (High Water): 248:1 7.78GiB ARC Size Breakdown: Recently Used Cache Size:50.00% 3.89GiB Frequently Used Cache Size: 50.00% 3.89GiB ARC Hash Breakdown: Elements Max:32.31k Elements Current:99.78% 32.24k Collisions: 40.54k Chain Max: 3 Chains: 240 ARC Total accesses: 4.54m Cache Hit Ratio: 99.39% 4.51m Cache Miss Ratio:0.61% 27.74k Actual Hit Ratio:98.76% 4.48m Data
[Kernel-packages] [Bug 1574342] Re: Ship arcstat.py and arc_summary.py with zfsutils-linux
That's great news that upstream ZFS will get that fixed up - I didn't see a an issue for it at the ZoL github, are there plans to do so. Would be nice from a tracking perspective so this issue is tied in. Also, I see the Ubuntu update was published for Yakkety, but I'm not sure where Zenial is - will the same Ubuntu fix go for Xenial, or is it waiting for validation, fix from upstream, or ? Thanks and let me know anything I can do to help. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1574342 Title: Ship arcstat.py and arc_summary.py with zfsutils-linux Status in zfs-linux package in Ubuntu: Fix Released Status in zfs-linux source package in Xenial: Fix Committed Status in zfs-linux source package in Yakkety: Fix Released Bug description: [Impact] zfsutils-linux missing some important tools. Python3 compatible scripts : arc_summary.py, arcstat.py and dbufstat.py found in src_code/cmd/ are not taken into account during compilation. * arc_summary.py : Provides a summary of the statistics https://github.com/zfsonlinux/zfs/blob/master/cmd/arc_summary * arcstat.py : Print out ZFS ARC Statistics exported via kstat(1) https://github.com/zfsonlinux/zfs/tree/master/cmd/arcstat * dbufstat.py: Print out statistics for all cached dmu buffers https://github.com/zfsonlinux/zfs/blob/master/cmd/dbufstat [Test Case] # rmadison zfsutils-linux ... zfsutils-linux | 0.6.5.6-0ubuntu10 | xenial-updates ... # List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.6-0ubuntu10_amd64.deb | egrep "arcstat|arc_summary|dbufstat" # You will notice there are not present. [Regression Potential] None expected, debian package[1] has already made the change to build the python scripts. [1] Package: zfsutils-linux Source: zfs-linux Version: 0.6.5.7-1 Architecture: amd64 Maintainer: Debian ZFS on Linux maintainers# List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.7-1_amd64.deb | egrep -i "arc|dbufstat" -rwxr-xr-x root/root 40751 2016-05-31 03:40 ./bin/arc_summary.py -rwxr-xr-x root/root 13057 2016-05-31 03:40 ./bin/arcstat.py -rwxr-xr-x root/root 18718 2016-05-31 03:40 ./bin/dbufstat.py [Other Info] * This is based on the following Debian commits : Debian git commits === commit 730989397063955d60a1845003cfb7433eba1e52 Author: Aron Xu Date: Tue Apr 26 17:01:29 2016 +0800 Add dh-python to b-d -- commit 814413118fb88de3ce596a87a8a011f50c225fdd Author: Aron Xu Date: Sun Nov 22 15:44:16 2015 +0800 Add python related dependencies === * Despite what Debian did by adding theses script under /bin. I'll add them under /usr/sbin, which is more appropriate. * Unfortunately manpage for theses 3 program seems limited. There is none in Debian nor git upstream repo. The only manpage I found is for arcstat online : http://linux.die.net/man/1/arcstat [Original Description] arcstat.py and arc_summary.py are valuable tools to determine ZFS' ARC usage, it is not obvious why they are not included in zfsutils-linux. As ubuntu-minimal already depends on python3 it should be safe to assume python is available, or am I mistaken here? arcstat.py gives an iostat-like overview about ARC reads, hit rate, current and target size in regular intervals: # ./arcstat.py 1 time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c 19:22:33 0 0 0 00 00 00 2.0G 7.8G 19:22:34 3 0 0 00 00 00 2.0G 7.8G 19:22:3521 0 0 00 00 00 2.0G 7.8G ^C arc_summary.py shows a more detailed overview of the current ARC status and ZFS tunables: # ./arc_summary.py ZFS Subsystem Report Sun Apr 24 19:23:25 2016 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 46 Mutex Misses:0 Evict Skips: 0 ARC Size: 25.10% 1.95GiB Target Size: (Adaptive) 100.00% 7.78GiB Min Size (Hard Limit): 0.40% 32.00 MiB Max Size (High Water): 248:1 7.78GiB ARC Size Breakdown: Recently Used Cache Size:50.00% 3.89GiB Frequently Used Cache Size: 50.00% 3.89GiB ARC Hash Breakdown: Elements Max:32.31k Elements Current:99.78% 32.24k Collisions: 40.54k Chain Max: 3 Chains: 240
[Kernel-packages] [Bug 1574342] Re: Ship arcstat.py and arc_summary.py with zfsutils-linux
To add to the confusion - arcstat.py and friends are already in /usr/sbin in the Debian packages I have on Debian Jessie... $ dpkg -S /usr/sbin/arcstat.py zfsutils: /usr/sbin/arcstat.py $ apt show zfsutils Package: zfsutils Source: zfs-linux Version: 0.6.5.7-8-jessie Maintainer: Debian ZFS on Linux maintainers[...] APT-Sources: http://archive.zfsonlinux.org/debian/ jessie/main amd64 Packages [...] -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1574342 Title: Ship arcstat.py and arc_summary.py with zfsutils-linux Status in zfs-linux package in Ubuntu: In Progress Status in zfs-linux source package in Xenial: Fix Committed Status in zfs-linux source package in Yakkety: In Progress Bug description: [Impact] zfsutils-linux missing some important tools. Python3 compatible scripts : arc_summary.py, arcstat.py and dbufstat.py found in src_code/cmd/ are not taken into account during compilation. * arc_summary.py : Provides a summary of the statistics https://github.com/zfsonlinux/zfs/blob/master/cmd/arc_summary * arcstat.py : Print out ZFS ARC Statistics exported via kstat(1) https://github.com/zfsonlinux/zfs/tree/master/cmd/arcstat * dbufstat.py: Print out statistics for all cached dmu buffers https://github.com/zfsonlinux/zfs/blob/master/cmd/dbufstat [Test Case] # rmadison zfsutils-linux ... zfsutils-linux | 0.6.5.6-0ubuntu10 | xenial-updates ... # List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.6-0ubuntu10_amd64.deb | egrep "arcstat|arc_summary|dbufstat" # You will notice there are not present. [Regression Potential] None expected, debian package[1] has already made the change to build the python scripts. [1] Package: zfsutils-linux Source: zfs-linux Version: 0.6.5.7-1 Architecture: amd64 Maintainer: Debian ZFS on Linux maintainers # List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.7-1_amd64.deb | egrep -i "arc|dbufstat" -rwxr-xr-x root/root 40751 2016-05-31 03:40 ./bin/arc_summary.py -rwxr-xr-x root/root 13057 2016-05-31 03:40 ./bin/arcstat.py -rwxr-xr-x root/root 18718 2016-05-31 03:40 ./bin/dbufstat.py [Other Info] * This is based on the following Debian commits : Debian git commits === commit 730989397063955d60a1845003cfb7433eba1e52 Author: Aron Xu Date: Tue Apr 26 17:01:29 2016 +0800 Add dh-python to b-d -- commit 814413118fb88de3ce596a87a8a011f50c225fdd Author: Aron Xu Date: Sun Nov 22 15:44:16 2015 +0800 Add python related dependencies === * Despite what Debian did by adding theses script under /bin. I'll add them under /usr/sbin, which is more appropriate. * Unfortunately manpage for theses 3 program seems limited. There is none in Debian nor git upstream repo. The only manpage I found is for arcstat online : http://linux.die.net/man/1/arcstat [Original Description] arcstat.py and arc_summary.py are valuable tools to determine ZFS' ARC usage, it is not obvious why they are not included in zfsutils-linux. As ubuntu-minimal already depends on python3 it should be safe to assume python is available, or am I mistaken here? arcstat.py gives an iostat-like overview about ARC reads, hit rate, current and target size in regular intervals: # ./arcstat.py 1 time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c 19:22:33 0 0 0 00 00 00 2.0G 7.8G 19:22:34 3 0 0 00 00 00 2.0G 7.8G 19:22:3521 0 0 00 00 00 2.0G 7.8G ^C arc_summary.py shows a more detailed overview of the current ARC status and ZFS tunables: # ./arc_summary.py ZFS Subsystem Report Sun Apr 24 19:23:25 2016 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 46 Mutex Misses:0 Evict Skips: 0 ARC Size: 25.10% 1.95GiB Target Size: (Adaptive) 100.00% 7.78GiB Min Size (Hard Limit): 0.40% 32.00 MiB Max Size (High Water): 248:1 7.78GiB ARC Size Breakdown: Recently Used Cache Size:50.00% 3.89GiB Frequently Used Cache Size: 50.00% 3.89GiB ARC Hash Breakdown: Elements Max:32.31k Elements Current:99.78% 32.24k Collisions: 40.54k Chain Max: 3 Chains:
[Kernel-packages] [Bug 1574342] Re: Ship arcstat.py and arc_summary.py with zfsutils-linux
Testing with 0.6.5.6-0ubuntu12 confirms the tools are now available as expected. Thank you! As far as names ending in .py - while it should be fixed, currently they come that way directly from zfsonlinux. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1574342 Title: Ship arcstat.py and arc_summary.py with zfsutils-linux Status in zfs-linux package in Ubuntu: In Progress Status in zfs-linux source package in Xenial: Fix Committed Status in zfs-linux source package in Yakkety: In Progress Bug description: [Impact] zfsutils-linux missing some important tools. Python3 compatible scripts : arc_summary.py, arcstat.py and dbufstat.py found in src_code/cmd/ are not taken into account during compilation. * arc_summary.py : Provides a summary of the statistics https://github.com/zfsonlinux/zfs/blob/master/cmd/arc_summary * arcstat.py : Print out ZFS ARC Statistics exported via kstat(1) https://github.com/zfsonlinux/zfs/tree/master/cmd/arcstat * dbufstat.py: Print out statistics for all cached dmu buffers https://github.com/zfsonlinux/zfs/blob/master/cmd/dbufstat [Test Case] # rmadison zfsutils-linux ... zfsutils-linux | 0.6.5.6-0ubuntu10 | xenial-updates ... # List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.6-0ubuntu10_amd64.deb | egrep "arcstat|arc_summary|dbufstat" # You will notice there are not present. [Regression Potential] None expected, debian package[1] has already made the change to build the python scripts. [1] Package: zfsutils-linux Source: zfs-linux Version: 0.6.5.7-1 Architecture: amd64 Maintainer: Debian ZFS on Linux maintainers# List contents of a deb package. $ dpkg -c zfsutils-linux_0.6.5.7-1_amd64.deb | egrep -i "arc|dbufstat" -rwxr-xr-x root/root 40751 2016-05-31 03:40 ./bin/arc_summary.py -rwxr-xr-x root/root 13057 2016-05-31 03:40 ./bin/arcstat.py -rwxr-xr-x root/root 18718 2016-05-31 03:40 ./bin/dbufstat.py [Other Info] * This is based on the following Debian commits : Debian git commits === commit 730989397063955d60a1845003cfb7433eba1e52 Author: Aron Xu Date: Tue Apr 26 17:01:29 2016 +0800 Add dh-python to b-d -- commit 814413118fb88de3ce596a87a8a011f50c225fdd Author: Aron Xu Date: Sun Nov 22 15:44:16 2015 +0800 Add python related dependencies === * Despite what Debian did by adding theses script under /bin. I'll add them under /usr/sbin, which is more appropriate. * Unfortunately manpage for theses 3 program seems limited. There is none in Debian nor git upstream repo. The only manpage I found is for arcstat online : http://linux.die.net/man/1/arcstat [Original Description] arcstat.py and arc_summary.py are valuable tools to determine ZFS' ARC usage, it is not obvious why they are not included in zfsutils-linux. As ubuntu-minimal already depends on python3 it should be safe to assume python is available, or am I mistaken here? arcstat.py gives an iostat-like overview about ARC reads, hit rate, current and target size in regular intervals: # ./arcstat.py 1 time read miss miss% dmis dm% pmis pm% mmis mm% arcsz c 19:22:33 0 0 0 00 00 00 2.0G 7.8G 19:22:34 3 0 0 00 00 00 2.0G 7.8G 19:22:3521 0 0 00 00 00 2.0G 7.8G ^C arc_summary.py shows a more detailed overview of the current ARC status and ZFS tunables: # ./arc_summary.py ZFS Subsystem Report Sun Apr 24 19:23:25 2016 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 46 Mutex Misses:0 Evict Skips: 0 ARC Size: 25.10% 1.95GiB Target Size: (Adaptive) 100.00% 7.78GiB Min Size (Hard Limit): 0.40% 32.00 MiB Max Size (High Water): 248:1 7.78GiB ARC Size Breakdown: Recently Used Cache Size:50.00% 3.89GiB Frequently Used Cache Size: 50.00% 3.89GiB ARC Hash Breakdown: Elements Max:32.31k Elements Current:99.78% 32.24k Collisions: 40.54k Chain Max: 3 Chains: 240 ARC Total accesses: 4.54m Cache Hit Ratio: 99.39% 4.51m Cache Miss Ratio:0.61% 27.74k Actual Hit Ratio:98.76% 4.48m Data Demand Efficiency:
[Kernel-packages] [Bug 1304001] Re: xen:balloon errors in 14.04 beta
Will this patch go out in 3.13.0-38 this week? I'm assuming that the discussion around 'linux-keystone' is something different. Regardless, please do not drop this fix from 3.13.0-38 as we're expecting it this week per Stefan's comments above - and we have already verified it resolves the issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1304001 Title: xen:balloon errors in 14.04 beta Status in “linux” package in Ubuntu: Fix Released Status in “linux” source package in Trusty: Fix Committed Status in “linux” source package in Utopic: Fix Committed Bug description: SRU Justification: [Impact] The following errors may occur on HVM instances on EC2: xen:balloon: reserve_additional_memory: add_memory() failed: -17 [Test Case] Boot Ubuntu Trusty 3.13 series HVM instances and check dmesg for this error message. [Fix] A minimal fix for this bug can be found here: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3dcf63677d4eb7fdfc13290c8558c301d2588fe8 This allows the ballooning to be cancelled if adding new memory failed which means that the error message is printed once instead of repeatedly. There should still be effort to root cause this issue and determine how to avoid the ballooning errors in the first place. I still think this patch should be applied to alleviate symptoms until root cause is discovered. -- Xen balloon errors on HVM instances on EC2 (Xen 4.2.amazon): ubuntu@ip-10-63-20-99:~$ uname -a Linux ip-10-63-20-99 3.13.0-23-generic #45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ubuntu@ip-10-63-20-99:~$ dmesg |grep xen [0.00] xen:events: Xen HVM callback vector for event delivery is enabled [0.494613] xen:balloon: Initialising balloon driver [0.496046] xen_balloon: Initialising balloon driver [0.500077] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [0.541047] Switched to clocksource xen [0.562579] xen: -- pirq=16 - irq=8 (gsi=8) [0.562622] xen: -- pirq=17 - irq=12 (gsi=12) [0.562649] xen: -- pirq=18 - irq=1 (gsi=1) [0.562673] xen: -- pirq=19 - irq=6 (gsi=6) [0.562705] xen: -- pirq=20 - irq=4 (gsi=4) [0.920527] xen: -- pirq=21 - irq=47 (gsi=47) [0.920596] xen:grant_table: Grant tables using version 1 layout [1.029661] xen_netfront: Initialising Xen virtual ethernet driver [1.236083] xenbus_probe_frontend: Device with no driver: device/vfb/0 [2.516067] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [6.533941] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 14.560075] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 30.592064] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 62.688153] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 94.752164] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 126.816161] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 158.880084] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 190.944069] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 223.008141] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 255.072112] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 287.136190] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 319.200053] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 351.264164] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 383.328080] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 415.392077] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 447.456112] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 479.520128] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 511.584110] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 543.648181] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 575.712070] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 607.776178] xen:balloon: reserve_additional_memory: add_memory() failed: -17 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1304001/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1304001] Re: xen:balloon errors in 14.04 beta
I tested with the latest 3.13 kernel in trusty-proposed: Version table: 3.13.0.40.47 0 500 http://archive.ubuntu.com/ubuntu/ trusty-proposed/main amd64 Packages I can confirm that the xen:balloon: Cannot add additional memory (-17) message do not occur with this kernel in AWS with an m3 class instance and HVM virtualization. This looks good to me - fix verified. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1304001 Title: xen:balloon errors in 14.04 beta Status in “linux” package in Ubuntu: Fix Released Status in “linux” source package in Trusty: Fix Committed Status in “linux” source package in Utopic: Fix Committed Bug description: SRU Justification: [Impact] The following errors may occur on HVM instances on EC2: xen:balloon: reserve_additional_memory: add_memory() failed: -17 [Test Case] Boot Ubuntu Trusty 3.13 series HVM instances and check dmesg for this error message. [Fix] A minimal fix for this bug can be found here: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=3dcf63677d4eb7fdfc13290c8558c301d2588fe8 This allows the ballooning to be cancelled if adding new memory failed which means that the error message is printed once instead of repeatedly. There should still be effort to root cause this issue and determine how to avoid the ballooning errors in the first place. I still think this patch should be applied to alleviate symptoms until root cause is discovered. -- Xen balloon errors on HVM instances on EC2 (Xen 4.2.amazon): ubuntu@ip-10-63-20-99:~$ uname -a Linux ip-10-63-20-99 3.13.0-23-generic #45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux ubuntu@ip-10-63-20-99:~$ dmesg |grep xen [0.00] xen:events: Xen HVM callback vector for event delivery is enabled [0.494613] xen:balloon: Initialising balloon driver [0.496046] xen_balloon: Initialising balloon driver [0.500077] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [0.541047] Switched to clocksource xen [0.562579] xen: -- pirq=16 - irq=8 (gsi=8) [0.562622] xen: -- pirq=17 - irq=12 (gsi=12) [0.562649] xen: -- pirq=18 - irq=1 (gsi=1) [0.562673] xen: -- pirq=19 - irq=6 (gsi=6) [0.562705] xen: -- pirq=20 - irq=4 (gsi=4) [0.920527] xen: -- pirq=21 - irq=47 (gsi=47) [0.920596] xen:grant_table: Grant tables using version 1 layout [1.029661] xen_netfront: Initialising Xen virtual ethernet driver [1.236083] xenbus_probe_frontend: Device with no driver: device/vfb/0 [2.516067] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [6.533941] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 14.560075] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 30.592064] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 62.688153] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 94.752164] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 126.816161] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 158.880084] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 190.944069] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 223.008141] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 255.072112] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 287.136190] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 319.200053] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 351.264164] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 383.328080] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 415.392077] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 447.456112] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 479.520128] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 511.584110] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 543.648181] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 575.712070] xen:balloon: reserve_additional_memory: add_memory() failed: -17 [ 607.776178] xen:balloon: reserve_additional_memory: add_memory() failed: -17 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1304001/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp