subject:"\[Kernel\-packages\] \[Bug 1706132\] Re\: xfs slab objects \(memory\) leak when xfs shutdown is called"

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2018-05-18 Thread Rafael David Tinoco

** Changed in: linux (Ubuntu)
   Status: In Progress => Fix Released

** Changed in: linux (Ubuntu)
 Assignee: Rafael David Tinoco (inaddy) => (unassigned)

** Changed in: linux (Ubuntu Xenial)
 Assignee: Rafael David Tinoco (inaddy) => (unassigned)

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  Fix Released
Status in linux source package in Xenial:
  Fix Released

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-08-28 Thread Launchpad Bug Tracker

This bug was fixed in the package linux - 4.4.0-93.116

---
linux (4.4.0-93.116) xenial; urgency=low

  * linux: 4.4.0-93.116 -proposed tracker (LP: #1709296)

  * Creating conntrack entry failure with kernel 4.4.0-89 (LP: #1709032)
- Revert "Revert "netfilter: synproxy: fix conntrackd interaction""
- netfilter: nf_ct_ext: fix possible panic after nf_ct_extend_unregister

  * CVE-2017-1000112
- Revert "udp: consistently apply ufo or fragmentation"
- udp: consistently apply ufo or fragmentation

  * CVE-2017-1000111
- Revert "net-packet: fix race in packet_set_ring on PACKET_RESERVE"
- packet: fix tp_reserve race in packet_set_ring

  * kernel BUG at [tty_ldisc_reinit] mm/slub.c! (LP: #1709126)
- tty: Simplify tty_set_ldisc() exit handling
- tty: Reset c_line from driver's init_termios
- tty: Handle NULL tty->ldisc
- tty: Move tty_ldisc_kill()
- tty: Use 'disc' for line discipline index name
- tty: Refactor tty_ldisc_reinit() for reuse
- tty: Destroy ldisc instance on hangup

  * atheros bt failed after S3 (LP: #1706833)
- SAUCE: Bluetooth: Make request workqueue freezable

  * The Precision Touchpad(PTP) button sends incorrect event code (LP: #1708372)
- HID: multitouch: handle external buttons for Precision Touchpads

  * Set CONFIG_SATA_HIGHBANK=y on armhf (LP: #1703430)
- [Config] CONFIG_SATA_HIGHBANK=y

  * xfs slab objects (memory) leak when xfs shutdown is called (LP: #1706132)
- xfs: fix xfs_log_ticket leak in xfs_end_io() after fs shutdown

  * Adt tests of src:linux time out often on armhf lxc containers (LP: #1705495)
- [Packaging] tests -- reduce rebuild test to one flavour

  * CVE-2017-7495
- ext4: fix data exposure after a crash

  * ubuntu/rsi driver downlink wifi throughput drops to 5-6 Mbps when BT
keyboard is connected (LP: #1706991)
- SAUCE: Redpine: enable power save by default for coex mode
- SAUCE: Redpine: uapsd configuration changes

  * [Hyper-V] hv_netvsc: Exclude non-TCP port numbers from vRSS hashing
(LP: #1690174)
- hv_netvsc: Exclude non-TCP port numbers from vRSS hashing

  * ath10k doesn't report full RSSI information (LP: #1706531)
- ath10k: add per chain RSSI reporting

  * ideapad_laptop don't support v310-14isk (LP: #1705378)
- platform/x86: ideapad-laptop: Add several models to no_hw_rfkill

  * [8087:0a2b] Failed to load bluetooth firmware(might affect some other Intel
bt devices) (LP: #1705633)
- Bluetooth: btintel: Create common Intel Version Read function
- Bluetooth: Use switch statement for Intel hardware variants
- Bluetooth: Replace constant hw_variant from Intel Bluetooth firmware
  filename
- Bluetooth: hci_intel: Fix firmware file name to use hw_variant
- Bluetooth: btintel: Add MODULE_FIRMWARE entries for iBT 3.5 controllers

  * xhci_hcd: ERROR Transfer event TRB DMA ptr not part of current TD ep_index 2
comp_code 13 (LP: #1667750)
- xhci: Bad Ethernet performance plugged in ASM1042A host

  * OpenPower: Some multipaths temporarily have only a single path
(LP: #1696445)
- scsi: ses: don't get power status of SES device slot on probe

  * Hotkeys on new Thinkpad systems aren't working (LP: #1705169)
- platform/x86: thinkpad_acpi: Adding new hotkey ID for Lenovo thinkpad
- platform/x86: thinkpad_acpi: guard generic hotkey case
- platform/x86: thinkpad_acpi: add mapping for new hotkeys

  * CVE-2015-7837
- SAUCE: (no-up) kexec/uefi: copy secure_boot flag in boot params across 
kexec
  reboot

  * misleading kernel warning skb_warn_bad_offload during checksum calculation
(LP: #1705447)
- net: reduce skb_warn_bad_offload() noise

  * bonding: stack dump when unregistering a netdev (LP: #1704102)
- bonding: avoid NETDEV_CHANGEMTU event when unregistering slave

  * Ubuntu 16.04 IOB Error when the Mustang board rebooted (LP: #1693673)
- drivers: net: xgene: Fix redundant prefetch buffer cleanup

  * Ubuntu16.04: NVMe 4K+T10 DIF/DIX format returns I/O error on dd with split
op (LP: #1689946)
- blk-mq: NVMe 512B/4K+T10 DIF/DIX format returns I/O error on dd with split
  op

  * linux >= 4.2: bonding 802.3ad does not work with 5G, 25G and 50G link speeds
(LP: #1697892)
- bonding: add 802.3ad support for 100G speeds
- bonding: fix 802.3ad aggregator reselection
- bonding: add 802.3ad support for 25G speeds
- bonding: fix 802.3ad support for 5G and 50G speeds

  * Xenial update to 4.4.79 stable release (LP: #1707233)
- disable new gcc-7.1.1 warnings for now
- ir-core: fix gcc-7 warning on bool arithmetic
- s5p-jpeg: don't return a random width/height
- thermal: cpu_cooling: Avoid accessing potentially freed structures
- ath9k: fix tx99 use after free
- ath9k: fix tx99 bus error
- NFC: fix broken device allocation
- NFC: nfcmrvl_uart: add missing tty-device sanity check
- NFC: nfcmrvl: do not use

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-08-23 Thread Rafael David Tinoco

Okay,

First I have failed the path without propagating transport layer error
and xfs stayed up (nmi watchdog would eventually detect a lockup if
pagecache was ever attempted to be flushed), i returned the path and
failed it again, now with error propagation. The error was detected and
all SCSI CDBs were failed, SCSI sensed errors and XFS called shutdown:

http://pastebin.ubuntu.com/25376025/

Aug 23 11:34:47 xfshang kernel: [ 1871.328044] blk_update_request: I/O error, 
dev sdb, sector 20481104
Aug 23 11:34:47 xfshang kernel: [ 1871.328051] XFS (sdb1): metadata I/O error: 
block 0x1387c50 ("xlog_iodone") error 5 numblks 64
Aug 23 11:34:47 xfshang kernel: [ 1871.328053] XFS (sdb1): 
xfs_do_force_shutdown(0x2) called from line 1197 of file 
/build/linux-lts-xenial-AsqcmJ/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c.  Return 
address = 0xc025f698
Aug 23 11:34:47 xfshang kernel: [ 1871.328066] XFS (sdb1): Log I/O Error 
Detected.  Shutting down filesystem
Aug 23 11:34:47 xfshang kernel: [ 1871.328067] XFS (sdb1): Please umount the 
filesystem and rectify the problem(s)
Aug 23 11:34:47 xfshang kernel: [ 1873.966241] buffer_io_error: 173396 
callbacks suppressed

Aug 23 11:34:47 xfshang kernel: [ 1873.966244] Buffer I/O error on dev sdb1, 
logical block 102421, lost async page write
Aug 23 11:34:47 xfshang kernel: [ 1873.967411] Buffer I/O error on dev sdb1, 
logical block 102422, lost async page write
Aug 23 11:34:47 xfshang kernel: [ 1873.968599] Buffer I/O error on dev sdb1, 
logical block 102423, lost async page write
...
Aug 23 11:34:56 xfshang dhclient: DHCPREQUEST of 192.168.50.95 on internal to 
192.168.48.1 port 67 (xid=0x87b4739)
Aug 23 11:34:56 xfshang dhclient: DHCPACK of 192.168.50.95 from 192.168.48.1
Aug 23 11:34:56 xfshang dhclient: bound to 192.168.50.95 -- renewal in 15 
seconds.
Aug 23 11:35:01 xfshang kernel: [ 1887.284525] XFS (sdb1): xfs_log_force: error 
-5 returned.
Aug 23 11:35:01 xfshang kernel: [ 1887.548087] XFS (sdb1): Unmounting Filesystem
Aug 23 11:35:01 xfshang kernel: [ 1887.548100] XFS (sdb1): xfs_log_force: error 
-5 returned.
Aug 23 11:35:01 xfshang kernel: [ 1887.548145] XFS (sdb1): xfs_log_force: error 
-5 returned.
Aug 23 11:35:01 xfshang kernel: [ 1887.548438] sd 5:0:0:1: rejecting I/O to 
offline device
Aug 23 11:35:01 xfshang kernel: [ 1887.549424] blk_update_request: I/O error, 
dev sdb, sector 0

Then I removed xfs module and no leak was found. I consider this
verified and good to go.

Thank you!

-Rafael

** Tags removed: verification-needed-xenial
** Tags added: verification-done

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  In Progress
Status in linux source package in Xenial:
  Fix Committed

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-08-23 Thread Rafael David Tinoco

Working on verification right now.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  In Progress
Status in linux source package in Xenial:
  Fix Committed

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070570] Disabling lock debugging due to kernel taint
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-08-16 Thread Kleber Sacilotto de Souza

This bug is awaiting verification that the kernel in -proposed solves
the problem. Please test the kernel and update this bug with the
results. If the problem is solved, change the tag 'verification-needed-
xenial' to 'verification-done-xenial'. If the problem still exists,
change the tag 'verification-needed-xenial' to 'verification-failed-
xenial'.

If verification is not done by 5 working days from today, this fix will
be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how
to enable and use -proposed. Thank you!


** Tags added: verification-needed-xenial

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  In Progress
Status in linux source package in Xenial:
  Fix Committed

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-08-08 Thread Rafael David Tinoco

** Tags added: sts

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  In Progress
Status in linux source package in Xenial:
  Fix Committed

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070570] Disabling lock debugging due to kernel taint
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-08-08 Thread Kleber Sacilotto de Souza

** Changed in: linux (Ubuntu Xenial)
   Status: In Progress => Fix Committed

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  In Progress
Status in linux source package in Xenial:
  Fix Committed

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070570] Disabling lock debugging due to kernel taint
  [82211.070580]

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-08-06 Thread Rafael David Tinoco

** Changed in: linux (Ubuntu Xenial)
   Status: New => In Progress

** Changed in: linux (Ubuntu)
   Status: Confirmed => In Progress

** Changed in: linux (Ubuntu Xenial)
   Importance: Undecided => Low

** Changed in: linux (Ubuntu Xenial)
 Assignee: (unassigned) => Rafael David Tinoco (inaddy)

** Changed in: linux (Ubuntu Xenial)
Milestone: None => ubuntu-16.04.3

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  In Progress
Status in linux source package in Xenial:
  In Progress

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-07-27 Thread Stefan Bader

** Also affects: linux (Ubuntu Xenial)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  Confirmed
Status in linux source package in Xenial:
  New

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070570] Disabling lock debugging due to kernel taint
  [82211.070580] INFO:

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-07-25 Thread Rafael David Tinoco

Testing this patch:

inaddy@iscsixfs:~$ sudo uname -a
Linux iscsixfs 4.4.70+ #2 SMP Mon Jul 24 19:25:49 UTC 2017 x86_64 x86_64 x86_64 
GNU/Linux

inaddy@iscsixfs:~$ sudo dmesg | grep xfs
[  109.876336] XFS (sda1): metadata I/O error: block 0xc34d83 
("xfs_trans_read_buf_map") error 5 numblks 1
[  109.876420] XFS (sda1): metadata I/O error: block 0xc34d83 
("xfs_trans_read_buf_map") error 5 numblks 1
[  109.876463] XFS (sda1): metadata I/O error: block 0xc34d83 
("xfs_trans_read_buf_map") error 5 numblks 1
[  109.876524] XFS (sda1): metadata I/O error: block 0xc34d83 
("xfs_trans_read_buf_map") error 5 numblks 1
... tons of times ...
[  114.612131] XFS (sda1): metadata I/O error: block 0xc34d83 
("xfs_trans_read_buf_map") error 5 numblks 1
[  114.612204] XFS (sda1): metadata I/O error: block 0xc34d83 
("xfs_trans_read_buf_map") error 5 numblks 1
[  114.612254] XFS (sda1): metadata I/O error: block 0xc34d83 
("xfs_trans_read_buf_map") error 5 numblks 1
[  114.631338] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 of 
file /home/inaddy/sources/trees/xenial/fs/xfs/xfs_log.c.  Return address = 
0xc032e0d8
[  114.631381] XFS (sda1): xfs_log_force: error -5 returned.
[  114.635642] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 of 
file /home/inaddy/sources/trees/xenial/fs/xfs/xfs_log.c.  Return address = 
0xc032e0d8
[  114.680054] XFS (sda1): xfs_log_force: error -5 returned.

inaddy@iscsixfs:~$ lsmod | grep xfs
xfs   970752  1 
libcrc32c  16384  1 xfs

inaddy@iscsixfs:~$ sudo umount /xfs01
[  139.491719] sd 2:0:0:1: rejecting I/O to offline device
[  139.492705] blk_update_request: I/O error, dev sda, sector 0

inaddy@iscsixfs:~$ sudo rmmod xfs

Patch fixes the issue.

inaddy@iscsixfs:~$ sudo modprobe xfs

inaddy@iscsixfs:~$ ls /dev/sd*
/dev/sda  /dev/sda1  /dev/sdb  /dev/sdb1

inaddy@iscsixfs:~$ sudo mount /dev/sda1 /xfs01

inaddy@iscsixfs:~$ ls /xfs01
Bonnie.1677

Disk is back online after module was reloaded.
(I could have checked slab objects for xfs_log_ticket but I guess this is 
enough).

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun
   * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
   * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * make sure the i/o errors led to xfs shutdown (comment #3)
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-07-25 Thread Rafael David Tinoco

Consistent reproducer:

inaddy@iscsixfs:~$ uname -a
Linux iscsixfs 4.4.0-87-generic #110~14.04.1-Ubuntu SMP Tue Jul 18 14:51:32 UTC 
2017 x86_64 x86_64 x86_64 GNU/Linux

inaddy@iscsixfs:~$ dmesg | grep xfs

[  381.648984] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 of 
file /build/linux-lts-xenial-N0BD4p/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c.  
Return address = 0xc02a16a8
[  381.700064] XFS (sda1): xfs_log_force: error -5 returned.

inaddy@iscsixfs:~$ lsmod | grep xfs
xfs   954368  1 
libcrc32c  16384  1 xfs


inaddy@iscsixfs:~$ sudo umount /xfs01
[  449.785939] sd 2:0:0:1: rejecting I/O to offline device
[  449.789923] blk_update_request: I/O error, dev sda, sector 0

inaddy@iscsixfs:~$ sudo rmmod xfs
[  452.928198] 
=
[  452.929647] BUG xfs_log_ticket (Not tainted): Objects remaining in 
xfs_log_ticket on kmem_cache_close()
[  452.931259] 
-
[  452.931259] 
[  452.933002] INFO: Slab 0xea0002e67a80 objects=22 used=3 
fp=0x8800b99eaf18 flags=0xc00080
[  452.934773] INFO: Object 0x8800b99ea000 @offset=0
[  452.935693] INFO: Object 0x8800b99ea228 @offset=552
[  452.936675] INFO: Object 0x8800b99ea2e0 @offset=736
[  452.937642] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects


** Description changed:

  [Impact]
  
-  * xfs kernel memory leak in case of xfs shutdown due to i/o errors
-  * if xfs on iscsi, iscsi disconnection and module unload will case mem leak
+  * xfs kernel memory leak in case of xfs shutdown due to i/o errors
+  * if xfs on iscsi, iscsi disconnection and module unload will case mem leak
  
  [Test Case]
  
-  * configure tgtd with 1 lun and make it available through tcp/ip
-  * configure open-iscsi to map this lun with a timeout >= 1
-  * in tgtd server, drop iscsi packets and watch client to have i/o errors
-  * after sometime (depending on timeout) xfs will call shutdown
-  * after shutdown you try to remove xfs module and it will leak
+  * configure tgtd with 1 lun and make it available through tcp/ip
+  * configure open-iscsi to map this lun
+  * make sure node.session.timeo.replacement_timeout = 0 in iscsid.conf
+  * mount a xfs volume using the lun from tgtd host, run bonnie -d /xfsdir
+  * in tgtd server, drop iscsi packets and watch client to have i/o errors
+  * after sometime (depending on timeout) xfs will call shutdown
+  * make sure the i/o errors led to xfs shutdown (comment #3)
+  * after shutdown you try to remove xfs module and it will leak
  
  [Regression Potential]
  
-  * based on upstream fix
-  * tested in the same environment
-  * potential damage to xfs
+  * based on upstream fix
+  * tested in the same environment
+  * potential damage to xfs
  
  [Other Info]
  
  Original Description:
  
     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]
  
  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device
  
  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.
  
  # when XFS shuts down because of an error (or offline disk, example):
  
  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem
  
  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.
  
  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)
  
  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  
  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.
  
  # when I execute:
  
  # sudo umount /dev/sda1:

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-07-25 Thread Rafael David Tinoco

** Changed in: linux (Ubuntu)
   Importance: Medium => Low

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  [Impact]

   * xfs kernel memory leak in case of xfs shutdown due to i/o errors
   * if xfs on iscsi, iscsi disconnection and module unload will case mem leak

  [Test Case]

   * configure tgtd with 1 lun and make it available through tcp/ip
   * configure open-iscsi to map this lun with a timeout >= 1
   * in tgtd server, drop iscsi packets and watch client to have i/o errors
   * after sometime (depending on timeout) xfs will call shutdown
   * after shutdown you try to remove xfs module and it will leak

  [Regression Potential]

   * based on upstream fix
   * tested in the same environment
   * potential damage to xfs

  [Other Info]

  Original Description:

     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450]
  [82211.070570] Disabling lock debugging due to kernel taint
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.073964] CPU: 3 PID: 32230 Comm: rmmod Tainted: GB  OE   
4.4.0-74-generic #95~14.04.1-Ubuntu
  [82211.073970] Hardware name: QEMU Standard PC (i440FX + PIIX,

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-07-24 Thread Rafael David Tinoco

** Description changed:

+ [Impact]
+ 
+  * xfs kernel memory leak in case of xfs shutdown due to i/o errors
+  * if xfs on iscsi, iscsi disconnection and module unload will case mem leak
+ 
+ [Test Case]
+ 
+  * configure tgtd with 1 lun and make it available through tcp/ip
+  * configure open-iscsi to map this lun with a timeout >= 1
+  * in tgtd server, drop iscsi packets and watch client to have i/o errors
+  * after sometime (depending on timeout) xfs will call shutdown
+  * after shutdown you try to remove xfs module and it will leak
+ 
+ [Regression Potential]
+ 
+  * based on upstream fix
+  * tested in the same environment
+  * potential damage to xfs
+ 
+ [Other Info]
+ 
+ Original Description:
+ 
     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]
  
  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device
  
  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.
  
  # when XFS shuts down because of an error (or offline disk, example):
  
  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem
  
  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.
  
  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)
  
  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  
  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.
  
  # when I execute:
  
  # sudo umount /dev/sda1:
  
  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0
  
  # i was able to umount and then to remove iscsi disk.
  
  # but if i try to unload the xfs module:
  
  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
- [82211.067450] 
+ [82211.067450]
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects
  
  AND
  
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
- [82211.067450] 
+ [82211.067450]
  [82211.070570] Disabling lock debugging due to kernel taint
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.073964] CPU: 3 PID: 32230 Comm: rmmod Tainted: GB  OE   
4.4.0-74-generic #95~14.04.1-Ubuntu
  [82211.073970] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
Ubuntu-1.8.2-1ubuntu1 04/01/2014
  [82211.073975]   8800baf53d28 813dce3c 
ea0002eb7640
  [82211.073984]  880036614300 8800baf53e00 811dc4b4 
8820
  [82211.073991]  8800baf53e10

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-07-24 Thread Rafael David Tinoco

$ git tag --contains af055e37a91d215d7174d0b84c86795ca81086a7 | grep -v -- -rc
v4.10
v4.11
v4.12
v4.6
v4.7
v4.8
v4.9

Only Xenial seems to be affected.

** Changed in: linux (Ubuntu)
 Assignee: (unassigned) => Rafael David Tinoco (inaddy)

** Changed in: linux (Ubuntu)
   Importance: Undecided => Medium

** Changed in: linux (Ubuntu)
   Status: New => Confirmed

** Changed in: linux (Ubuntu)
Milestone: None => ubuntu-16.04.3

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  Confirmed

Bug description:
     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450] 
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket: Slab cache still has objects

  AND

  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450] 
  [82211.070570] Disabling lock debugging due to kernel taint
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.073964] CPU: 3 PID: 32230 Comm: rmmod Tainted: GB  OE   
4.4.0-74-generic #95~14.04.1-Ubuntu
  [82211.073970] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
Ubuntu-1.8.2-1ubuntu1 04/01/2014
  [82211.073975]   8800baf53d28 813dce3c 
ea0002eb7640
  [82211.073984]  880036614300 8800baf53e00 811dc4b4 
8820
  [82211.073991]  8800baf53e10 8800baf53dc0 656a624f02e294c0

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

2017-07-24 Thread Rafael David Tinoco

# the following commit solves the issue:

commit af055e37a91d215d7174d0b84c86795ca81086a7
Author: Brian Foster 
Date:   Mon Feb 8 15:00:02 2016 +1100

xfs: fix xfs_log_ticket leak in xfs_end_io() after fs shutdown

If the filesystem has shut down, xfs_end_io() currently sets an
error on the ioend and proceeds to ioend destruction. The ioend
might contain a truncate transaction if the I/O extended the size of
the file. This transaction is only cleaned up in
xfs_setfilesize_ioend(), however, which is skipped in this case.
This results in an xfs_log_ticket leak message when the associate
cache slab is destroyed (e.g., on rmmod).

This was originally reproduced by xfs/141 on a distro kernel. The
problem is reproducible on an upstream kernel, but not easily
detected in current upstream if the xfs_log_ticket cache happens to
be merged with another cache. This can be reproduced more
deterministically with the 'slab_nomerge' kernel boot option.

Update xfs_end_io() to proceed with normal end I/O processing after
an error is set on an ioend due to fs shutdown. The I/O type-based
processing is already designed to handle an I/O error and ensure
that the ioend is cleaned up correctly.

Signed-off-by: Brian Foster 
Reviewed-by: Dave Chinner 
Signed-off-by: Dave Chinner 

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1706132

Title:
  xfs slab objects (memory) leak when xfs shutdown is called

Status in linux package in Ubuntu:
  Confirmed

Bug description:
     This scenario is testing [iscsi <-> scsi <-> disk <-> xfs]

  [  551.125604] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125615] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125627] sd 2:0:0:1: rejecting I/O to offline device
  [  551.125639] sd 2:0:0:1: rejecting I/O to offline device

  [  551.135216] XFS (sda1): metadata I/O error: block 0xeffe01 
("xfs_trans_read_buf_map") error 5 numblks 1
  [  551.135274] XFS (sda1): page discard on page ea0002a89cc0, inode 0x83, 
offset 6442385408.

  # when XFS shuts down because of an error (or offline disk, example):

  [  551.850498] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388
  [  551.850568] XFS (sda1): Log I/O Error Detected.  Shutting down filesystem

  [  551.850618] XFS (sda1): xfs_log_force: error -5 returned.

  [  551.850630] XFS (sda1): Failing async write on buffer block 0x77ff08. 
Retrying async write.
  [  551.850634] XFS (sda1): Failing async write on buffer block 0x77ff10. 
Retrying async write.
  [  551.850638] XFS (sda1): Failing async write on buffer block 0x77ff01. 
Retrying async write.
  [  551.853171] XFS (sda1): Please umount the filesystem and rectify the 
problem(s)

  [  551.874131] XFS (sda1): metadata I/O error: block 0x1dffc49 
("xlog_iodone") error 5 numblks 64
  [  551.877993] XFS (sda1): xfs_do_force_shutdown(0x2) called from line 1197 
of file /build/linux-lts-xenial-roXrYH/linux-lts-xenial-4.4.0/fs/xfs/xfs_log.c. 
 Return address = 0xc0300388

  [  551.899036] XFS (sda1): xfs_log_force: error -5 returned.
  [  569.323074] XFS (sda1): xfs_log_force: error -5 returned.
  [  599.403085] XFS (sda1): xfs_log_force: error -5 returned.
  [  629.483111] XFS (sda1): xfs_log_force: error -5 returned.
  [  659.563115] XFS (sda1): xfs_log_force: error -5 returned.
  [  689.643014] XFS (sda1): xfs_log_force: error -5 returned.

  # when I execute:

  # sudo umount /dev/sda1:

  [81634.923043] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739097] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739137] XFS (sda1): Unmounting Filesystem
  [81640.739463] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.739508] XFS (sda1): xfs_log_force: error -5 returned.
  [81640.742741] sd 2:0:0:1: rejecting I/O to offline device
  [81640.745576] blk_update_request: 25 callbacks suppressed
  [81640.745601] blk_update_request: I/O error, dev sda, sector 0

  # i was able to umount and then to remove iscsi disk.

  # but if i try to unload the xfs module:

  inaddy@(trustyiscsicli):~$ sudo rmmod xfs
  [82211.059301] 
=
  [82211.063764] BUG xfs_log_ticket (Tainted: G   OE  ): Objects 
remaining in xfs_log_ticket on kmem_cache_close()
  [82211.067450] 
-
  [82211.067450] 
  [82211.070580] INFO: Slab 0xea0002eb7640 objects=22 used=1 
fp=0x8800badd9f18 flags=0xc00080
  [82211.074430] INFO: Object 0x8800badd9228 @offset=552
  [82211.076133] kmem_cache_destroy xfs_log_ticket:

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

[Kernel-packages] [Bug 1706132] Re: xfs slab objects (memory) leak when xfs shutdown is called

15 matches

Site Navigation

Mail list logo

Footer information