[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
This was fixed in 0.36.2. ** Changed in: ufw Importance: Undecided => Medium ** Changed in: ufw Assignee: (unassigned) => Jamie Strandboge (jdstrand) ** Changed in: ufw Status: Fix Committed => Fix Released ** Changed in: ufw (Ubuntu) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Released Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Hi. No, I haven't tested on that; but if this helps, the issue is independent of platform/hypervisor as it's in ufw/firewall, thus the fix should help anyway. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Hello folks, sorry for digging the old thread. I'd like to ask whether you have tested the bug on Ubuntu VM? VMware specifically. Cheers -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds.
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Tags removed: sts-sponsor-mfo -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917]
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
This bug was fixed in the package ufw - 0.36-0ubuntu0.18.04.2 --- ufw (0.36-0ubuntu0.18.04.2) bionic; urgency=medium * d/p/0002-set-default-policy-after-load.patch: fix boot stall on iscsi/network root filesystem when starting ufw (LP: #1946804) * d/p/0003-unconditionally-reload-with-delete.patch: fix corner case of rule deletion with specific/any proto (LP: #1933117) -- Mauricio Faria de Oliveira Mon, 25 Oct 2021 14:30:24 -0300 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1:
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
This bug was fixed in the package ufw - 0.36-7.1ubuntu1 --- ufw (0.36-7.1ubuntu1) hirsute; urgency=medium * d/p/0015-set-default-policy-after-load.patch: fix boot stall on iscsi/network root filesystem when starting ufw (LP: #1946804) * d/p/0016-unconditionally-reload-with-delete.patch: fix corner case of rule deletion with specific/any proto (LP: #1933117) -- Mauricio Faria de Oliveira Mon, 25 Oct 2021 17:58:58 -0300 ** Changed in: ufw (Ubuntu Hirsute) Status: Fix Committed => Fix Released ** Changed in: ufw (Ubuntu Focal) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
This bug was fixed in the package ufw - 0.36-6ubuntu1 --- ufw (0.36-6ubuntu1) focal; urgency=medium * d/p/0012-set-default-policy-after-load.patch: fix boot stall on iscsi/network root filesystem when starting ufw (LP: #1946804) * d/p/0013-unconditionally-reload-with-delete.patch: fix corner case of rule deletion with specific/any proto (LP: #1933117) -- Mauricio Faria de Oliveira Mon, 25 Oct 2021 14:30:14 -0300 ** Changed in: ufw (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
This bug was fixed in the package ufw - 0.36.1-1ubuntu1 --- ufw (0.36.1-1ubuntu1) impish; urgency=medium * d/p/0004-set-default-policy-after-load.patch: fix boot stall on iscsi/network root filesystem when starting ufw (LP: #1946804) -- Mauricio Faria de Oliveira Mon, 25 Oct 2021 14:25:30 -0300 ** Changed in: ufw (Ubuntu Impish) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Released Status in ufw source package in Focal: Fix Released Status in ufw source package in Hirsute: Fix Released Status in ufw source package in Impish: Fix Released Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144]
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Tested with Bionic, Focal, Hirsute, and Impish with the test steps provided, on Oracle Cloud. All good. With the packages in -proposed, the system can reboot correctly. --- bionic Version: 0.36-0ubuntu0.18.04.2 focal Version: 0.36-6ubuntu1 hirsute Version: 0.36-7.1ubuntu1 impish Version: 0.36.1-1ubuntu1 ... With the packages in -updates, the system stalls on boot --- bionic Version: 0.36-0ubuntu0.18.04.1 focal Version: 0.36-6 hirsute Version: 0.36-7.1 impish Version: 0.36.1-1 ** Tags removed: verification-needed verification-needed-bionic verification-needed-focal verification-needed-hirsute verification-needed-impish ** Tags added: verification-done verification-done-bionic verification-done-focal verification-done-hirsute verification-done-impish -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Committed Status in ufw source package in Focal: Fix Committed Status in ufw source package in Hirsute: Fix Committed Status in ufw source package in Impish: Fix Committed Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Tested 0.36-0ubuntu0.18.04.2 on bionic. apt upgrade succeeded and after reboot the firewall came up with the expected rules in the expected order and I spot-checked allowed and deny traffic. I didn't test on an iSCSI system so won't add verification-done-focal at this time, but I think the testing is probably sufficient for that (I'll let others decide). -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Committed Status in ufw source package in Focal: Fix Committed Status in ufw source package in Hirsute: Fix Committed Status in ufw source package in Impish: Fix Committed Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Tested 0.36-6ubuntu1 on focal. apt upgrade succeeded and after reboot the firewall came up with the expected rules in the expected order and I spot-checked allowed and deny traffic. I didn't test on an iSCSI system so won't add verification-done-focal at this time, but I think the testing is probably sufficient for that (I'll let others decide). -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Committed Status in ufw source package in Focal: Fix Committed Status in ufw source package in Hirsute: Fix Committed Status in ufw source package in Impish: Fix Committed Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Hello Mauricio, or anyone else affected, Accepted ufw into hirsute-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ufw/0.36-7.1ubuntu1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- hirsute to verification-done-hirsute. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-hirsute. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: ufw (Ubuntu Hirsute) Status: In Progress => Fix Committed ** Tags added: verification-needed-hirsute ** Changed in: ufw (Ubuntu Focal) Status: In Progress => Fix Committed ** Tags added: verification-needed-focal -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Committed Status in ufw source package in Focal: Fix Committed Status in ufw source package in Hirsute: Fix Committed Status in ufw source package in Impish: Fix Committed Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Hello Mauricio, or anyone else affected, Accepted ufw into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ufw/0.36.1-1ubuntu1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-impish. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: ufw (Ubuntu Impish) Status: In Progress => Fix Committed ** Tags added: verification-needed verification-needed-impish -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: Fix Committed Status in ufw source package in Focal: Fix Committed Status in ufw source package in Hirsute: Fix Committed Status in ufw source package in Impish: Fix Committed Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Uploaded to I/H/F/B. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Patch added: "ufw-impish.debdiff" https://bugs.launchpad.net/ufw/+bug/1946804/+attachment/5536527/+files/ufw-impish.debdiff -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Patch added: "ufw-bionic.debdiff" https://bugs.launchpad.net/ufw/+bug/1946804/+attachment/5536530/+files/ufw-bionic.debdiff -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Patch added: "ufw-focal.debdiff" https://bugs.launchpad.net/ufw/+bug/1946804/+attachment/5536529/+files/ufw-focal.debdiff -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Patch added: "ufw-hirsute.debdiff" https://bugs.launchpad.net/ufw/+bug/1946804/+attachment/5536528/+files/ufw-hirsute.debdiff -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Verified test packages (ppa:mfo/lp1946804) for the Impish, Hirsute, Focal, and Bionic releases on Oracle Cloud's 'BM.Standard1.36' systems. (Impish/Hirsute: Focal and do-release-upgrade.) ... Without the patch, the system boot stalls. With the patch, the system boot continues. (Note: netfilter-persistent.service needed to be disabled, otherwise it flushes ufw's rules.) ... The output of `iptables -L -n` was the same with/without the patch. # diff iptables.before iptables.after; echo $? 0 # wc -l iptables.before iptables.after 170 iptables.before 170 iptables.after 340 total ... Versions tested (original/without patch) I: Version: 0.36.1-1 H: Version: 0.36-7.1 F: Version: 0.36-6 B: Version: 0.36-0ubuntu0.18.04.1 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. (eg, Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable * Observed: system may stall immediately if no prior iptables rules. (eg, iptables -A INPUT -p tcp -s 169.254.0.2 --sport 3260 -j ACCEPT) * Expected: system continues working. * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554]
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Description changed: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. [Test Steps] * Install Ubuntu on an iSCSI (or other network-based) root filesystem. -(e.g., Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) + (e.g., Oracle Cloud's bare-metal 'BM.Standard1.36' shape.) * sudo ufw enable + + * Observed: system may stall immediately if no prior iptables rules. + * Expected: system continues working. + + * sudo reboot * Observed: system boot stalls once ufw.service starts (see below.) * Expected: system boot should move on. [Regression Potential] * Potential regressions would be observed on ufw start/reload, when iptables rules are configured. * The resulting iptables configuration has been compared before/after the change, with identical rules on both. [Other Info] * Fixed in Debian and Jammy. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Description changed: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. + [Test Steps] - - Functional tests summary - - Attempted: 22 (3339 individual tests) - Skipped: 0 - Errors: 0 + * Install Ubuntu on an iSCSI (or other network-based) root filesystem. + + * sudo ufw enable + * sudo reboot + + * Observed: system boot stalls once ufw.service starts (see below.) + * Expected: system boot should move on. + + [Regression Potential] + + * Potential regressions would be observed on ufw start/reload, +when iptables rules are configured. + + * The resulting iptables configuration has been compared +before/after the change, with identical rules on both. [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline device [ 436.134354] sd 12:0:0:1: [sda] tag#105 CDB: Read(10) 28 00 00 05 8d d8 00 00 08 00 [ 436.198807]
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Tags added: sts sts-sponsor-mfo -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline device [ 436.134354] sd 12:0:0:1: [sda] tag#105 CDB: Read(10) 28 00 00 05 8d d8 00 00 08 00 [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Changed in: ufw (Ubuntu Bionic) Status: New => In Progress ** Changed in: ufw (Ubuntu Bionic) Importance: Undecided => Medium ** Changed in: ufw (Ubuntu Bionic) Assignee: (unassigned) => Mauricio Faria de Oliveira (mfo) ** Changed in: ufw (Ubuntu Focal) Status: New => In Progress ** Changed in: ufw (Ubuntu Focal) Importance: Undecided => Medium ** Changed in: ufw (Ubuntu Focal) Assignee: (unassigned) => Mauricio Faria de Oliveira (mfo) ** Changed in: ufw (Ubuntu Hirsute) Status: New => In Progress ** Changed in: ufw (Ubuntu Hirsute) Importance: Undecided => Medium ** Changed in: ufw (Ubuntu Hirsute) Assignee: (unassigned) => Mauricio Faria de Oliveira (mfo) ** Changed in: ufw (Ubuntu Impish) Status: New => In Progress ** Changed in: ufw (Ubuntu Impish) Importance: Undecided => Medium ** Changed in: ufw (Ubuntu Impish) Assignee: (unassigned) => Mauricio Faria de Oliveira (mfo) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: In Progress Status in ufw source package in Focal: In Progress Status in ufw source package in Hirsute: In Progress Status in ufw source package in Impish: In Progress Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result =
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
This bug was fixed in the package ufw - 0.36.1-2 --- ufw (0.36.1-2) unstable; urgency=medium [ Mauricio Faria de Oliveira ] * 0004-set-default-policy-after-load.patch: fix boot stall on iscsi/network root filesystem when starting ufw (LP: #1946804) [ Jamie Strandboge ] * rename python3-versions.diff as 0003-python3-versions.patch * debian/upstream/metadata: add Bug-Submit and Bug-Database -- Jamie Strandboge Wed, 13 Oct 2021 19:02:20 + ** Changed in: ufw (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: Fix Released Status in ufw source package in Bionic: New Status in ufw source package in Focal: New Status in ufw source package in Hirsute: New Status in ufw source package in Impish: New Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Ah, I hadn't checked that yet. Yes, please feel free to do the Impish SRU and the 0.36.1-2 that I just uploaded to Debian will float into 'J' after it opens. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: New Status in ufw source package in Bionic: New Status in ufw source package in Focal: New Status in ufw source package in Hirsute: New Status in ufw source package in Impish: New Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline device [ 436.134354] sd
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
MR for debian/master submitted [1]. Since Impish is in Final Freeze as of last week, this would fit a post-release SRU per [2] IIUIC, so a sync wouldn't be possible, I think. Since the devel/J series isn't open yet, perhaps just an Impish SRU is enough now, as the devel release will start from its packages in a bit? I'll check that, and get back to you. Thanks! [1] https://code.launchpad.net/~mfo/ufw/+git/ufw/+merge/410152 [2] https://lists.ubuntu.com/archives/ubuntu-devel-announce/2021-October/001301.html -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: New Status in ufw source package in Bionic: New Status in ufw source package in Focal: New Status in ufw source package in Hirsute: New Status in ufw source package in Impish: New Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1:
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Merge proposal linked: https://code.launchpad.net/~mfo/ufw/+git/ufw/+merge/410152 -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: New Status in ufw source package in Bionic: New Status in ufw source package in Focal: New Status in ufw source package in Hirsute: New Status in ufw source package in Impish: New Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline device [ 436.134354] sd 12:0:0:1: [sda] tag#105 CDB: Read(10) 28 00 00 05 8d d8 00 00 08 00
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
For Impish, lets update debian/master, then I'll upload there and sync to Ubuntu. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: New Status in ufw source package in Bionic: New Status in ufw source package in Focal: New Status in ufw source package in Hirsute: New Status in ufw source package in Impish: New Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline device [ 436.134354] sd 12:0:0:1: [sda] tag#105 CDB: Read(10) 28 00 00 05 8d d8 00 00 08 00 [
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
I merged the changes into master. Thanks Mauricio! ** Changed in: ufw Status: New => Fix Committed -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: Fix Committed Status in ufw package in Ubuntu: New Status in ufw source package in Bionic: New Status in ufw source package in Focal: New Status in ufw source package in Hirsute: New Status in ufw source package in Impish: New Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. This allows the system to boot due to the RELATED,ESTABLISHED rule, that is introduced by before.rules in INPUT/ufw-before-input chain. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline device [ 436.134354] sd 12:0:0:1: [sda] tag#105 CDB: Read(10) 28 00 00 05
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
** Description changed: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) - [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. + This allows the system to boot due to the RELATED,ESTABLISHED rule, + that is introduced by before.rules in INPUT/ufw-before-input chain. + The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. + Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 - [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal - [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. - Begin: Mounting root file system ... Begin: Running /scripts/local-top ... + Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd 12:0:0:1: [sda] tag#105 FAILED Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK cmd_age=169s [ 436.015520] sd 12:0:0:1: rejecting I/O to offline device [ 436.134354] sd 12:0:0:1: [sda] tag#105 CDB: Read(10) 28 00 00 05 8d d8 00 00 08 00 [ 436.198807] blk_update_request: I/O error, dev sda, sector 360816 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0 [ 436.198818] blk_update_request: I/O error, dev sda, sector 2324480 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 [ 436.198852] EXT4-fs warning (device sda1): htree_dirblock_to_tree:1004: inode #1398: lblock 0: comm systemd: error -5 reading directory block [ 436.290259] blk_update_request: I/O error, dev sda,
[Touch-packages] [Bug 1946804] Re: ufw breaks boot on network root filesystem
Merge Proposal submitted: https://code.launchpad.net/~mfo/ufw/+git/ufw/+merge/410091 ** Also affects: ufw (Ubuntu) Importance: Undecided Status: New ** Also affects: ufw (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: ufw (Ubuntu Impish) Importance: Undecided Status: New ** Also affects: ufw (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: ufw (Ubuntu Hirsute) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to ufw in Ubuntu. https://bugs.launchpad.net/bugs/1946804 Title: ufw breaks boot on network root filesystem Status in ufw: New Status in ufw package in Ubuntu: New Status in ufw source package in Bionic: New Status in ufw source package in Focal: New Status in ufw source package in Hirsute: New Status in ufw source package in Impish: New Bug description: [Impact] A system with rootfs on iSCSI stops booting when ufw.service starts. The kernel logs iSCSI command/reset timeout until I/O fails and the root filesystem/journal break. The issue is that ufw_start() sets the default policy _first_, then adds rules _later_. So, a default INPUT policy of DROP (default setting in ufw) prevents further access to the root filesystem (blocks incoming iSCSI traffic) thus any rules that could help are not loaded (nor anything else.) [Fix] The fix is to set default policy after loading rules in ufw_start(). That seems to be OK as `ip[6]tables-restore -n/--noflush` is used, and per iptables source, that only sets the chain policy. The comparison of `iptables -L` before/after shows no differences (verified on a local rootfs); `run_tests.sh` has 0 skipped/errors. Functional tests summary Attempted: 22 (3339 individual tests) Skipped: 0 Errors: 0 [ufw info] # ufw --version ufw 0.36 Copyright 2008-2015 Canonical Ltd. # lsb_release -cs focal [Boot Log] [ 232.168355] iBFT detected. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... Setting up software interface enp45s0f0np0 ... [ 254.644505] Loading iSCSI transport class v2.0-870. [ 254.714938] iscsi: registered transport (tcp) [ 254.780129] scsi host12: iSCSI Initiator over TCP/IP ... [ 255.433491] sd 12:0:0:1: [sda] 251658240 512-byte logical blocks: (129 GB/120 GiB) ... [ 256.379550] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) ... [ 266.620860] systemd[1]: Starting Uncomplicated firewall... Starting Uncomplicated firewall... ... [ 298.491560] session1: iscsi_eh_cmd_timed_out scsi cmd 310a6696 timedout [ 298.580803] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.656262] session1: iscsi_eh_cmd_timed_out scsi cmd 94ad9246 timedout [ 298.745237] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 298.745270] session1: iscsi_eh_abort aborting sc 310a6696 [ 298.899644] session1: iscsi_eh_abort aborting [sc 310a6696 itt 0x13] [ 298.985788] session1: iscsi_exec_task_mgmt_fn tmf set timeout [ 302.075554] session1: iscsi_eh_cmd_timed_out scsi cmd 1a9458b5 timedout [ 302.164786] session1: iscsi_eh_cmd_timed_out return shutdown or nh [ 314.107541] session1: iscsi_tmf_timedout tmf timedout [ 314.169797] connection1:0: detected conn error (1021) [ 314.232266] session1: iscsi_eh_abort abort failed [sc 310a6696 itt 0x13] [ 314.323531] session1: iscsi_eh_abort aborting sc 94ad9246 [ 314.399640] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.495578] session1: iscsi_eh_abort aborting sc 1a9458b5 [ 314.571554] session1: iscsi_eh_abort sc never reached iscsi layer or it completed. [ 314.664050] session1: iscsi_eh_device_reset LU Reset [sc 310a6696 lun 1] [ 314.755773] session1: iscsi_eh_device_reset dev reset result = FAILED [ 314.834736] session1: iscsi_eh_target_reset tgt Reset [sc 310a6696 tgt <...>] [ 314.954144] session1: iscsi_eh_target_reset tgt <...> reset result = FAILED [ 315.063456] connection1:0: detected conn error (1021) [ 315.125743] session1: iscsi_eh_session_reset wait for relogin [ 398.843556] INFO: task systemd:1 blocked for more than 120 seconds. ... [ 401.039006] INFO: task jbd2/sda1-8:2522 blocked for more than 123 seconds. ... [ 402.483917] INFO: task iptables-restor:2648 blocked for more than 124 seconds. ... [ 435.707549] session1: session recovery timed out after 120 secs [ 435.780058] session1: iscsi_eh_session_reset failing session reset: Could not log back into <...> [age 0] [ 435.920710] sd 12:0:0:1: Device offlined - not ready after error recovery [ 436.003563] sd
