[Kernel-packages] [Bug 2008069] Re: IO stop running on NVME host when target is down
[Expired for linux (Ubuntu) because there has been no activity for 60 days.] ** Changed in: linux (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2008069 Title: IO stop running on NVME host when target is down Status in linux package in Ubuntu: Expired Bug description: Description: When running nvme with multipath enabled, it seems as host does not try to use another path. Steps to reproduce: Step 1: Install Ubuntu 22.04 LTS (ubuntu-22.04-live-server-amd64) on system. Step 2: Install nvme-cli package,currently installed version is nvme version 1.16 Step 3: Run IO on the nvme volume. step 4: Stop the service of the target controller. Expected Behavior: Upon stopping the service, host will try using another path for failed IO. from kernel log: Nov 11 08:58:10 localhost kernel: [88496.416058] sysfs: cannot create duplicate filename '/class/block/nvme0n1' Nov 11 08:58:10 localhost kernel: [88496.416062] CPU: 17 PID: 6358 Comm: kworker/u49:0 Kdump: loaded Tainted: GW 5.15.0-47-generic #51-Ubuntu Nov 11 08:58:10 localhost kernel: [88496.416067] Hardware name: Cisco Systems Inc UCSC-C240-M3S2/UCSC-C240-M3S2, BIOS C240M3.2.0.9a.0.113020152237 11/30/2015 Nov 11 08:58:10 localhost kernel: [88496.416069] Workqueue: nvme-wq nvme_scan_work [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416089] Call Trace: Nov 11 08:58:10 localhost kernel: [88496.416093] Nov 11 08:58:10 localhost kernel: [88496.416097] show_stack+0x52/0x5c Nov 11 08:58:10 localhost kernel: [88496.416107] dump_stack_lvl+0x4a/0x63 Nov 11 08:58:10 localhost kernel: [88496.416115] dump_stack+0x10/0x16 Nov 11 08:58:10 localhost kernel: [88496.416118] sysfs_warn_dup.cold+0x17/0x2b Nov 11 08:58:10 localhost kernel: [88496.416123] sysfs_do_create_link_sd+0xb7/0xd0 Nov 11 08:58:10 localhost kernel: [88496.416131] sysfs_create_link+0x21/0x40 Nov 11 08:58:10 localhost kernel: [88496.416135] device_add+0x2a3/0x7b0 Nov 11 08:58:10 localhost kernel: [88496.416141] ? dev_set_name+0x53/0x70 Nov 11 08:58:10 localhost kernel: [88496.416147] device_add_disk+0xfb/0x3c0 Nov 11 08:58:10 localhost kernel: [88496.416153] ? nvme_update_ns_info+0x14b/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416162] nvme_alloc_ns+0x1ff/0x400 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416172] nvme_validate_or_alloc_ns+0xcc/0x190 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416181] nvme_scan_ns_list+0x109/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416191] nvme_scan_work+0xd9/0x250 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416200] process_one_work+0x22b/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416205] worker_thread+0x53/0x420 Nov 11 08:58:10 localhost kernel: [88496.416207] ? process_one_work+0x3d0/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416210] kthread+0x12a/0x150 Nov 11 08:58:10 localhost kernel: [88496.416214] ? set_kthread_struct+0x50/0x50 Nov 11 08:58:10 localhost kernel: [88496.416218] ret_from_fork+0x22/0x30 Nov 11 08:58:10 localhost kernel: [88496.416225] Nov 11 09:00:06 localhost kernel: [88612.227916] nvme nvme2: starting error recovery Nov 11 09:00:06 localhost kernel: [88612.228115] nvme nvme2: queue 4 socket state 11 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2008069/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2008069] Re: IO stop running on NVME host when target is down
[Expired for linux (Ubuntu) because there has been no activity for 60 days.] ** Changed in: linux (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2008069 Title: IO stop running on NVME host when target is down Status in linux package in Ubuntu: Expired Bug description: Description: When running nvme with multipath enabled, it seems as host does not try to use another path. Steps to reproduce: Step 1: Install Ubuntu 22.04 LTS (ubuntu-22.04-live-server-amd64) on system. Step 2: Install nvme-cli package,currently installed version is nvme version 1.16 Step 3: Run IO on the nvme volume. step 4: Stop the service of the target controller. Expected Behavior: Upon stopping the service, host will try using another path for failed IO. from kernel log: Nov 11 08:58:10 localhost kernel: [88496.416058] sysfs: cannot create duplicate filename '/class/block/nvme0n1' Nov 11 08:58:10 localhost kernel: [88496.416062] CPU: 17 PID: 6358 Comm: kworker/u49:0 Kdump: loaded Tainted: GW 5.15.0-47-generic #51-Ubuntu Nov 11 08:58:10 localhost kernel: [88496.416067] Hardware name: Cisco Systems Inc UCSC-C240-M3S2/UCSC-C240-M3S2, BIOS C240M3.2.0.9a.0.113020152237 11/30/2015 Nov 11 08:58:10 localhost kernel: [88496.416069] Workqueue: nvme-wq nvme_scan_work [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416089] Call Trace: Nov 11 08:58:10 localhost kernel: [88496.416093] Nov 11 08:58:10 localhost kernel: [88496.416097] show_stack+0x52/0x5c Nov 11 08:58:10 localhost kernel: [88496.416107] dump_stack_lvl+0x4a/0x63 Nov 11 08:58:10 localhost kernel: [88496.416115] dump_stack+0x10/0x16 Nov 11 08:58:10 localhost kernel: [88496.416118] sysfs_warn_dup.cold+0x17/0x2b Nov 11 08:58:10 localhost kernel: [88496.416123] sysfs_do_create_link_sd+0xb7/0xd0 Nov 11 08:58:10 localhost kernel: [88496.416131] sysfs_create_link+0x21/0x40 Nov 11 08:58:10 localhost kernel: [88496.416135] device_add+0x2a3/0x7b0 Nov 11 08:58:10 localhost kernel: [88496.416141] ? dev_set_name+0x53/0x70 Nov 11 08:58:10 localhost kernel: [88496.416147] device_add_disk+0xfb/0x3c0 Nov 11 08:58:10 localhost kernel: [88496.416153] ? nvme_update_ns_info+0x14b/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416162] nvme_alloc_ns+0x1ff/0x400 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416172] nvme_validate_or_alloc_ns+0xcc/0x190 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416181] nvme_scan_ns_list+0x109/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416191] nvme_scan_work+0xd9/0x250 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416200] process_one_work+0x22b/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416205] worker_thread+0x53/0x420 Nov 11 08:58:10 localhost kernel: [88496.416207] ? process_one_work+0x3d0/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416210] kthread+0x12a/0x150 Nov 11 08:58:10 localhost kernel: [88496.416214] ? set_kthread_struct+0x50/0x50 Nov 11 08:58:10 localhost kernel: [88496.416218] ret_from_fork+0x22/0x30 Nov 11 08:58:10 localhost kernel: [88496.416225] Nov 11 09:00:06 localhost kernel: [88612.227916] nvme nvme2: starting error recovery Nov 11 09:00:06 localhost kernel: [88612.228115] nvme nvme2: queue 4 socket state 11 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2008069/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2008069] Re: IO stop running on NVME host when target is down
** Attachment added: "kern.log" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2008069/+attachment/5649139/+files/kern.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2008069 Title: IO stop running on NVME host when target is down Status in linux package in Ubuntu: Incomplete Bug description: Description: When running nvme with multipath enabled, it seems as host does not try to use another path. Steps to reproduce: Step 1: Install Ubuntu 22.04 LTS (ubuntu-22.04-live-server-amd64) on system. Step 2: Install nvme-cli package,currently installed version is nvme version 1.16 Step 3: Run IO on the nvme volume. step 4: Stop the service of the target controller. Expected Behavior: Upon stopping the service, host will try using another path for failed IO. from kernel log: Nov 11 08:58:10 localhost kernel: [88496.416058] sysfs: cannot create duplicate filename '/class/block/nvme0n1' Nov 11 08:58:10 localhost kernel: [88496.416062] CPU: 17 PID: 6358 Comm: kworker/u49:0 Kdump: loaded Tainted: GW 5.15.0-47-generic #51-Ubuntu Nov 11 08:58:10 localhost kernel: [88496.416067] Hardware name: Cisco Systems Inc UCSC-C240-M3S2/UCSC-C240-M3S2, BIOS C240M3.2.0.9a.0.113020152237 11/30/2015 Nov 11 08:58:10 localhost kernel: [88496.416069] Workqueue: nvme-wq nvme_scan_work [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416089] Call Trace: Nov 11 08:58:10 localhost kernel: [88496.416093] Nov 11 08:58:10 localhost kernel: [88496.416097] show_stack+0x52/0x5c Nov 11 08:58:10 localhost kernel: [88496.416107] dump_stack_lvl+0x4a/0x63 Nov 11 08:58:10 localhost kernel: [88496.416115] dump_stack+0x10/0x16 Nov 11 08:58:10 localhost kernel: [88496.416118] sysfs_warn_dup.cold+0x17/0x2b Nov 11 08:58:10 localhost kernel: [88496.416123] sysfs_do_create_link_sd+0xb7/0xd0 Nov 11 08:58:10 localhost kernel: [88496.416131] sysfs_create_link+0x21/0x40 Nov 11 08:58:10 localhost kernel: [88496.416135] device_add+0x2a3/0x7b0 Nov 11 08:58:10 localhost kernel: [88496.416141] ? dev_set_name+0x53/0x70 Nov 11 08:58:10 localhost kernel: [88496.416147] device_add_disk+0xfb/0x3c0 Nov 11 08:58:10 localhost kernel: [88496.416153] ? nvme_update_ns_info+0x14b/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416162] nvme_alloc_ns+0x1ff/0x400 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416172] nvme_validate_or_alloc_ns+0xcc/0x190 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416181] nvme_scan_ns_list+0x109/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416191] nvme_scan_work+0xd9/0x250 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416200] process_one_work+0x22b/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416205] worker_thread+0x53/0x420 Nov 11 08:58:10 localhost kernel: [88496.416207] ? process_one_work+0x3d0/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416210] kthread+0x12a/0x150 Nov 11 08:58:10 localhost kernel: [88496.416214] ? set_kthread_struct+0x50/0x50 Nov 11 08:58:10 localhost kernel: [88496.416218] ret_from_fork+0x22/0x30 Nov 11 08:58:10 localhost kernel: [88496.416225] Nov 11 09:00:06 localhost kernel: [88612.227916] nvme nvme2: starting error recovery Nov 11 09:00:06 localhost kernel: [88612.228115] nvme nvme2: queue 4 socket state 11 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2008069/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 2008069] Re: IO stop running on NVME host when target is down
** Description changed: - [ Impact ] - - Tab completion for the nvme command is completely broken in bash. - - [ Test Plan ] - - 1. Install nvme-cli package - 2. Open terminal - 3. type any nvme command and hit tab - - Tab completion should show entries like "list", "create-ns", etc. - - [ Where problems could occur ] - - The patch only changes the tab completion for nvme, but rebuilding nvme- - cli can reveal bugs on package upgrade, etc. - - [ Original report ] Description: When running nvme with multipath enabled, it seems as host does not try to use another path. Steps to reproduce: Step 1: Install Ubuntu 22.04 LTS (ubuntu-22.04-live-server-amd64) on system. Step 2: Install nvme-cli package,currently installed version is nvme version 1.16 - Step 3: Run IO on the nvme volume. + Step 3: Run IO on the nvme volume. step 4: Stop the service of the target controller. Expected Behavior: Upon stopping the service, host will try using another path for failed IO. from kernel log: Nov 11 08:58:10 localhost kernel: [88496.416058] sysfs: cannot create duplicate filename '/class/block/nvme0n1' Nov 11 08:58:10 localhost kernel: [88496.416062] CPU: 17 PID: 6358 Comm: kworker/u49:0 Kdump: loaded Tainted: GW 5.15.0-47-generic #51-Ubuntu Nov 11 08:58:10 localhost kernel: [88496.416067] Hardware name: Cisco Systems Inc UCSC-C240-M3S2/UCSC-C240-M3S2, BIOS C240M3.2.0.9a.0.113020152237 11/30/2015 Nov 11 08:58:10 localhost kernel: [88496.416069] Workqueue: nvme-wq nvme_scan_work [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416089] Call Trace: Nov 11 08:58:10 localhost kernel: [88496.416093] Nov 11 08:58:10 localhost kernel: [88496.416097] show_stack+0x52/0x5c Nov 11 08:58:10 localhost kernel: [88496.416107] dump_stack_lvl+0x4a/0x63 Nov 11 08:58:10 localhost kernel: [88496.416115] dump_stack+0x10/0x16 Nov 11 08:58:10 localhost kernel: [88496.416118] sysfs_warn_dup.cold+0x17/0x2b Nov 11 08:58:10 localhost kernel: [88496.416123] sysfs_do_create_link_sd+0xb7/0xd0 Nov 11 08:58:10 localhost kernel: [88496.416131] sysfs_create_link+0x21/0x40 Nov 11 08:58:10 localhost kernel: [88496.416135] device_add+0x2a3/0x7b0 Nov 11 08:58:10 localhost kernel: [88496.416141] ? dev_set_name+0x53/0x70 Nov 11 08:58:10 localhost kernel: [88496.416147] device_add_disk+0xfb/0x3c0 Nov 11 08:58:10 localhost kernel: [88496.416153] ? nvme_update_ns_info+0x14b/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416162] nvme_alloc_ns+0x1ff/0x400 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416172] nvme_validate_or_alloc_ns+0xcc/0x190 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416181] nvme_scan_ns_list+0x109/0x390 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416191] nvme_scan_work+0xd9/0x250 [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416200] process_one_work+0x22b/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416205] worker_thread+0x53/0x420 Nov 11 08:58:10 localhost kernel: [88496.416207] ? process_one_work+0x3d0/0x3d0 Nov 11 08:58:10 localhost kernel: [88496.416210] kthread+0x12a/0x150 Nov 11 08:58:10 localhost kernel: [88496.416214] ? set_kthread_struct+0x50/0x50 Nov 11 08:58:10 localhost kernel: [88496.416218] ret_from_fork+0x22/0x30 Nov 11 08:58:10 localhost kernel: [88496.416225] Nov 11 09:00:06 localhost kernel: [88612.227916] nvme nvme2: starting error recovery Nov 11 09:00:06 localhost kernel: [88612.228115] nvme nvme2: queue 4 socket state 11 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2008069 Title: IO stop running on NVME host when target is down Status in linux package in Ubuntu: Incomplete Bug description: Description: When running nvme with multipath enabled, it seems as host does not try to use another path. Steps to reproduce: Step 1: Install Ubuntu 22.04 LTS (ubuntu-22.04-live-server-amd64) on system. Step 2: Install nvme-cli package,currently installed version is nvme version 1.16 Step 3: Run IO on the nvme volume. step 4: Stop the service of the target controller. Expected Behavior: Upon stopping the service, host will try using another path for failed IO. from kernel log: Nov 11 08:58:10 localhost kernel: [88496.416058] sysfs: cannot create duplicate filename '/class/block/nvme0n1' Nov 11 08:58:10 localhost kernel: [88496.416062] CPU: 17 PID: 6358 Comm: kworker/u49:0 Kdump: loaded Tainted: GW 5.15.0-47-generic #51-Ubuntu Nov 11 08:58:10 localhost kernel: [88496.416067] Hardware name: Cisco Systems Inc UCSC-C240-M3S2/UCSC-C240-M3S2, BIOS C240M3.2.0.9a.0.113020152237 11/30/2015 Nov 11 08:58:10 localhost kernel: [88496.416069] Workqueue: nvme-wq nvme_scan_work [nvme_core] Nov 11 08:58:10 localhost kernel: [88496.416089] Call Trace: Nov 11 08:58:10 localhost kern
