[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** Tags added: cscc -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: Won't Fix Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2019-3462 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: Won't Fix Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** Changed in: linux (Ubuntu Trusty) Status: In Progress => Won't Fix -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: Won't Fix Status in linux source package in Xenial: Fix Released Status in linux source package in Yakkety: Fix Released Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
This bug was fixed in the package linux - 4.4.0-75.96 --- linux (4.4.0-75.96) xenial; urgency=low * linux: 4.4.0-75.96 -proposed tracker (LP: #1684441) * [Hyper-V] hv: util: move waiting for release to hv_utils_transport itself (LP: #1682561) - Drivers: hv: util: move waiting for release to hv_utils_transport itself linux (4.4.0-74.95) xenial; urgency=low * linux: 4.4.0-74.95 -proposed tracker (LP: #1682041) * [Hyper-V] hv: vmbus: Raise retry/wait limits in vmbus_post_msg() (LP: #1681893) - Drivers: hv: vmbus: Raise retry/wait limits in vmbus_post_msg() linux (4.4.0-73.94) xenial; urgency=low * linux: 4.4.0-73.94 -proposed tracker (LP: #1680416) * CVE-2017-6353 - sctp: deny peeloff operation on asocs with threads sleeping on it * vfat: missing iso8859-1 charset (LP: #1677230) - [Config] NLS_ISO8859_1=y * Regression: KVM modules should be on main kernel package (LP: #1678099) - [Config] powerpc: Add kvm-hv and kvm-pr to the generic inclusion list * linux-lts-xenial 4.4.0-63.84~14.04.2 ADT test failure with linux-lts-xenial 4.4.0-63.84~14.04.2 (LP: #1664912) - SAUCE: apparmor: fix link auditing failure due to, uninitialized var * regession tests failing after stackprofile test is run (LP: #1661030) - SAUCE: fix regression with domain change in complain mode * Permission denied and inconsistent behavior in complain mode with 'ip netns list' command (LP: #1648903) - SAUCE: fix regression with domain change in complain mode * unexpected errno=13 and disconnected path when trying to open /proc/1/ns/mnt from a unshared mount namespace (LP: #1656121) - SAUCE: apparmor: null profiles should inherit parent control flags * apparmor refcount leak of profile namespace when removing profiles (LP: #1660849) - SAUCE: apparmor: fix ns ref count link when removing profiles from policy * tor in lxd: apparmor="DENIED" operation="change_onexec" namespace="root//CONTAINERNAME_" profile="unconfined" name="system_tor" (LP: #1648143) - SAUCE: apparmor: Fix no_new_privs blocking change_onexec when using stacked namespaces * apparmor oops in bind_mnt when dev_path lookup fails (LP: #1660840) - SAUCE: apparmor: fix oops in bind_mnt when dev_path lookup fails * apparmor auditing denied access of special apparmor .null fi\ le (LP: #1660836) - SAUCE: apparmor: Don't audit denied access of special apparmor .null file * apparmor label leak when new label is unused (LP: #1660834) - SAUCE: apparmor: fix label leak when new label is unused * apparmor reference count bug in label_merge_insert() (LP: #1660833) - SAUCE: apparmor: fix reference count bug in label_merge_insert() * apparmor's raw_data file in securityfs is sometimes truncated (LP: #1638996) - SAUCE: apparmor: fix replacement race in reading rawdata * unix domain socket cross permission check failing with nested namespaces (LP: #1660832) - SAUCE: apparmor: fix cross ns perm of unix domain sockets * Xenial update to v4.4.59 stable release (LP: #1678960) - xfrm: policy: init locks early - virtio_balloon: init 1st buffer in stats vq - pinctrl: qcom: Don't clear status bit on irq_unmask - c6x/ptrace: Remove useless PTRACE_SETREGSET implementation - h8300/ptrace: Fix incorrect register transfer count - mips/ptrace: Preserve previous registers for short regset write - sparc/ptrace: Preserve previous registers for short regset write - metag/ptrace: Preserve previous registers for short regset write - metag/ptrace: Provide default TXSTATUS for short NT_PRSTATUS - metag/ptrace: Reject partial NT_METAG_RPIPE writes - fscrypt: remove broken support for detecting keyring key revocation - sched/rt: Add a missing rescheduling point - Linux 4.4.59 * Update ENA driver to 1.1.2 from net-next (LP: #1664312) - net: ena: Remove unnecessary pci_set_drvdata() - net: ena: Fix error return code in ena_device_init() - net: ena: change the return type of ena_set_push_mode() to be void. - net: ena: use setup_timer() and mod_timer() - net/ena: remove ntuple filter support from device feature list - net/ena: fix queues number calculation - net/ena: fix ethtool RSS flow configuration - net/ena: fix RSS default hash configuration - net/ena: fix NULL dereference when removing the driver after device reset failed - net/ena: refactor ena_get_stats64 to be atomic context safe - net/ena: fix potential access to freed memory during device reset - net/ena: use READ_ONCE to access completion descriptors - net/ena: reduce the severity of ena printouts - net/ena: change driver's default timeouts - net/ena: change condition for host attribute configuration - net/ena: update driver version to 1.1.2 * Xenial update to v4.4.58 stable release (LP: #1677600) - net/openvswitch: Set the ipv6 source tunnel key
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
This bug was fixed in the package linux - 4.8.0-49.52 --- linux (4.8.0-49.52) yakkety; urgency=low * linux: 4.8.0-49.52 -proposed tracker (LP: #1684427) * [Hyper-V] hv: util: move waiting for release to hv_utils_transport itself (LP: #1682561) - Drivers: hv: util: move waiting for release to hv_utils_transport itself linux (4.8.0-48.51) yakkety; urgency=low * linux: 4.8.0-48.51 -proposed tracker (LP: #1682034) * [Hyper-V] hv: vmbus: Raise retry/wait limits in vmbus_post_msg() (LP: #1681893) - Drivers: hv: vmbus: Raise retry/wait limits in vmbus_post_msg() linux (4.8.0-47.50) yakkety; urgency=low * linux: 4.8.0-47.50 -proposed tracker (LP: #1679678) * CVE-2017-6353 - sctp: deny peeloff operation on asocs with threads sleeping on it * CVE-2017-5986 - sctp: avoid BUG_ON on sctp_wait_for_sndbuf * vfat: missing iso8859-1 charset (LP: #1677230) - [Config] NLS_ISO8859_1=y * [Hyper-V] pci-hyperv: Use device serial number as PCI domain (LP: #1667527) - net/mlx4_core: Use cq quota in SRIOV when creating completion EQs * Regression: KVM modules should be on main kernel package (LP: #1678099) - [Config] powerpc: Add kvm-hv and kvm-pr to the generic inclusion list * linux-lts-xenial 4.4.0-63.84~14.04.2 ADT test failure with linux-lts-xenial 4.4.0-63.84~14.04.2 (LP: #1664912) - SAUCE: apparmor: fix link auditing failure due to, uninitialized var * regession tests failing after stackprofile test is run (LP: #1661030) - SAUCE: fix regression with domain change in complain mode * Permission denied and inconsistent behavior in complain mode with 'ip netns list' command (LP: #1648903) - SAUCE: fix regression with domain change in complain mode * unexpected errno=13 and disconnected path when trying to open /proc/1/ns/mnt from a unshared mount namespace (LP: #1656121) - SAUCE: apparmor: null profiles should inherit parent control flags * apparmor refcount leak of profile namespace when removing profiles (LP: #1660849) - SAUCE: apparmor: fix ns ref count link when removing profiles from policy * tor in lxd: apparmor="DENIED" operation="change_onexec" namespace="root//CONTAINERNAME_" profile="unconfined" name="system_tor" (LP: #1648143) - SAUCE: apparmor: Fix no_new_privs blocking change_onexec when using stacked namespaces * apparmor oops in bind_mnt when dev_path lookup fails (LP: #1660840) - SAUCE: apparmor: fix oops in bind_mnt when dev_path lookup fails * apparmor auditing denied access of special apparmor .null fi\ le (LP: #1660836) - SAUCE: apparmor: Don't audit denied access of special apparmor .null file * apparmor label leak when new label is unused (LP: #1660834) - SAUCE: apparmor: fix label leak when new label is unused * apparmor reference count bug in label_merge_insert() (LP: #1660833) - SAUCE: apparmor: fix reference count bug in label_merge_insert() * apparmor's raw_data file in securityfs is sometimes truncated (LP: #1638996) - SAUCE: apparmor: fix replacement race in reading rawdata * unix domain socket cross permission check failing with nested namespaces (LP: #1660832) - SAUCE: apparmor: fix cross ns perm of unix domain sockets * [Hyper-V][Mellanox] net/mlx4_core: Avoid delays during VF driver device shutdown (LP: #1672785) - Revert "net/mlx4_en: Avoid unregister_netdev at shutdown flow" - net/mlx4_core: Avoid delays during VF driver device shutdown * Update ENA driver to 1.1.2 from net-next (LP: #1664312) - net: ena: Remove unnecessary pci_set_drvdata() - net: ena: Fix error return code in ena_device_init() - net: ena: change the return type of ena_set_push_mode() to be void. - net: ena: use setup_timer() and mod_timer() - net/ena: remove ntuple filter support from device feature list - net/ena: fix queues number calculation - net/ena: fix ethtool RSS flow configuration - net/ena: fix RSS default hash configuration - net/ena: fix NULL dereference when removing the driver after device reset failed - net/ena: refactor ena_get_stats64 to be atomic context safe - net/ena: fix potential access to freed memory during device reset - net/ena: use READ_ONCE to access completion descriptors - net/ena: reduce the severity of ena printouts - net/ena: change driver's default timeouts - net/ena: change condition for host attribute configuration - net/ena: update driver version to 1.1.2 * ISST-LTE:pVM:roselp4:ubuntu16.04.2: number of numa_miss and numa_foreign wrong in numastat (LP: #1672953) - mm: fix remote numa hits statistics - mm: get rid of __GFP_OTHER_NODE * Using an NVMe drive causes huge power drain (LP: #1664602) - nvme/scsi: Remove power management support - nvme: Pass pointers, not dma addresses, to nvme_get/set_features() - nvme: introduce struct nvme_request - nvme: Add a quirk mechanis
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- yakkety' to 'verification-done-yakkety'. If the problem still exists, change the tag 'verification-needed-yakkety' to 'verification-failed- yakkety'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-yakkety -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed- xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed- xenial'. If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you! ** Tags added: verification-needed-xenial -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
The fixes for Xenial, Yakkety and Zesty are committed or released. However, I built one more Trusty test kernel with all of the prereq commits and the patches. Would it be possible to have this kernel tested one more time before it is SRU'd? It would be good to confirm no regressions are introduced in Trusty due to the number of changes needed. It can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1470250/trusty/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Trusty requires quite a few prereq commits(See comment #353), so it's SRU will be a little behind. However, it will be SRU'd shortly as well. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
How about rusty? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** Changed in: linux (Ubuntu Zesty) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: Fix Released Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Status in linux source package in Zesty: Fix Released Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** Changed in: linux (Ubuntu Xenial) Status: In Progress => Fix Committed ** Changed in: linux (Ubuntu Yakkety) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: Fix Committed Status in linux source package in Yakkety: Fix Committed Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Sorry guys I had been swamped with other stuff and then simply forgot to test @alexng's patch I ran it overnight on my original test setup and it also worked for me. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
The patch from @faulpeltz hasn't been mainlined because of some feedback that it shouldn't have to close and reopen the /dev/vmbus/hv_vss device after failure. I addressed this comment this in a modified version of @faulpeltz's patch (see comment #345). I haven't heard from @faulpeltz whether he's tested it. From my testing, it seems fine and I expect to resubmit it upstream. If we can make it a SAUCE patch, that would be nice. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
By the way, the "operation times should match host expectation" patch in #343 (and listed in #347) is upstream: commit b357fd3908c1191f2f56e38aa77f2aecdae18bc8 Author: Alex Ng Date: Sun Nov 6 13:14:11 2016 -0800 Drivers: hv: vss: Operation timeouts should match host expectation Increase the timeout of backup operations. When system is under I/O load, it needs more time to freeze. These timeout values should also match the host timeout values more closely. Signed-off-by: Alex Ng Signed-off-by: K. Y. Srinivasan Signed-off-by: Greg Kroah-Hartman -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
That patch was submitted upstream under the title "[PATCH] Tools: hv: recover after hv_vss_daemon freeze times out" but I don't see that it was committed. I'll poke around. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
The patch from @faulpeltz posted in comment #345 has never landed in mainline. Is there plans for this patch to be sent upstream, or do we just want to include it as a SAUCE patch? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I can confirm that the trusty patches are preventing the filesystem failure. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Just waiting on test results for Trusty per comments #352 and #353. It sounds like the test kernels resolved the issue? If that is the case, I'll submit the SRU request. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Hi @jsalisbury, Any status update on the patches for this issue? It appears the test kernels have resolved the issue. Let us know if you need additional testing or have questions about the patches. Thanks, Alex -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
So I'm the OP on this bug report from what seems like ages ago... While a lot of hard work as been done by some of the members posting here to try and replicate it, it's been very difficult to track the specific event that has been causing this.While we "fixed" our problem a long time ago by moving all the VM's effected to a server and disabling Hyper-V nightly snapshots, instead opting to shut down all VM's once a month on a weekend and shapshot the whole system and bring it back online, it hasn't been optimal. Lately, we've been upgrading the 14.04LTS systems one at a time to 16.04 with the latest kernel and dropping them back into active nightly backup systems and so far, have not seen a read-only file system failure with systems running the latest updates. Now these systems are not write heavy either, which tends to trigger this bug more often. At least we've seen some improvement in our environment just by keeping up to date with the latest distribution updates. Outside of this, microsoft has released server 2016 during our lifetime, and while we are not running, it would be interesting to see if that Hyper-V server OS is any better/worse then 2012R2. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Yeah an official fix would be nice. I am tired of getting up early to see if I have to reboot any vm's :) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
It's been more than 1.5 years! When can we expect this fix to be released? $ uname -a Linux Server123 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 14.04.5 LTS Release:14.04 Codename: trusty Jan 30 01:54:55 Server123 kernel: [78577.687381] cifs_vfs_err: 18 callbacks suppressed Jan 30 01:54:55 Server123 kernel: [78577.687389] CIFS VFS: open dir failed Jan 30 01:54:55 Server123 kernel: [78577.692074] CIFS VFS: open dir failed Jan 30 01:54:56 Server123 kernel: [78578.027096] CIFS VFS: open dir failed Jan 30 01:54:56 Server123 kernel: [78578.031664] CIFS VFS: open dir failed Jan 30 03:00:27 Server123 Hyper-V VSS: VSS: freeze of /archive: Success Jan 30 03:00:27 Server123 Hyper-V VSS: VSS: freeze of /: Success Jan 30 03:00:27 Server123 kernel: [82509.382621] hv_storvsc vmbus_0_1: cmd 0x28 scsi status 0x2 srb status 0x82 Jan 30 03:00:27 Server123 kernel: [82509.382730] hv_storvsc vmbus_0_1: stor pkt 8800d306d368 autosense data valid - len 18 Jan 30 03:00:27 Server123 kernel: [82509.382735] storvsc: Sense Key : Unit Attention [current] Jan 30 03:00:27 Server123 kernel: [82509.382741] storvsc: Add. Sense: Changed operating definition Jan 30 03:00:27 Server123 kernel: [82509.382811] sd 2:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Jan 30 03:00:27 Server123 Hyper-V VSS: VSS: thaw of /archive: Success Jan 30 03:00:27 Server123 Hyper-V VSS: VSS: thaw of /: Success Jan 30 03:00:57 Server123 kernel: [82539.424627] hv_storvsc vmbus_0_1: cmd 0x2a scsi status 0x2 srb status 0x82 Jan 30 03:00:57 Server123 kernel: [82539.424678] hv_storvsc vmbus_0_1: stor pkt 8800b89df468 autosense data valid - len 18 Jan 30 03:00:57 Server123 kernel: [82539.424683] storvsc: Sense Key : Unit Attention [current] Jan 30 03:00:57 Server123 kernel: [82539.424689] storvsc: Add. Sense: Changed operating definition Jan 30 03:00:57 Server123 kernel: [82539.424834] sd 2:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
3.13.0-103-generic #150~lp1470250 SMP looks promising. Several days without crash. I encountered some IO errors but the filesystem remains rw. [Wed Jan 11 13:47:35 2017] hv_storvsc vmbus_0_1: cmd 0x35 scsi status 0x2 srb status 0x82 [Wed Jan 11 13:47:35 2017] hv_storvsc vmbus_0_1: stor pkt 8800f0b0f728 autosense data valid - len 18 [Wed Jan 11 13:47:35 2017] storvsc: Sense Key : Unit Attention [current] [Wed Jan 11 13:47:35 2017] storvsc: Add. Sense: Changed operating definition [Wed Jan 11 13:47:35 2017] sd 2:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. [Wed Jan 11 13:47:35 2017] end_request: I/O error, dev sda, sector 0 [Wed Jan 11 13:48:56 2017] hv_storvsc vmbus_0_1: cmd 0x28 scsi status 0x2 srb status 0x82 [Wed Jan 11 13:48:56 2017] hv_storvsc vmbus_0_1: stor pkt 8800f0fa0668 autosense data valid - len 18 [Wed Jan 11 13:48:56 2017] storvsc: Sense Key : Unit Attention [current] [Wed Jan 11 13:48:56 2017] storvsc: Add. Sense: Changed operating definition [Wed Jan 11 13:48:56 2017] sd 2:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. [Wed Jan 11 14:07:56 2017] hv_storvsc vmbus_0_1: cmd 0x2a scsi status 0x2 srb status 0x82 [Wed Jan 11 14:07:56 2017] hv_storvsc vmbus_0_1: stor pkt 8800f0fa50a8 autosense data valid - len 18 [Wed Jan 11 14:07:56 2017] storvsc: Sense Key : Unit Attention [current] [Wed Jan 11 14:07:56 2017] storvsc: Add. Sense: Changed operating definition [Wed Jan 11 14:07:56 2017] sd 2:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~ker
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Microsoft will test the trusty kernel in the coming days. The Xenial kernel may be getting all the patches from the 4.9 rebase that's currently in progress. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Emsi and whoever can test Trusty. There is a test kernel available at: http://kernel.ubuntu.com/~jsalisbury/lp1470250/trusty This test kernel has the updated 3 patches. It also required 19 prerequisite patches. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
For those interested, the Trusty test kernel contains the following commits: e81d871 UBUNTU: SAUCE: (no-up) Tools: hv: vss: Thaw the filesystem and continue after freeze fails 01032c1 UBUNTU: SAUCE: (no-up) Drivers: hv: vss: Operation timeouts should match host expectation 4a6f680 Drivers: hv: utils: Continue to poll VSS channel after handling requests. 61c18ef Drivers: hv: kvp: fix IP Failover db70a1c Drivers: hv: utils: Invoke the poll function after handshake 3c8486d Drivers: hv: utils: run polling callback always in interrupt context e6e9811 Drivers: hv: fcopy: full handshake support 8f0d521 Drivers: hv: vss: full handshake support 8b167aa Tools: hv: vss: use misc char device to communicate with kernel 6712c24 Drivers: hv: kvp: convert to hv_utils_transport e5bd829 Drivers: hv: fcopy: convert to hv_utils_transport 9ec74ad Drivers: hv: vss: convert to hv_utils_transport a2adece Drivers: hv: util: introduce hv_utils_transport abstraction 7aa0716 Drivers: hv: fcopy: switch to using the hvutil_device_state state machine d804513 Drivers: hv: vss: switch to using the hvutil_device_state state machine 5c9bfa1 Drivers: hv: kvp: switch to using the hvutil_device_state state machine 11aef70 Drivers: hv: fcopy: rename fcopy_work -> fcopy_timeout_work 742f132 Drivers: hv: kvp: rename kvp_work -> kvp_timeout_work a5a8e26 Drivers: hv: kvp,vss: Fast propagation of userspace communication failure 77fbf9a Drivers: hv: vss: Introduce timeout for communication with userspace d1401e0 Tools: hv: vssdaemon: ignore the EBUSY on multiple freezing the same partition fcee4902 connector: add portid to unicast in addition to broadcasting -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Emsi, right now there are only Xenial and Yakkety test kernels with all the updated patches. I will have a Trusty test kernel available shortly. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
The link to trusty kernels is gone :( http://kernel.ubuntu.com/~jsalisbury/lp1470250/AlexFaulpeltzPatch/trusty/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I built Yakkety and Xenial test kernels with the three patches mentioned in comment #347. It would be great if these test kernels can be tested. If they resolve the bug, I'll submit and SRU request to have them included in Yakkety and Xenial. The test kernels can be downloaded from: Xenial: http://kernel.ubuntu.com/~jsalisbury/lp1470250/xenial/ Yakkety: http://kernel.ubuntu.com/~jsalisbury/lp1470250/yakkety The Trusty kernel requires quite a bit of prerequisites commits and backporting. I'll post a trusty test kernel once it's available. The Zesty kernel will pick up these patches when they land in mainline. The patches will be SRU'd to Zesty if they don't land in mainline in time for release. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Thanks Joseph for compiling the list. The patches you've outlined should be sufficient. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz, Thanks for the update. I'll build a test kernel with all the latest patches and backport them to prior releases. Just to confirm, so I build the test kernels with everything needed, we need the following: New patch from comment #343: Drivers: hv: vss: Operation timeouts should match host expectation The updated patch from @faulpeltz posted in comment #343 Tools: hv: vss: Thaw the filesystem and continue after freeze fails And lastly, Alex's patch, which has already landed in mainline: 497af84 Drivers: hv: utils: Continue to poll VSS channel after handling requests. I have not submitted an SRU request for these three patches yet. I just want to confirm all we need are these three patches, and have good testing feedback. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** Also affects: linux (Ubuntu Zesty) Importance: Critical Assignee: Joseph Salisbury (jsalisbury) Status: In Progress -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Status in linux source package in Zesty: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Alex I will try as soon as I have some spare time @jsalisbury Unfortunately I didnt have time to test the kernel from #341. Including the #343 upstream commit should take care of our issue, and using the patch from #345 (replacing my initial patch) should prevent any hv_vss_daemon crashes in extremely slow cases -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz I agree. Your patch should also be included in case a FREEZE operation does exceed the increased timeout. I've attached a modified version of your patch that addresses some concerns we had offline. Could you give it a try? ** Patch added: "0004-Tools-hv-vss-Thaw-the-filesystem-and-continue-after-.patch" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+attachment/4778824/+files/0004-Tools-hv-vss-Thaw-the-filesystem-and-continue-after-.patch -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Alex The modified timeout should take care of the issue, but I think its a good idea for the VSS daemon to issue a THAW before either exiting or trying to recover -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I'll let @jsalisbury comment on the status of his backported patches. @jsalisbury, I'd also take this recently submitted patch from upstream kernel as well. It ensures that the VSS driver doesn't timeout any long running FREEZE operations too early. This should preclude the need for faulpeltz's patch in most cases, unless the FREEZE operation takes extraordinarily long (> 15 mins). https://git.kernel.org/cgit/linux/kernel/git/next/linux- next.git/commit/drivers/hv?id=b357fd3908c1191f2f56e38aa77f2aecdae18bc8 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Any news on this? Recently it seemed like a fix was found, but no update for about a month now. Thanks guys! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I was able to backport the two patches to Trusty. It required quite a few prerequisite commits. The following commits were backported to a Trusty test kernel: f9cc88d lp1470250: patch from faulpeltz 6f3e8a2 Drivers: hv: utils: Continue to poll VSS channel after handling requests. 725a85d Drivers: hv: utils: fix a race on userspace daemons registration a0c12c6 Drivers: hv: kvp: fix IP Failover 51fea7d Drivers: hv: utils: Invoke the poll function after handshake 42fb309 Drivers: hv: utils: run polling callback always in interrupt context 505de58 Drivers: hv: fcopy: full handshake support 2ec9789 Drivers: hv: vss: full handshake support e12c519 Tools: hv: vss: use misc char device to communicate with kernel 1f3abc0 Drivers: hv: kvp: convert to hv_utils_transport d789589 Drivers: hv: fcopy: convert to hv_utils_transport 51ed5da Drivers: hv: vss: convert to hv_utils_transport edbff5d Drivers: hv: util: introduce hv_utils_transport abstraction 056fbb5 Drivers: hv: fcopy: switch to using the hvutil_device_state state machine 5eb0af4 Drivers: hv: vss: switch to using the hvutil_device_state state machine a170e78 Drivers: hv: kvp: switch to using the hvutil_device_state state machine a539598 Drivers: hv: fcopy: rename fcopy_work -> fcopy_timeout_work 7894b35 Drivers: hv: kvp: rename kvp_work -> kvp_timeout_work d3fc031 Drivers: hv: kvp,vss: Fast propagation of userspace communication failure ee85362 Drivers: hv: vss: Introduce timeout for communication with userspace d714c5a Tools: hv: vssdaemon: ignore the EBUSY on multiple freezing the same partition b957545 connector: add portid to unicast in addition to broadcasting The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1470250/AlexFaulpeltzPatch/trusty/ Can this kernel be tested to see if it resolves this bug? Note, that both the linux-image and linux-image-extra .deb packages need to be installed. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~ker
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Thanks @faulpeltz for the info. I sent you a private message with the list of maintainers to send the patch (trying to avoid pasting it here in case spam bots can go through this archive). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Alex 1) Yes, but I might need some help with that. Which list/maintainer should I submit it to? 2) On our test machine, with both the hyper-v host as well as the guest under heavy i/o load, it was a few hundred ms but with high variance and spiking (quite often) in to the 2-4s range, with some extreme values being 10s and more. I assume that this varies a lot with regards to the test hardware, especially storage. With light load (but not idle) this was well in the <100ms range but with occasional spikes. Another thing I noticed was that the freeze operation often took a lot longer (but not always) when triggered directly by the vss daemon through a host backup and not just from a standalone test program which just called FREEZE/THAW in a loop. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Hi @faulpeltz, A few questions/comments about your patch: 1) Can you submit your patch to the upstream kernel? 2) Under load, were you able to measure how long the FIFREEZE operation took before it succeeded? I'm trying to see if we can increase the timeout of the kernel driver before it hits the error condition that you encountered. Thanks, Alex -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
There should be a Trusty test kernel soon. I've had to identify 13 prereq commits to get the two patches to apply and build properly. For Xenial and newer, there has been positive test results from Emsi and faulpeltz with Alexs' patch and the one from faulpeltz. Are we comfortable with submitting an SRU request for those two patches, or do we think more testing is required. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** No longer affects: linux (Ubuntu Wily) ** No longer affects: linux (Ubuntu Vivid) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Thank you for the update. I'm staying tuned :) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I'm still working on getting a Trusty test kernel build with both patches. Alexs' patch is requiring some prereq commits to work with Trusty. I should have a test kernel ready shortly. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Thanks for the update. I'll rebuild the 14.04 test kernel, but this time with both patches. That is what the Xenial test kernel has. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
No luck with 14.04. After couple of days two test machines restarted unexpectedly at the very same moment. I sustepct kernel crash but the virtual console is gone and nothing was logged. On the other hand the xenial machine is working for weeks now without any issues. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Over 24h of trusty tests under heavy load and so good so far. IMHO the kernel patch should suffice. I don't see "trying to recover VSS connection" messages neither on 16.04 kernel nor 14.04. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz, have had a chance to test the Trusty kernel, posted in comment #323? That kernel only has your patch and not Alex's. It would be good to know if both are needed. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
We moved some machines back to their regular backup schedule with the new kernel, no problems so far -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Er, not "filesystem system" but "filesystem consistent". -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@trash1-z, Application-consistent backups will fall back to filesystem system backups as before as there is no API for informing Linux applications to complete operations for a backup. VSS has used filesystem freeze operations for years to do backups on Linux. Hyper-V integration services is integrated into the Ubuntu kernel through our collaboration with Canonical, and is quite up to date in Xenial. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
The xenial kernel works like a charm since the patch. Great work! :) I'm using application consistent backup (the Hyper-V signals snapshot to the guest OS) rather than crash consistent (snapshoting without informing the guest). The Hyper-h integration services in the guest are required for that (I'm using ubuntu cloud images that are preloaded with that). I'll test the trusty patches now. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I was able to patch the kernel, however the fault still exists. I'm uncertain if issue is related to VSS, but in ballpark (see screenshots). Ubuntu 16.04.1 on same server smooth as silk. ** Attachment added: "Ubuntu Error 4c.png" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+attachment/4742912/+files/Ubuntu%20Error%204c.png -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
You can install the .deb packages by using the dpkg command from a terminal. For example: sudo dpkg -i linux- image-3.13.0-95-generic_3.13.0-95.142~lp1470250faulpeltzPatchBAckported_amd64.deb -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I'd love to. Can you advise how I do that? I'm a good typist, but cut n paste is so much better! -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I backported the patch from faulpeltz to trusty and built a test kernel. The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1470250/trusty/ Can you give this kernel a test? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I'm really looking forward to test the path on trusty. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Yes. Linux backup-01 4.4.0-36-generic #55~lp1470250PatchedAlexFaulpeltz SMP Wed Aug 31 16:14:16 UTC 20 x86_64 x86_64 x86_64 GNU/Linux Another 24h and no crash. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Emsi, that is great news. Is this with the kernel that includes both the patch from Alex and the one from faulpeltz, which was posted in comment #317? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
It looks like it does the trick! Already over 90h of testing with tiobench and it works like a charm! Usually it took less than 4h (several at best) to crash. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Alex's patch is in commit http://git.kernel.org/cgit/linux/kernel/git/next/linux- next.git/commit/?id=497af84b81b98b27e9ba7aebb8a373412e328497 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I built a Xenial test kernel with the first patch from Alex and the patch from faulpeltz. The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1470250/AlexFaulpeltzPatch/xenial/ A little backporting is needed to get the patches to apply to Trusty, but I'll also post that kernel shortly. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz, that is great work creating the new patch. I'll build test kernels for all releases with the first patch from Alex and your patch posted in comment #313. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
** Tags added: patch -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Frederik: yes, as far as I have seen. The file systems are still frozen between FREEZE and THAW, which in the case of timeouts, is >10s, I have seen about 30s in some of our error cases. But they do recover. I only tried the patched 4.4.0-34 version for now, though Some testing would be appreciated by everyone here I think :) The kernel version I tried is ubuntu-xenial 4.4.0-34 with @alexng patch 1/2 from #308 and my patch -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpletz Do you mean with "and the guest systems continue to work normally" that the guest does not have any read-only problems any more with your patch? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I spent some time investigating our issue further. As far as I can tell, the main issue is that ioctl(FIFREEZE) can take a long time when running VSS backups, and the default timeout is 10s. This is very noticeable under load, with rare peaks of >5s seen, so 10s seem plausible If the timeout is hit in the kernel module, the hv_vss_daemon doesnt recover and quits, with the FS still frozen. This fixes some HV VSS daemon behavior where it doesnt recover on a write failed if the previous request timed out (e.g. THAW takes too long) We are currently running this patch including @AlexNg 's patch (1 of 2) in the usual backup loop We already hit the bug at least 5 times, which causes the VSS backup to fail, but subsequent backups work without problems, and the guest systems continue to work normally ** Patch added: "Experimental fix/workaround for hv_vss_daemon (based on 4.4.0-34)" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+attachment/4730147/+files/vss_timeout_fix.patch -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
One reason you may see the timeout messages is due to having a mismatch between the user-space hv_vss_daemon version and the kernel version. Can you rebuild the user-space hv_vss_daemon under the source tree's tools/hv directory and replace the one provided that's provided in Ubuntu by default? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
And in response to Joseph's comment #309, the second patch shouldn't be required as it's related to a feature introduced in Windows Server 2016 (I'm assuming you folks are testing in Windows Server 2012 R2). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
v4.8-rc2 (with patch 1/2 from #308) failed after 18h :( With the first patch applied, the VSS daemon decides to quit, but a THAW is missing after the FREEZE there are the usual syscall timeouts afterwards kernel: sd 2:0:0:0: [storvsc] Add. Sense: Changed operating definition kernel: sd 2:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automa kernel: hv_utils: VSS: timeout waiting for daemon to reply kernel: hv_utils: VSS: Transaction not active systemd[1]: hv-vss-daemon.service: Main process exited, code=exited, status=1/FAILURE unknown[2966]: Hyper-V VSS: VSS: op=FREEZE: succeeded systemd[1]: hv-vss-daemon.service: Unit entered failed state. unknown[2966]: Hyper-V VSS: write failed; error: 22 Invalid argument systemd[1]: hv-vss-daemon.service: Failed with result 'exit-code'. kernel: hv_utils: VSS: failed to communicate to the daemon: -22 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I built a v4.8-rc2 test kernel with the first patch Alex mentions in comment #308. Can this kernel be tested to see if it still exhibits the bug? The test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1470250/081816-Patch/ Alex, do you think we need the second patch for testing of this particular bug? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Might be worth trying this patchset: https://lkml.org/lkml/2016/8/18/859 The first patch in the set addresses some issues with VSS that would cause it to take a long time to initiate backup (and even may timeout). The second patch is not necessary (but will need VSS daemon to be replaced if you choose to apply it). -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
unfortunately still the same problem using the v4.8 tools (on 4.4 and up it doesnt remount the filesystem read-only, it just hangs on any write operation) dmesg output: [30626.788513] hv_utils: VSS: timeout waiting for daemon to reply [30627.100164] hv_utils: VSS: Transaction not active [30813.152039] INFO: task rs:main Q:Reg:1765 blocked for more than 120 seconds. [30813.152569] Not tainted 4.8.0-040800rc1-generic #201608072231 [30813.153094] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [30813.153627] rs:main Q:Reg D 8b44c2657fc0 0 1765 1 0x [30813.153630] 8b44c14d0300 8b44b2b2e9c0 854a7bc8 8b44b14a8000 [30813.153631] 8b44b14a7e78 8b44ae46c2c8 8b44ae46c2e0 [30813.153632] 8b44b14a7e10 94215131 8b44b2b2e9c0 94217a8a [30813.153633] Call Trace: [30813.153637] [] ? schedule+0x31/0x80 [30813.153639] [] ? rwsem_down_read_failed+0xfa/0x150 [30813.153640] [] ? call_rwsem_down_read_failed+0x14/0x30 [30813.153642] [] ? percpu_down_read+0x30/0x50 [30813.153644] [] ? __sb_start_write+0x26/0x30 [30813.153645] [] ? vfs_write+0x173/0x190 [30813.153646] [] ? SyS_write+0x52/0xc0 [30813.153647] [] ? entry_SYSCALL_64_fastpath+0x1e/0xa8 (repeats every 120 seconds) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz, I created athe tools and uploaded them to: http://kernel.ubuntu.com/~jsalisbury/lp1470250/v4.8-rc2/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
488347f seems stable after 25h 4.8-rc1 crashed/hung after ~7h, but I didnt have the 4.8 cloud tools (using the 4.4 ones), @jsalisbury: maybe you could build those? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@jsalisbury: Yes, as far as I can trust my results Currently running 488347f (which is reasonably close to 71425a9) If the current one doesnt fail I would test d215f91 next, then 71425a9 and the 4.8rc1 mainline -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz, so it sounds like we can confirm the latest commit to pass in 3.13.0-34 is 95d1181? If that is the case, it might be worthwhile for me to build a test kernel with d215f91 as tip of the tree and a second kernel with 71425a9 as the tip. Just let me know if you could test those two additional kernels and I'll build them. In addition, the v4.8-rc1 mainline kernel is now available. There is always the change, that this bug was fixed there and we can "Reverse" bisect. 4.8-rc1 is available here: http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.8-rc1/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Currently re-running a few test kernels. Current results: 3.13.0-35-generic #62-Commitd215f91Reverted: BAD 3.13.0-34-generic #61 @4c48c359b: GOOD 3.13.0-34-generic #61 @95d1181: GOOD -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@Tommy Eriksen, I believe the current status is that we have no idea. Every time when I think we are getting close on the cause of the problem, there comes proof that it is not the cause of the problems. For us it already critical for months. We are in the middle of moving our systems. I know @jsalisbury is doing the best he can and @faulpeltz is doing his best effort too, but the truth is that we are nowhere. It can still take months for the issue to be solved. Sorry for this message, but I think we should be clear on this. This problem can only be solved with extra (testing) manpower. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Hi Guys, Any news on the testing? We have an increasingly large number of servers failing after their backup run; it is getting a bit critical, unfortunately. Thanks a lot, Tommy -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Update (no real results yet) I tried to improve on the test cycle by stopping the hyper-v backup immediately after it has begun, then waiting until the delta disks have been merged back (rinse and repeat) It took some time to get stable but it seems to have a 6-7x speedup compared to the original variant. Currently running various kernels, unfortunately we have to rebuild parts of our test environment because it seems like cheap SSDs dont like 600TB writen to them... -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I built a 3.13.0-35.62 kernel with d215f91 reverted. It is available here: http://kernel.ubuntu.com/~jsalisbury/lp1470250/d215f91-reverted/3.13.0-35/ Can you test this kernel when you have a chance? If it runs past the 80 hour mark, I'd say let it run for a week if you can. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
The test kernel I built an post in comment #295 was Ubuntu-3.13.0-86.131 with d215f91 reverted. However, d215f91 was committed in Ubuntu-3.13.0-35.61. I wonder if it could be that the second bad commit is actually between 3.13.0-35.61 and 3.13.0-86.131. Can you continue to run 4c48c35 from the original bisect and see if it is good for a long period? As another test to see if there is a second bad commit added later in Trusty 3.13, I can build a 3.13.0-35.62 kernel with d215f91 reverted. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Unfortunately, yes. I did not find any evidence of other Hyper-V/Host related problems. I will try to repro the crash at least once to be sure. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
The Trusty test kernel with d215f91 reverted failed after 80 hours? That is the kernel that can be downloaded from here: http://kernel.ubuntu.com/~jsalisbury/lp1470250/d215f91-reverted/trusty/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Unfortunately it crashed after about 80 hours. I am currently running 4c48c35 from the original bisect (95h+) But things seem increasingly random at this point. I tweaked the IO load on the host and guest machine and it seems the crashes are now reproducible a bit faster, but I think we might need a few ideas on how to force the crashes. I already tried creating and deleting machine snapshots in a loop but that didn't do anything. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz, were you able to run the test kernel with d215f91 reverted for a long duration, like a week? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
i meant crashed after 12 hours *g* -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
3.14rc crashed after 3.14-rc1 However isn't the crash to be expected in all the mainline kernels >=3.14 since d215f91 has not been reverted in them? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Started run on 3.14-rc1 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
That would be a great test. I will also be out next week, so it would be good timing for a week long run. The reverted test kernel can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1470250/d215f91-reverted/trusty/ Before starting that test, can you test the upstream v3.14-rc1 kernel? It is available from: http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14-rc1-trusty/ -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Both crashed, 3.15rc1 after about 20h, 3.14 only after 66h I am starting to wonder if it might be a good idea to run the good(?) 3.13.0-86(+revert) kernel for a week or two to make sure its actually good. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
It's looking like the second bad commit came in the 3.14 or 3.15 kernel. Can you test the following two mainline kernels: 3.15-rc1: http://kernel.ubuntu.com/~kernel- ppa/mainline/v3.15-rc1-trusty/ 3.14 final: http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.14-trusty/ These won't have hte cloud tools built, but you should be able to use the 3.16 version like you used in comment #285. Let me know if you can't, and I'll see if I can build them. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Crashed after 30 and 12 hours. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Started run on 3.15, but there were no cloud tools in your build, so I used the linux-cloud-tools from 3.16.0-76 (jus copied the hv_* daemons over) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
I built a 3.15 based kernel, which can be downloaded from: http://kernel.ubuntu.com/~jsalisbury/lp1470250/3.15/ Can you give this kernel a test? It never had commit 89fb4cd applied, so it did not need to be reverted. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
Thanks again for testing! It seems like we now need to perform another round of bisecting between 3.13 and 3.16 since reverting all the commits mentioned in comment #279 did not resolve the bug. So what we need to do is revert 89fb4cd before each step on the bisect. I'll get started on building the first kernel now. We first need to find out if this second offending commit was introduced in 3.14 or 3.15. I'll post a test kernel shortly. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1470250] Re: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups
@faulpeltz I confirm the timing. I was never as specific as you are now. I already mentioned earlier that it always happens towards the end of the backup. I would not be surprised if the crash happens during the merge. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1470250 Title: [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based Backups Status in linux package in Ubuntu: In Progress Status in linux source package in Trusty: In Progress Status in linux source package in Vivid: In Progress Status in linux source package in Wily: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Yakkety: In Progress Bug description: Customers have reported running various versions of Ubuntu 14.04.2 LTS on Generation 2 Hyper-V Hosts.On a random Basis, the file system will be mounted Read-Only due to a "disk error" (which really isn't the case here).As a result, they must reboot the Ubuntu guest to get the file system to mount RW again. The Error seen are the following: Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt 88006eb6c700 autosense data valid - len 18 Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters. This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later. Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have. We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both. As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this. We can stop the backup process and all is good, but we need nightly backups to image all of our VM's. All the windows guests have no issues of course. We also have some CentOS based guests running without issues from what we've seen. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp