This is a very old and long running issue OpenStack has had with KVM. In
essence, the KVM (libvirt) hypervisor lacks the ability to set the mount
points on instances and thus regardless of what nova instructs the
hypervisor to do KVM will simply attach the volume to the next available
drive mapping (vdb, vdc, vdd) at boot.

This is unfortunately expected behavior if you ask me.

** Changed in: nova
       Status: Confirmed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1593684

Title:
  Nova & Cinder - Disk attachement changes after shutoff/start

Status in OpenStack Compute (nova):
  Invalid

Bug description:
  Description
  ===========
  After shut off an instance and starting it again the disk path changes inside 
the virtual instance. This is usually a small issue if you are only using the 
disk label/or any other ID rather than disk path.
  If any application is using the disk path inside windows the application 
becamomes failed until replacing the device path.

  Also I understand that most people won't even notice the problem, but
  we don't have similar issues with VMware and we believe that the path
  should be the same regardless of any kind of shut off/startup
  operation.

  Steps to reproduce
  ==================
  - Deploy a new instance - eg: Windows 2012 R2 
  - Attach a new CEPH block device volume 
  - Verify the disk path inside Windows
      Device instance path: 
SCSI\DISK&VEN_RED_HAT&PROD_VIRTIO\4&1618751F&0&000000
        Parent: PCI\VEN_1AF4&DEV_1001&SUBSYS_00021AF4&REV_00\3&13c0b0c5&0&30
        Location: Bus Number 0, Target Id 0, LUN 0
  - Shut off the instance
  - Start the instance
  - The disk is offline after the startup, changing to online working
  - Verify the disk path:
        Device instance path: 
SCSI\DISK&VEN_RED_HAT&PROD_VIRTIO\4&352177D0&1&000000
        Parent: PCI\VEN_1AF4&DEV_1001&SUBSYS_00021AF4&REV_00\3&13c0b0c5&0&28
        Location: Bus Number 0, Target Id 0, LUN 0

  Using  Ubuntu 16.04 LTS:
  - Spin up a new Instance using Ubuntu 16.04 LTS
  - Attach a new CEPH block device
        root@attach-test-2:~# lsblk
        NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
        vda    253:0    0  16G  0 disk
        └─vda1 253:1    0  16G  0 part /
        vdb    253:16   0   1G  0 disk
        
        root@attach-test-2:~# ls -la /dev/disk/by-path
        total 0
        drwxr-xr-x 2 root root 100 Jun 17 11:26 .
        drwxr-xr-x 6 root root 120 Jun 17 11:26 ..
        lrwxrwxrwx 1 root root   9 Jun 17 11:23 virtio-pci-0000:00:04.0 -> 
../../vda
        lrwxrwxrwx 1 root root  10 Jun 17 11:23 virtio-pci-0000:00:04.0-part1 
-> ../../vda1
        lrwxrwxrwx 1 root root   9 Jun 17 11:26 virtio-pci-0000:00:06.0 -> 
../../vdb

  - Shut ioff the instance
  - Start the instance
  - Check disk path
        root@attach-test-2:~# ls -la /dev/disk/by-path
        total 0
        drwxr-xr-x 2 root root 100 Jun 17 11:28 .
        drwxr-xr-x 6 root root 120 Jun 17 11:28 ..
        lrwxrwxrwx 1 root root   9 Jun 17 11:28 virtio-pci-0000:00:04.0 -> 
../../vda
        lrwxrwxrwx 1 root root  10 Jun 17 11:28 virtio-pci-0000:00:04.0-part1 
-> ../../vda1
        lrwxrwxrwx 1 root root   9 Jun 17 11:28 virtio-pci-0000:00:05.0 -> 
../../vdb

  THe disk path changed using Linux too

  I have a feeling that this is a KVM/LibVirt issue...
        
        
  Expected result
  ===============
  Nova & Libvirt & KVM should use the same details when attaching the disk on 
the instance regardless how many time we shut it down and start it.
  So both instance path and parent should be unchanged.

  Actual result
  =============
  Path changes

  Environment
  ===========
  Control node:
  root@openstack1:/etc/nova# dpkg -l | grep nova
  ii nova-api 2:13.0.0-0ubuntu2 all OpenStack Compute - API frontend
  ii nova-cells 2:13.0.0-0ubuntu2 all Openstack Compute - cells
  ii nova-cert 2:13.0.0-0ubuntu2 all OpenStack Compute - certificate management
  ii nova-common 2:13.0.0-0ubuntu2 all OpenStack Compute - common files
  ii nova-conductor 2:13.0.0-0ubuntu2 all OpenStack Compute - conductor service
  ii nova-consoleauth 2:13.0.0-0ubuntu2 all OpenStack Compute - Console 
Authenticator
  ii nova-novncproxy 2:13.0.0-0ubuntu2 all OpenStack Compute - NoVNC proxy
  ii nova-scheduler 2:13.0.0-0ubuntu2 all OpenStack Compute - virtual machine 
scheduler
  ii python-nova 2:13.0.0-0ubuntu2 all OpenStack Compute Python libraries
  ii python-novaclient 2:3.3.1-2 all client library for OpenStack Compute API - 
Python 2.7

  Compute node1:
  root@openstackcompute:~# dpkg -l | grep nova
  ii nova-common 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - common files
  ii nova-compute 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node 
base
  ii nova-compute-kvm 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute 
node (KVM)
  ii nova-compute-libvirt 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - 
compute node libvirt support
  ii python-nova 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute Python libraries
  ii python-novaclient 2:3.3.1-2~cloud0 all client library for OpenStack 
Compute API - Python 2.7

  Compute node2:
  root@openstackcompute2:~# dpkg -l | grep nova
  ii nova-common 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - common files
  ii nova-compute 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute node 
base
  ii nova-compute-kvm 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - compute 
node (KVM)
  ii nova-compute-libvirt 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute - 
compute node libvirt support
  ii python-nova 2:13.0.0-0ubuntu2~cloud0 all OpenStack Compute Python libraries
  ii python-novaclient 2:3.3.1-2~cloud0 all client library for OpenStack 
Compute API - Python 2.7

  Compute node3:
  root@openstackcompute3:~# dpkg -l | grep nova
  ii nova-common 2:13.0.0-0ubuntu2 all OpenStack Compute - common files
  ii nova-compute 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node base
  ii nova-compute-kvm 2:13.0.0-0ubuntu2 all OpenStack Compute - compute node 
(KVM)
  ii nova-compute-libvirt 2:13.0.0-0ubuntu2 all OpenStack Compute - compute 
node libvirt support
  ii python-nova 2:13.0.0-0ubuntu2 all OpenStack Compute Python libraries
  ii python-novaclient 2:3.3.1-2 all client library for OpenStack Compute API - 
Python 2.7

  Hypervisors:
  KVM + Libvirt on all compute nodes

  Storage:
  CEPH, iSCSI and FC using Netapp

  CEPH version - the same on all storage nodes
  ii ceph 10.2.0-0ubuntu0.16.04.1 amd64 distributed storage and file system
  ii ceph-common 10.2.0-0ubuntu0.16.04.1 amd64 common utilities to mount and 
interact with a ceph storage cluster
  ii ceph-deploy 1.5.32-0ubuntu1 all Deployment and configuration of Ceph.
  ii ceph-mds 10.2.0-0ubuntu0.16.04.1 amd64 metadata server for the ceph 
distributed file system
  ii libcephfs1 10.2.0-0ubuntu0.16.04.1 amd64 Ceph distributed file system 
client library
  ii python-cephfs 10.2.0-0ubuntu0.16.04.1 amd64 Python libraries for the Ceph 
libcephfs library

  Networking
  Neutron networking with Openvswitch
     
  ----
  root@openstack1:~# nova show Attach_Test_Win2k12

  
+--------------------------------------+----------------------------------------------------------------------------------+
  | Property                             | Value                                
                                            |
  
+--------------------------------------+----------------------------------------------------------------------------------+
  | OS-DCF:diskConfig                    | AUTO                                 
                                            |
  | OS-EXT-AZ:availability_zone          | nova                                 
                                            |
  | OS-EXT-SRV-ATTR:host                 | openstackcompute3                    
                                            |
  | OS-EXT-SRV-ATTR:hostname             | attach-test-win2k12                  
                                            |
  | OS-EXT-SRV-ATTR:hypervisor_hostname  | openstackcompute3                    
                                            |
  | OS-EXT-SRV-ATTR:instance_name        | instance-00000114                    
                                            |
  | OS-EXT-SRV-ATTR:kernel_id            |                                      
                                            |
  | OS-EXT-SRV-ATTR:launch_index         | 0                                    
                                            |
  | OS-EXT-SRV-ATTR:ramdisk_id           |                                      
                                            |
  | OS-EXT-SRV-ATTR:reservation_id       | r-jiddtkfq                           
                                            |
  | OS-EXT-SRV-ATTR:root_device_name     | /dev/vda                             
                                            |
  | OS-EXT-SRV-ATTR:user_data            | -                                    
                                            |
  | OS-EXT-STS:power_state               | 1                                    
                                            |
  | OS-EXT-STS:task_state                | -                                    
                                            |
  | OS-EXT-STS:vm_state                  | active                               
                                            |
  | OS-SRV-USG:launched_at               | 2016-06-17T10:44:43.000000           
                                            |
  | OS-SRV-USG:terminated_at             | -                                    
                                            |
  | accessIPv4                           |                                      
                                            |
  | accessIPv6                           |                                      
                                            |
  | config_drive                         |                                      
                                            |
  | created                              | 2016-06-17T10:44:27Z                 
                                            |
  | description                          | Attach_Test_Win2k12                  
                                            |
  | flavor                               | CCL-Win2k12 
(b383ddeb-f52b-4554-94b6-49981bada108)                               |
  | hostId                               | 
c340bdd941f659cce547150bf407432536d9a856192a100ec3a2ebec                        
 |
  | host_status                          | UP                                   
                                            |
  | id                                   | 08c04ee9-b922-408d-8642-0e8f1fa59a1d 
                                            |
  | image                                | Windows 2012 R2 X64 
(86612cca-ce9f-4920-997d-6e81d9ea0068)                       |
  | key_name                             | CCLTests                             
                                            |
  | locked                               | False                                
                                            |
  | metadata                             | {}                                   
                                            |
  | name                                 | Attach_Test_Win2k12                  
                                            |
  | net-10.10.128 network                | 10.10.128.51                         
                                            |
  | os-extended-volumes:volumes_attached | [{"id": 
"32595603-6c42-4bcb-818d-53b0c93b7c06", "delete_on_termination": false}] |
  | progress                             | 0                                    
                                            |
  | security_groups                      | CCL                                  
                                            |
  | status                               | ACTIVE                               
                                            |
  | tenant_id                            | 6d0fa2c2b45a4f6c8aac185b4592d44f     
                                            |
  | updated                              | 2016-06-17T10:46:15Z                 
                                            |
  | user_id                              | 75f412ba1601457a8415519bbdd3a932     
                                            |
  
+--------------------------------------+----------------------------------------------------------------------------------+
  root@openstack1:~# openstack volume show 32595603-6c42-4bcb-818d-53b0c93b7c06
  
+---------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | Field                                 | Value                               
                                                                                
                                                                                
                                                                    |
  
+---------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
  | attachments                           | [{u'server_id': 
u'08c04ee9-b922-408d-8642-0e8f1fa59a1d', u'attachment_id': 
u'280a5b3b-599b-41a9-9793-ea6d774a1cb7', u'host_name': None, u'volume_id': 
u'32595603-6c42-4bcb-818d-53b0c93b7c06', u'device': u'/dev/vdb', u'id': 
u'32595603-6c42-4bcb-818d-53b0c93b7c06'}] |
  | availability_zone                     | nova                                
                                                                                
                                                                                
                                                                    |
  | bootable                              | false                               
                                                                                
                                                                                
                                                                    |
  | consistencygroup_id                   | None                                
                                                                                
                                                                                
                                                                    |
  | created_at                            | 2016-06-17T10:53:38.000000          
                                                                                
                                                                                
                                                                    |
  | description                           |                                     
                                                                                
                                                                                
                                                                    |
  | encrypted                             | False                               
                                                                                
                                                                                
                                                                    |
  | id                                    | 
32595603-6c42-4bcb-818d-53b0c93b7c06                                            
                                                                                
                                                                                
                        |
  | migration_status                      | None                                
                                                                                
                                                                                
                                                                    |
  | multiattach                           | False                               
                                                                                
                                                                                
                                                                    |
  | name                                  | Attachment_Test_1                   
                                                                                
                                                                                
                                                                    |
  | os-vol-host-attr:host                 | openstackblock.ccl-dev.net@rbd#CEPH 
                                                                                
                                                                                
                                                                    |
  | os-vol-mig-status-attr:migstat        | None                                
                                                                                
                                                                                
                                                                    |
  | os-vol-mig-status-attr:name_id        | None                                
                                                                                
                                                                                
                                                                    |
  | os-vol-tenant-attr:tenant_id          | 6d0fa2c2b45a4f6c8aac185b4592d44f    
                                                                                
                                                                                
                                                                    |
  | os-volume-replication:driver_data     | None                                
                                                                                
                                                                                
                                                                    |
  | os-volume-replication:extended_status | None                                
                                                                                
                                                                                
                                                                    |
  | properties                            | attached_mode='rw', 
readonly='False'                                                                
                                                                                
                                                                                
    |
  | replication_status                    | disabled                            
                                                                                
                                                                                
                                                                    |
  | size                                  | 1                                   
                                                                                
                                                                                
                                                                    |
  | snapshot_id                           | None                                
                                                                                
                                                                                
                                                                    |
  | source_volid                          | None                                
                                                                                
                                                                                
                                                                    |
  | status                                | in-use                              
                                                                                
                                                                                
                                                                    |
  | type                                  | CEPH                                
                                                                                
                                                                                
                                                                    |
  | user_id                               | 75f412ba1601457a8415519bbdd3a932    
                                                                                
                                                                                
                                                                    |
  
+---------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1593684/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to