[Yahoo-eng-team] [Bug 2013228] [NEW] Non-admin users unable to create SR-IOV switch dev ports
Public bug reported: Currently only admin users can create SR-IOV direct ports that are marked as switch_dev, i.e. the ones that do OVS hardware offload. If we use Neutron RBAC to allow access to the binding profile, users can create big problems by modifying it after port binding when nova has stored private details in their like the specific PCI device that has been bound. Example operator deployments: * all SR-IOV direct ports are hardware offloaded with switch_dev using ovn * all SR-IOV direct ports are hardware offloaded with switch_dev using ovs ml2 * all SR-IOV direct ports created using legacy SR-IOV driver * (not a use case I have, but theoretically its possible) some mix of the above >From and end user, the direct nic, hardware offloaded or not, by ovs ml2 or ovn, all appear the same. Its operator internal setup details on what happens. Why do we expose this to the end user in this way? Ideally the user doesn't care between these, and the neutron API is able to extract away the implementation details of getting a direct type of vnic. Moreover we don't want users to have to know how their operator has configured their system, be it ovn or ovs or sriov. They just want the direct NIC that get them RDMA within their VM for the requested nova flavor of their instance. This could be fixed by having ovn drivers and ovs ml2 drivers respect a configuration like this, for the non mixed case: bind_direct_nics_as_switch_dev = True/False At the PTG it was mentioned that this was configuration that change what the API does. But I don't understand how using ovn vs ovs ml2 is any different to hardware offloaded vs not-hardware offloads direct NICs. ** Affects: neutron Importance: Undecided Status: New ** Tags: rfe -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2013228 Title: Non-admin users unable to create SR-IOV switch dev ports Status in neutron: New Bug description: Currently only admin users can create SR-IOV direct ports that are marked as switch_dev, i.e. the ones that do OVS hardware offload. If we use Neutron RBAC to allow access to the binding profile, users can create big problems by modifying it after port binding when nova has stored private details in their like the specific PCI device that has been bound. Example operator deployments: * all SR-IOV direct ports are hardware offloaded with switch_dev using ovn * all SR-IOV direct ports are hardware offloaded with switch_dev using ovs ml2 * all SR-IOV direct ports created using legacy SR-IOV driver * (not a use case I have, but theoretically its possible) some mix of the above From and end user, the direct nic, hardware offloaded or not, by ovs ml2 or ovn, all appear the same. Its operator internal setup details on what happens. Why do we expose this to the end user in this way? Ideally the user doesn't care between these, and the neutron API is able to extract away the implementation details of getting a direct type of vnic. Moreover we don't want users to have to know how their operator has configured their system, be it ovn or ovs or sriov. They just want the direct NIC that get them RDMA within their VM for the requested nova flavor of their instance. This could be fixed by having ovn drivers and ovs ml2 drivers respect a configuration like this, for the non mixed case: bind_direct_nics_as_switch_dev = True/False At the PTG it was mentioned that this was configuration that change what the API does. But I don't understand how using ovn vs ovs ml2 is any different to hardware offloaded vs not-hardware offloads direct NICs. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2013228/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1974070] [NEW] Ironic builds fail when landing on a cleaning node, it doesn't try to reschedule
Public bug reported: In a happy world, placement reserved gets updated when a node is not availabe any more, so the scheduler doesn't pick that one, everyone it happy. Howerver, as is fairly well known, it takes a while for Nova to notice if a node has been marked as in maintenance or if it has started cleaning due to the instance now having been deleted, and you can still reach a node in a bad state. This actually fails hard when setting the instance uuid, as expected here: https://github.com/openstack/nova/blob/4939318649650b60dd07d161b80909e70d0e093e/nova/virt/ironic/driver.py#L378 You get a conflict errors, as the ironic node is in a transitioning state (i.e. its not actually available any more). When people are busy rebuilding large numbers of nodes, they tend to hit this problem, even when only building when you know there available nodes, you sometimes pick the ones you just deleted. In an idea world this would trigger a re-schedule, a bit like when you hit errors in the resource tracker such as ComputeResourcesUnavailable ** Affects: nova Importance: Low Assignee: John Garbutt (johngarbutt) Status: In Progress ** Changed in: nova Status: New => In Progress ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) ** Changed in: nova Importance: Undecided => Low -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1974070 Title: Ironic builds fail when landing on a cleaning node, it doesn't try to reschedule Status in OpenStack Compute (nova): In Progress Bug description: In a happy world, placement reserved gets updated when a node is not availabe any more, so the scheduler doesn't pick that one, everyone it happy. Howerver, as is fairly well known, it takes a while for Nova to notice if a node has been marked as in maintenance or if it has started cleaning due to the instance now having been deleted, and you can still reach a node in a bad state. This actually fails hard when setting the instance uuid, as expected here: https://github.com/openstack/nova/blob/4939318649650b60dd07d161b80909e70d0e093e/nova/virt/ironic/driver.py#L378 You get a conflict errors, as the ironic node is in a transitioning state (i.e. its not actually available any more). When people are busy rebuilding large numbers of nodes, they tend to hit this problem, even when only building when you know there available nodes, you sometimes pick the ones you just deleted. In an idea world this would trigger a re-schedule, a bit like when you hit errors in the resource tracker such as ComputeResourcesUnavailable To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1974070/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1891346] Re: Cannot delete nova-compute service due to service ID conflict
** Changed in: nova Status: New => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1891346 Title: Cannot delete nova-compute service due to service ID conflict Status in OpenStack Compute (nova): Won't Fix Bug description: I am trying to delete a nova-compute service for a retired hypervisor: $ openstack compute service delete 124 Failed to delete compute service with ID '124': Service id 124 refers to multiple services. (HTTP 400) (Request-ID: req-05e01880-237c-4efd-8c54-2899ccbf7316) 1 of 1 compute services failed to delete. This is caused by a conflicting service with the same ID in nova_cell0: MariaDB [nova_cell0]> SELECT * FROM services WHERE id = 124; +-+++-+---+---+---+--+--+-+-+--+-+-+--+ | created_at | updated_at | deleted_at | id | host | binary| topic | report_count | disabled | deleted | disabled_reason | last_seen_up | forced_down | version | uuid | +-+++-+---+---+---+--+--+-+-+--+-+-+--+ | 2020-05-27 18:43:34 | NULL | NULL | 124 | 172.16.52.246 | nova-metadata | NULL |0 |0 | 0 | NULL| NULL | 0 | 40 | cb03be2c-cd62-4d48-a2eb-424df70862c5 | +-+++-+---+---+---+--+--+-+-+--+-+-+--+ This service in cell0 appears to have been created at the time of an upgrade from Stein to Train. Environment === python2-novaclient-15.1.0-1.el7.noarch python2-nova-20.2.0-1.el7.noarch openstack-nova-api-20.2.0-1.el7.noarch openstack-nova-common-20.2.0-1.el7.noarch To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1891346/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1835400] [NEW] Issues booting with os_distro=centos7.0
Public bug reported: If we have os_distro=centos this isn't known by os-info, so we get: Cannot find OS information - Reason: (No configuration information found for operating system centos7): OsInfoNotFound: No configuration information found for operating system centos7 If we "fix" it to os_distro=centos7.0 we get: Instance failed to spawn: UnsupportedHardware: Requested hardware 'virtio1.0-net' is not supported by the 'kvm' virt driver This is with Rocky, but was also happening with Queens, I believe. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1835400 Title: Issues booting with os_distro=centos7.0 Status in OpenStack Compute (nova): New Bug description: If we have os_distro=centos this isn't known by os-info, so we get: Cannot find OS information - Reason: (No configuration information found for operating system centos7): OsInfoNotFound: No configuration information found for operating system centos7 If we "fix" it to os_distro=centos7.0 we get: Instance failed to spawn: UnsupportedHardware: Requested hardware 'virtio1.0-net' is not supported by the 'kvm' virt driver This is with Rocky, but was also happening with Queens, I believe. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1835400/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1525101] Re: floating ip info is not updated correctly when remove a fix ip which is associated to a floating ip
*** This bug is a duplicate of bug 1444966 *** https://bugs.launchpad.net/bugs/1444966 Because we have deprecated this API, I have marked it as will not fix. ** Changed in: nova Status: In Progress => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1525101 Title: floating ip info is not updated correctly when remove a fix ip which is associated to a floating ip Status in OpenStack Compute (nova): Won't Fix Bug description: [Summary] floating ip info is not updated correctly when remove a fix ip which is associated to a floating ip [Topo] devstack all-in-one node [Description and expect result] floating ip info can be updated correctly when remove a fix ip which is associated to a floating ip. [Reproduceable or not] reproduceable [Recreate Steps] 1) launch 1 instance: root@45-59:/opt/stack/devstack# nova list +--+--+++-+---+ | ID | Name | Status | Task State | Power State | Networks | +--+--+++-+---+ | 4608feb6-c825-46ae-8dcf-3d8839e51865 | inst | ACTIVE | - | Running | net2=2.0.0.30 | +--+--+++-+---+ 2) associate a floating ip to it: root@45-59:/opt/stack/devstack# nova floating-ip-associate --fixed-address 2.0.0.30 inst 172.168.0.10 root@45-59:/opt/stack/devstack# nova list +--+--+++-+-+ | ID | Name | Status | Task State | Power State | Networks| +--+--+++-+-+ | 4608feb6-c825-46ae-8dcf-3d8839e51865 | inst | ACTIVE | - | Running | net2=2.0.0.30, 172.168.0.10 | +--+--+++-+-+ 3)remove the fix ip of the instance, the fix ip and floating ip info can be removed when show "nova list": root@45-59:/opt/stack/devstack# nova remove-fixed-ip inst 2.0.0.30 root@45-59:/opt/stack/devstack# nova list +--+--+++-+--+ | ID | Name | Status | Task State | Power State | Networks | +--+--+++-+--+ | 4608feb6-c825-46ae-8dcf-3d8839e51865 | inst | ACTIVE | - | Running | | +--+--+++-+--+ 4) but if show the floating ip info via below cmd, the fix ip info still exist: ISSUE root@45-59:/opt/stack/devstack# neutron floatingip-show 46af70eb-e62b-471c-8694-0dc14060c372 +-+--+ | Field | Value| +-+--+ | fixed_ip_address| 2.0.0.30ISSUE| | floating_ip_address | 172.168.0.10 | | floating_network_id | c2592b59-f621-479c-8eaf-9b23f41c64d4 | | id | 46af70eb-e62b-471c-8694-0dc14060c372 | | port_id | 8c57f3ea-7cbf-4ad6-98d9-8c7a0d743bbb | | router_id | 37b26f1a-6086-4d64-bf8d-bd2ba27e5fee | | status | ACTIVE | | tenant_id | f75256da799642e0ab597a7533918714 | +-+--+ root@45-59:/opt/stack/devstack# 5)the issue results in another problem: the network info of the instance in dashboard is incorrect. [Configration] reproduceable bug, no need [logs] reproduceable bug, no need [Root cause anlyze or debug inf] reproduceable bug [Attachment] None To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1525101/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1724524] Re: Ironic nodes report CUSTOM_CUSTOM_FOO resource class
This is considered as designed, so will not be fixed. It complicates the general case of mapping from ironic resource classes to placement resource classes. I still think this is going to trip up many Ironic administrators. The docs are not clear that CUSTOM_FOO maps to CUSTOM_CUSTOM_FOO. ** No longer affects: nova/pike ** Changed in: nova Status: In Progress => Won't Fix ** Changed in: nova Assignee: John Garbutt (johngarbutt) => (unassigned) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1724524 Title: Ironic nodes report CUSTOM_CUSTOM_FOO resource class Status in OpenStack Compute (nova): Won't Fix Bug description: Currently if you set CUSTOM_FOO in ironic, the virt driver now sends CUSTOM_CUSTOM_FOO to placement. Really we shouldn't force users to drop the CUSTOM_ inside ironic. Expected: User sets CUSTOM_FOO in ironic. Placement shows CUSTOM_FOO resources. Actual: Placement shows CUSTOM_CUSTOM_FOO resources To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1724524/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1724589] [NEW] Unable to transition to Ironic Node Resource Classes in Pike
Public bug reported: In Pike we ask people to: * Update Ironic Node with a Resource Class * Update flavors to request the new Resource Class (and not request VCPU, RAM, DISK), using the docs: https://docs.openstack.org/ironic/latest/install/configure-nova-flavors.html#scheduling-based-on-resource-classes Consider this case: * some old instances are running from before the updates * some new instances are created after the updates In placement: * all inventory is correct, new resource class and legacy resource classes are both present * old instance allocations: only request In nova db: * old instances and new instances correctly request the new resource class in their flavor * new instances also include the anti-request for VCPU, DISK and RAM Now this is the flow that shows the problem: * get list of candidate allocations * this includes nodes that already have instances on (they only claim part of the inventory, but the new instance is only requesting the bit of the inventory the old instance isn't using) * boom, scheduling new instances fails after you hit the retry count, unless you got lucky and found a free slot by accident Possible reason for this: * Pike no longer updated instance allocations, if we updated the allocations of old instances to request the new custom resource class allocations, we would fix the above issue. Possible work around: * in the new flavor, keep requesting VCPU, RAM and CPU resources for pike, fix that up in queens? ** Affects: nova Importance: High Status: New ** Tags: ironic placement ** Tags added: ironic placement ** Changed in: nova Importance: Undecided => High -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1724589 Title: Unable to transition to Ironic Node Resource Classes in Pike Status in OpenStack Compute (nova): New Bug description: In Pike we ask people to: * Update Ironic Node with a Resource Class * Update flavors to request the new Resource Class (and not request VCPU, RAM, DISK), using the docs: https://docs.openstack.org/ironic/latest/install/configure-nova-flavors.html#scheduling-based-on-resource-classes Consider this case: * some old instances are running from before the updates * some new instances are created after the updates In placement: * all inventory is correct, new resource class and legacy resource classes are both present * old instance allocations: only request In nova db: * old instances and new instances correctly request the new resource class in their flavor * new instances also include the anti-request for VCPU, DISK and RAM Now this is the flow that shows the problem: * get list of candidate allocations * this includes nodes that already have instances on (they only claim part of the inventory, but the new instance is only requesting the bit of the inventory the old instance isn't using) * boom, scheduling new instances fails after you hit the retry count, unless you got lucky and found a free slot by accident Possible reason for this: * Pike no longer updated instance allocations, if we updated the allocations of old instances to request the new custom resource class allocations, we would fix the above issue. Possible work around: * in the new flavor, keep requesting VCPU, RAM and CPU resources for pike, fix that up in queens? To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1724589/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1724524] [NEW] Ironic nodes report CUSTOM_CUSTOM_FOO resource class
Public bug reported: Currently if you set CUSTOM_FOO in ironic, the virt driver now sends CUSTOM_CUSTOM_FOO to placement. Really we shouldn't force users to drop the CUSTOM_ inside ironic. Expected: User sets CUSTOM_FOO in ironic. Placement shows CUSTOM_FOO resources. Actual: Placement shows CUSTOM_CUSTOM_FOO resources ** Affects: nova Importance: High Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: placement -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1724524 Title: Ironic nodes report CUSTOM_CUSTOM_FOO resource class Status in OpenStack Compute (nova): In Progress Bug description: Currently if you set CUSTOM_FOO in ironic, the virt driver now sends CUSTOM_CUSTOM_FOO to placement. Really we shouldn't force users to drop the CUSTOM_ inside ironic. Expected: User sets CUSTOM_FOO in ironic. Placement shows CUSTOM_FOO resources. Actual: Placement shows CUSTOM_CUSTOM_FOO resources To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1724524/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1523668] Re: valid instance name before proceed
*** This bug is a duplicate of bug 1603728 *** https://bugs.launchpad.net/bugs/1603728 ** This bug has been marked a duplicate of bug 1603728 search with error regex causes a 500 error -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1523668 Title: valid instance name before proceed Status in OpenStack Compute (nova): In Progress Bug description: DEBUG (session:198) REQ: curl -g -i -X GET http://192.168.122.239:8774/v2.1/d1c5aa58af6c426492c642eb649017be/servers?name=%5Bu%27380e55e3-9726-4928-a44c-a206bc656f48%27%2C+u%27c426c3d0-a981-4839-969a-50d828e05459%27%5D -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.6" -H "X-Auth-Token: {SHA1}eb34c508b9ac1b119fabf1c5d8a0834f2ce32936" DEBUG (connectionpool:387) "GET /v2.1/d1c5aa58af6c426492c642eb649017be/servers?name=%5Bu%27380e55e3-9726-4928-a44c-a206bc656f48%27%2C+u%27c426c3d0-a981-4839-969a-50d828e05459%27%5D HTTP/1.1" 500 199 DEBUG (session:216) RESP: [500] Content-Length: 199 X-Compute-Request-Id: req-2590cdcb-7766-4b79-906e-404f8ba243de Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.6 Date: Mon, 07 Dec 2015 20:45:04 GMT Content-Type: application/json; charset=UTF-8 RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n", "code": 500}} DEBUG (shell:916) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. (HTTP 500) (Request-ID: req-2590cdcb-7766-4b79-906e-404f8ba243de) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1523668/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1669746] [NEW] sample config for host is unclear
Public bug reported: When we default the host and my_ip setting using oslo_config we end up reporting the details of the infra worker that generated the docs. We should make use of the sample default to add some text that looks better in the docs. In addition the description of host doesn't mention this is used as the oslo.messaging queue name for nova-compute worker. It is also used for the neutron bind host, so should match the host config of the neutorn agent. It is also used for the cinder host attachment information. For context, here is the current rendering of the conf: # # Hostname, FQDN or IP address of this host. Must be valid within AMQP key. # # Possible values: # # * String with hostname, FQDN or IP address. Default is hostname of this host. # (string value) #host = ubuntu-xenial-osic-cloud1-disk-7584065 Note there are other ones needing sample default to be set: # # The IP address which the host is using to connect to the management network. # # Possible values: # # * String with valid IP address. Default is IPv4 address of this host. # # Related options: # # * metadata_host # * my_block_storage_ip # * routing_source_ip # * vpn_ip # (string value) #my_ip = 10.29.14.104 ** Affects: nova Importance: Medium Status: New ** Tags: doc ** Tags added: doc ** Changed in: nova Importance: Undecided => Medium -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1669746 Title: sample config for host is unclear Status in OpenStack Compute (nova): New Bug description: When we default the host and my_ip setting using oslo_config we end up reporting the details of the infra worker that generated the docs. We should make use of the sample default to add some text that looks better in the docs. In addition the description of host doesn't mention this is used as the oslo.messaging queue name for nova-compute worker. It is also used for the neutron bind host, so should match the host config of the neutorn agent. It is also used for the cinder host attachment information. For context, here is the current rendering of the conf: # # Hostname, FQDN or IP address of this host. Must be valid within AMQP key. # # Possible values: # # * String with hostname, FQDN or IP address. Default is hostname of this host. # (string value) #host = ubuntu-xenial-osic-cloud1-disk-7584065 Note there are other ones needing sample default to be set: # # The IP address which the host is using to connect to the management network. # # Possible values: # # * String with valid IP address. Default is IPv4 address of this host. # # Related options: # # * metadata_host # * my_block_storage_ip # * routing_source_ip # * vpn_ip # (string value) #my_ip = 10.29.14.104 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1669746/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1607350] Re: floating-ip info doesn't contain information about instance if associated (with nova network installation)
While this bug is correct, this feature has now been deprecated, and is bug frozen. ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1607350 Title: floating-ip info doesn't contain information about instance if associated (with nova network installation) Status in OpenStack Compute (nova): Invalid Bug description: [Summary] floating ip info does not contain information about associated instance if nova-network is used. behaviour was changed between 11.05.16 and 21.06.16 [Topo] devstack all-in-one node libvirt+qemu nova-network [Description and expect result] floating ip info contains information about associated instance as in previous releases. [Reproduceable or not] reproduceable [Recreate Steps] 0) source any credentials. Result is the same for demo credentials of devstack (user=demo project=demo) and for admin credentials. 1) boot instance nova boot --image cirros-0.3.4-x86_64-uec --flavor 1 ttt 2) create floating ip nova floating-ip-create 3) associate floating-ip nova floating-ip-associate ttt 172.24.4.1 4) list intsances nova list +--+--+++-+--+ | ID | Name | Status | Task State | Power State | Networks | +--+--+++-+--+ | 8ad61db0-f388-4bc7-bfbd-728782a5b505 | ttt | ACTIVE | - | Running | private=10.0.0.4, 172.24.4.1 | +--+--+++-+--+ 5) list floating ips nova floating-ip-list +++---+--++ | Id | IP | Server Id | Fixed IP | Pool | +++---+--++ | 1 | 172.24.4.1 | - | -| public | +++---+--++ [Root cause anlyze or debug inf] - database contains information about floating ip and record has a correct id of fixed ip - database contains informtaion about fixed ip and record has a correct instance uuid nova 'os-floating-ips' rest api calls network_api.get_floating_ips_by_project it calls objects.FloatingIPList.get_by_project it retrieves floating ips from DB and calls obj_base.obj_make_list for each record obj_make_list calls _from_db_object of passed type and creates FloatingIP object _from_db_object takes 'fixed_ip' as expected attributes but only FloatingIP.get_by_id passes it. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1607350/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1662626] [NEW] live-migrate left in migrating as domain not found
017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] guest.delete_configuration() 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] File "/openstack/venvs/nova-14.0.4/lib/python2.7/site-packages/nova/virt/libvirt/guest.py", line 266, in delete_configuration 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] self._domain.undefine() 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] File "/openstack/venvs/nova-14.0.4/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in doit 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] File "/openstack/venvs/nova-14.0.4/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in proxy_call 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] rv = execute(f, *args, **kwargs) 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] File "/openstack/venvs/nova-14.0.4/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in execute 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] six.reraise(c, e, tb) 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] File "/openstack/venvs/nova-14.0.4/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in tworker 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] rv = meth(*args, **kwargs) 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] File "/openstack/venvs/nova-14.0.4/lib/python2.7/site-packages/libvirt.py", line 2701, in undefine 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] if ret == -1: raise libvirtError ('virDomainUndefine() failed', dom=self) 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] libvirtError: Domain not found: no domain with matching uuid '62034d78-3144-4efd-9c2c-8a792aed3d6b' (instance-0431) 2017-02-05 02:33:46.364 19770 ERROR nova.compute.manager [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] ** Affects: nova Importance: Undecided Assignee: John Garbutt (johngarbutt) Status: New ** Tags: live-migration -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1662626 Title: live-migrate left in migrating as domain not found Status in OpenStack Compute (nova): New Bug description: A live-migration stress test was working fine when suddenly a VM stopped migrating. It failed with this error: ERROR nova.virt.libvirt.driver [req-df91ac40-820f-4aa9-945b- b2fce73461f8 29c0371e35f84fdaa033f2dbfe2c042c 669472610b194bfa9bf03f50f86d725a - - -] [instance: 62034d78-3144-4efd- 9c2c-8a792aed3d6b] Error from libvirt during undefine. Code=42 Error=Domain not found: no domain with matching uuid '62034d78-3144 -4efd-9c2c-8a792aed3d6b' (instance-0431) The full stack trace: 2017-02-05 02:33:41.787 19770 INFO nova.virt.libvirt.driver [req-df91ac40-820f-4aa9-945b-b2fce73461f8 29c0371e35f84fdaa033f2dbfe2c042c 669472610b194bfa9bf03f50f86d725a - - -] [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] Migration running for 240 secs, memory 9% remaining; (bytes processed=15198240264, remaining=1680875520, total=17314955264) 2017-02-05 02:33:45.795 19770 INFO nova.compute.manager [req-abff9c69-5f82-4ed6-af8a-fd1dc81a72a6 - - - - -] [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] VM Paused (Lifecycle Event) 2017-02-05 02:33:45.870 19770 INFO nova.compute.manager [req-abff9c69-5f82-4ed6-af8a-fd1dc81a72a6 - - - - -] [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] During sync_power_state the instance has a pending task (migrating). Skip. 2017-02-05 02:33:45.883 19770 INFO nova.virt.libvirt.driver [req-df91ac40-820f-4aa9-945b-b2fce73461f8 29c0371e35f84fdaa033f2dbfe2c042c 669472610b194bfa9bf03f50f86d725a - - -] [instance: 62034d78-3144-4efd-9c2c-8a792aed3d6b] Migration operation has completed 2017-02-05 02:33:45.884 19770 INFO nova.compute.manager [req-df91ac40-820f-4aa9-945b-b2fce73461f8 29c0371e35f84fdaa033f2dbfe2c042c 669472610b194bfa9bf03f50f86d725a - - -] [instance: 62034d78-3144-4efd-9c2
[Yahoo-eng-team] [Bug 1580967] Re: nova limits does not show keypair count properly
So the problem here is keypairs are owned by the user, and not owned by the tenant. The API you mention lists what a tenant has used, and any limits that apply to the tenant. So in this case, the API is saying for every user in the tenant they are allowed 100 keypairs. In a similar way, every server group is allowed 10 members. In neither case does it make sense to say how much the tenant has used, because the usage is not related to a tenant. As such, this bug is invalid. ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1580967 Title: nova limits does not show keypair count properly Status in OpenStack Compute (nova): Invalid Bug description: The keypair count with nova limits is the number of keypairs associated with the VMs, But it has to be number of keypair created for that tenant regardless of it is being used or not. $ nova limits ++--+---+ | Name | Used | Max | ++--+---+ | Cores | 0| 20| | FloatingIps| 0| 10| | ImageMeta | -| 128 | | Instances | 0| 10| | Keypairs | -| 100 | | Personality| -| 5 | | Personality Size | -| 10240 | | RAM| 0| 51200 | | SecurityGroupRules | -| 20| | SecurityGroups | 0| 10| | Server Meta| -| 128 | | ServerGroupMembers | -| 10| | ServerGroups | 0| 10| ++--+---+ There is a keypair created though it is not associated with any instance, It has to be counted when we do nova limits as for nova it is used. $nova keypair-list | Name | Type | Fingerprint | +--+--+-+ | test | ssh | c8:e8:3e:8f:98:89:18:90:80:c5:55:f9:21:49:59:d9 | +--+--+-+ The reverse happens for security groups, For security groups It is the number of security groups created in nova regardless of it is used or not. Which I feel is expected behaviour. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1580967/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1580552] Re: Developer Guide: Nova API v2 is deprecated
As per my previous comments, the v2.0 API is not deprecated. Once version of the code behind the API was deprecated and has been removed. The Public API should be just as it always was, and should keep working. ** Changed in: nova Status: In Progress => Won't Fix ** Changed in: nova Status: Won't Fix => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1580552 Title: Developer Guide: Nova API v2 is deprecated Status in OpenStack Compute (nova): Invalid Bug description: Compute API v2 is deprecated but http://docs.openstack.org/developer/nova/#welcome-to-nova-s-developer- documentation still says that v2 API is supported. Both v2 API and v2 extensions are deprecated. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1580552/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1631918] [NEW] _determine_version_cap fails with MessagingTimeout when starting nova-compute
Public bug reported: On a fresh deployment, there are issues when starting up nova-compute, before nova-conductor has started responding to RPC requests. The first is a MessagingTimeout in the _determine_version_cap call, that is triggered by creating the ComputeManager class. This cases the process to exit, but it doesn't seem to quite fully exit the process. It seems like this happens only when CONF.upgrade_levels.compute = "auto" This was spotted in this OSA change: https://review.openstack.org/#/c/367752 ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Status: New => In Progress ** Description changed: On a fresh deployment, there are issues when starting up nova-compute, before nova-conductor has started responding to RPC requests. The first is a MessagingTimeout in the _determine_version_cap call, that is triggered by creating the ComputeManager class. This cases the process to exit, but it doesn't seem to quite fully exit the process. + + It seems like this happens only when CONF.upgrade_levels.compute = + "auto" + + This was spotted in this OSA change: + https://review.openstack.org/#/c/367752 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1631918 Title: _determine_version_cap fails with MessagingTimeout when starting nova- compute Status in OpenStack Compute (nova): In Progress Bug description: On a fresh deployment, there are issues when starting up nova-compute, before nova-conductor has started responding to RPC requests. The first is a MessagingTimeout in the _determine_version_cap call, that is triggered by creating the ComputeManager class. This cases the process to exit, but it doesn't seem to quite fully exit the process. It seems like this happens only when CONF.upgrade_levels.compute = "auto" This was spotted in this OSA change: https://review.openstack.org/#/c/367752 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1631918/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1616240] Re: Traceback in vif.py execv() arg 2 must contain only strings
So sbezverk_ confirmed the issue was the --config-dir that priv-sep was passing to the worker. I am looking into a fix for that now. ** Also affects: oslo.privsep Importance: Undecided Status: New ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1616240 Title: Traceback in vif.py execv() arg 2 must contain only strings Status in OpenStack Compute (nova): Invalid Status in oslo.privsep: New Bug description: While bringing up VM with the latest master (August 23,2016) I see this traceback and VM fails to launch. Complete log is here: http://paste.openstack.org/show/562688/ nova.conf used is here: http://paste.openstack.org/show/562757/ The issue is 100% reproducible in my testbed. 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [req-81060644-0dd7-453c-a68c-0d9cffe28fe7 3d1cd826f71a49cc81b33e85329f94b3 f738285a670c4be08d8a5e300aa25504 - - -] [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] Instance failed to spawn 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] Traceback (most recent call last): 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 2075, in _build_resources 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] yield resources 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/compute/manager.py", line 1919, in _build_and_run_instance 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] block_device_info=block_device_info) 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2583, in spawn 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] post_xml_callback=gen_confdrive) 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4803, in _create_domain_and_network 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] self.plug_vifs(instance, network_info) 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 684, in plug_vifs 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] self.vif_driver.plug(instance, vif) 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 801, in plug 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] self._plug_os_vif(instance, vif_obj) 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] File "/var/lib/kolla/venv/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 783, in _plug_os_vif 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] raise exception.NovaException(msg) 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] NovaException: Failure running os_vif plugin plug method: Failed to plug VIF VIFBridge(active=False,address=fa:16:3e:c0:4a:fd,bridge_name='qbrb7b522a4-3f',has_traffic_filtering=True,id=b7b522a4-3faa-42ca-8e0f-d8c241432e1f,network=Network(f32fdde6-bb99-4981-926b-a7df30f0a612),plugin='ovs',port_profile=VIFPortProfileBase,preserve_on_delete=True,vif_name='tapb7b522a4-3f'). Got error: execv() arg 2 must contain only strings 2016-08-23 17:17:28.941 8808 ERROR nova.compute.manager [instance: 95f11702-9e64-445d-a3cd-2cde074a4219] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1616240/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1599111] Re: HTTP exception thrown: Unexpected API Error
nova-docker is not part of upstream nova, or supported by upstream Nova. Moving to the nova-docker project. ** Also affects: nova-docker Importance: Undecided Status: New ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1599111 Title: HTTP exception thrown: Unexpected API Error Status in OpenStack Compute (nova): Invalid Status in nova-docker: New Bug description: Description === Exception is thrown when creating container from nova. Steps to reproduce == 1. create a container image in glance glance image-create --container-format=docker --disk-format=raw --name ubuntu 2. create a container nova boot --flavor m1.small --image ubuntu ubuntucontainer Expected result === Container should be created Actual result = API Error Environment === 1. Nova version commit b9d757bc0429159a235a397c51d510bd40e19709 Merge: 44db7db 566bdf1 Author: Jenkins Date: Wed Apr 13 03:21:25 2016 + Merge "Remove unused parameter from _get_requested_instance_group" 2. Nova-docker version commit 034a4842fc1ebba5912e02cff8cd197ae81eb0c3 Author: zhangguoqing Date: Mon May 23 12:17:07 2016 + add active_migrations attribute to DockerDriver 1. For passing the nova unit tests about the active_migrations attribute. 2. Fix test_get_dns_entries DNS IPs that changed from nova. 3. Add conf path to netconf that changed from nova. Closes-Bug: #1584741 Closes-Bug: #1582615 Change-Id: Iaab7e695055f042b9060f07e31681c66197b8c79 3. Glance version commit bded216e10b07735a09077f0d4f4901e963c83b5 Author: OpenStack Proposal Bot Date: Tue Apr 12 23:08:25 2016 + Updated from global requirements Change-Id: I706c9ea19e8ab2c49ce748bba31ae03dd0ec6d74 4. Compute Driver compute_driver=novadocker.virt.docker.DockerDriver Logs 2016-07-05 15:52:02.835 DEBUG nova.api.openstack.wsgi [req-a956f02f-ef9f-4309-8c4d-fa3ab355bd5a admin admin] Calling method '>' from (pid=30686) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:699 2016-07-05 15:52:03.451 ERROR nova.api.openstack.extensions [req-a956f02f-ef9f-4309-8c4d-fa3ab355bd5a admin admin] Unexpected exception in API method 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions Traceback (most recent call last): 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/opt/stack/nova/nova/api/openstack/extensions.py", line 478, in wrapped 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions return f(*args, **kwargs) 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 87, in show 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions image = self._image_api.get(context, id) 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/opt/stack/nova/nova/image/api.py", line 93, in get 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions show_deleted=show_deleted) 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/opt/stack/nova/nova/image/glance.py", line 283, in show 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions include_locations=include_locations) 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/opt/stack/nova/nova/image/glance.py", line 513, in _translate_from_glance 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions include_locations=include_locations) 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/opt/stack/nova/nova/image/glance.py", line 597, in _extract_attributes 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions output[attr] = getattr(image, attr) or 0 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/glanceclient/openstack/common/apiclient/base.py", line 490, in __getattr__ 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions self.get() 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions File "/usr/local/lib/python2.7/dist-packages/glanceclient/openstack/common/apiclient/base.py", line 512, in get 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions {'x_request_id': self.manager.client.last_request_id}) 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions AttributeError: 'HTTPClient' object has no attribute 'last_request_id' 2016-07-05 15:52:03.451 TRACE nova.api.openstack.extensions To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1599111/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@list
[Yahoo-eng-team] [Bug 1537625] Re: invalid path for plugin skeleton in api_plugins.rst
This doc should not be fixed, it has now been removed. ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1537625 Title: invalid path for plugin skeleton in api_plugins.rst Status in OpenStack Compute (nova): Invalid Bug description: In doc/source/api_plugins.rst invalid path [1] is given for plugin skeleton example which is not exist. [1] https://github.com/openstack/nova/blob/master/doc/source/api_plugins.rst #basic-plugin-structure To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1537625/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1558503] Re: Flavor m1.tiny could not be found Exception while creating instance
So I think this is actually python-novaclient. It checks to see if its a valid id, and correctly gets 404. This seems like the correct/expected behaviour. ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1558503 Title: Flavor m1.tiny could not be found Exception while creating instance Status in OpenStack Compute (nova): Invalid Bug description: While creating instance using "nova boot --flavor ", Instance is created successfully, but in /var/log/nova/nova-api.log file we find the following error log:- HTTP exception thrown: Flavor m1.tiny could not be found. "GET /v2/cf66e8c655474008a8c1fc088665df83/flavors/m1.tiny HTTP/1.1" status: 404 len: 298 time: 0.0457311 By Logs we can see :- 1. nova-api first tries to find the flavor using flavor name and then throw the exception. 2. then nova-api tries to find the flavor using flavor id. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1558503/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1504725] Re: rabbitmq-server restart twice, log is crazy increasing until service restart
** Changed in: nova Status: Confirmed => Invalid ** No longer affects: nova -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1504725 Title: rabbitmq-server restart twice, log is crazy increasing until service restart Status in neutron: New Status in oslo.messaging: Won't Fix Bug description: After I restart the rabbitmq-server for the second time, the service log(such as nova,neutron and so on) is increasing crazy, log is such as " TypeError: 'NoneType' object has no attribute '__getitem__'". It seems that the channel is setted to None. trace log: 2015-10-10 15:20:59.413 29515 TRACE root Traceback (most recent call last): 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 95, in inner_func 2015-10-10 15:20:59.413 29515 TRACE root return infunc(*args, **kwargs) 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/oslo_messaging/_executors/impl_eventlet.py", line 96, in _executor_thread 2015-10-10 15:20:59.413 29515 TRACE root incoming = self.listener.poll() 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 122, in poll 2015-10-10 15:20:59.413 29515 TRACE root self.conn.consume(limit=1, timeout=timeout) 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1202, in consume 2015-10-10 15:20:59.413 29515 TRACE root six.next(it) 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 1100, in iterconsume 2015-10-10 15:20:59.413 29515 TRACE root error_callback=_error_callback) 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py", line 868, in ensure 2015-10-10 15:20:59.413 29515 TRACE root ret, channel = autoretry_method() 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 458, in _ensured 2015-10-10 15:20:59.413 29515 TRACE root return fun(*args, **kwargs) 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 545, in __call__ 2015-10-10 15:20:59.413 29515 TRACE root self.revive(create_channel()) 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/kombu/connection.py", line 251, in channel 2015-10-10 15:20:59.413 29515 TRACE root chan = self.transport.create_channel(self.connection) 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 91, in create_channel 2015-10-10 15:20:59.413 29515 TRACE root return connection.channel() 2015-10-10 15:20:59.413 29515 TRACE root File "/usr/lib/python2.7/site-packages/amqp/connection.py", line 289, in channel 2015-10-10 15:20:59.413 29515 TRACE root return self.channels[channel_id] 2015-10-10 15:20:59.413 29515 TRACE root TypeError: 'NoneType' object has no attribute '__getitem__' 2015-10-10 15:20:59.413 29515 TRACE root To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1504725/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1533867] Re: In cell mode and latest kilo code, nova get-vnc-console throw 500 error
Given this is for cells, and cells is now largely frozen code, marking as opinion. ** Changed in: nova Status: Confirmed => Opinion ** Tags added: cells -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1533867 Title: In cell mode and latest kilo code, nova get-vnc-console throw 500 error Status in OpenStack Compute (nova): Opinion Bug description: We are using kilo version of nova (commit id b8c4f1bce356838dd3dac3b59734cf47f72373e5). Setup 3 cells with their own rabbitmq and mysql. Try nova get-vnc-console vm_id, got 500 error and error in compute side complain nova.api.openstack AttributeError: 'dict' object has no attribute 'uuid' After dive into it, the message compute received from AMQ was not been serialized to instance object but to a dict. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1533867/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1483236] Re: It´s not possible to disable the nova compute service on one compute node
So that behaviour is as designed. Disabled only stops new instances landing on that host. You can still manage your existing instances. The intent is to allow an admin to migrate or live-migrate instances off the host. I think we probably want some extra level of disabled to signal that existing VMs should not bar allowed to have any action performed. FWIW stopping nova-compute might be what you want to do. That would stop instance from running again. In fact if you configure nova correctly, when you start nova-compute it will start back up only the VMs that were running when it shutdown. ** Changed in: nova Importance: Undecided => Wishlist ** Changed in: nova Status: Confirmed => Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1483236 Title: It´s not possible to disable the nova compute service on one compute node Status in OpenStack Compute (nova): Opinion Bug description: I have multiple nova compute nodes and for maintenance reasons I want to disable one of them. Migration of a virtual machine is not possible, due to bug https://bugs.launchpad.net/cinder/+bug/1348811, so it would be ok, if it is not possible, to start a machine on that compute node. I stopped virtual machine "foo" on compute node "bar" $ nova stop 86db11f7-a1e4-4be0-880c-0720cd24c3aa disabled nova-compute service $ sudo nova-manage service disable bar nova-compute and checked it with $ sudo nova-manage service list --host bar -- Binary Host Zone Status State Updated_At nova-compute barCluster Xdisabled :-) 2015-08-10 12:29:10 -- Expected result: VM foo is not able to run on that host Actual result: VM foo is able to run on that host $ nova start 86db11f7-a1e4-4be0-880c-0720cd24c3aa The machine is running I tested, if its possible to create new virtual machines on that compute node bar with the disabled nova-compute service $ nova boot --image "c196615d-d3ed-49d7-8bc6-c65f8360aa02" --flavor m1.smaller --nic net-id=6a38402f-4a5b-4239-a683-ceb4ae220c30 --availability-zone "Cluster X":bar foo2 This machine is also running. Versions: $ dpkg -l | grep nova ii nova-common 1:2014.1.4-0ubuntu2.1 all OpenStack Compute - common files ii nova-compute1:2014.1.4-0ubuntu2.1 all OpenStack Compute - compute node base ii nova-compute-kvm1:2014.1.4-0ubuntu2.1 all OpenStack Compute - compute node (KVM) ii nova-compute-libvirt1:2014.1.4-0ubuntu2.1 all OpenStack Compute - compute node libvirt support ii python-nova 1:2014.1.4-0ubuntu2.1 all OpenStack Compute Python libraries ii python-novaclient 1:2.17.0-0ubuntu1.2 all client library for OpenStack Compute API $ dpkg -l | grep ceph ii ceph-common 0.94.1-1trusty amd64common utilities to mount and interact with a ceph storage cluster ii ceph-fs-common 0.94.1-1trusty amd64common utilities to mount and interact with a ceph file system ii ceph-fuse 0.94.1-1trusty amd64FUSE-based client for the Ceph distributed file system ii libcephfs1 0.94.1-1trusty amd64Ceph distributed file system client library ii python-cephfs 0.94.1-1trusty amd64Python libraries for the Ceph libcephfs library To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1483236/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1482029] Re: Delete "vm that has snapshot" spend more time
This is too old to be useful, let's kill this one. ** Changed in: nova Status: Confirmed => Incomplete ** Changed in: nova Status: Incomplete => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1482029 Title: Delete "vm that has snapshot" spend more time Status in OpenStack Compute (nova): Invalid Bug description: We are now using ceph as nova backend in version juno/stable. In the following scene, deleting instance will take more time than icehouse. 1. Create an instance. 2. Wait for instance becoming "Active" status. 3. Wait for instance having fixed ip address. 4. Create an snapshot from the new instance. 5. Wait for image snapshot becoming "Active" status. 6. Delete the instance created in step 1. in icehouse, we can delete instance in 10 seconds. Recently we upgrade to juno , the time increased to 30~40 seconds which was discovered by our Monitoring System。 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1482029/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1475655] Re: Unit_add call fails for fcp volumes when target port has not been configured
This sounds like something that needs fixing in the appropriate os-brick logic for system z ** Also affects: os-brick Importance: Undecided Status: New ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1475655 Title: Unit_add call fails for fcp volumes when target port has not been configured Status in OpenStack Compute (nova): Invalid Status in os-brick: New Bug description: Linux on System z can be configured for automated port and LUN scanning. If both features are turned off, ports and LUNs need to be added using explicit calls. While os-brick currently uses explicit calls to add LUNs, the calls for adding ports are missing. If an administrator does not manually issue the port_rescan call to add fibre-channel target ports, OpenStack will fail to add any fibre-channel LUN on System z. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1475655/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1475256] Re: sriov: VFs attributes (vlan, mac address) are not cleaned up after port delete
Looks like this got fixed in libvirt. ** Tags added: neutron sriov ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1475256 Title: sriov: VFs attributes (vlan, mac address) are not cleaned up after port delete Status in OpenStack Compute (nova): Invalid Bug description: Image we create a port like this: $ neutron port-create --binding:vnic_type=direct --name rjuly013 sriovtest0 Created a new port: +---+-+ | Field | Value | +---+-+ | admin_state_up| True | | allowed_address_pairs | | | binding:host_id | | | binding:profile | {} | | binding:vif_details | {} | | binding:vif_type | unbound | | binding:vnic_type | direct | | device_id | | | device_owner | | | fixed_ips | {"subnet_id": "ffa84ccf-ba49-4a23-a8ab-9295bc7d93f2", "ip_address": "166.168.0.15"} | | id| 2ec3b30e-e3cf-4a8f-a7cb-68a910a59e9a | | mac_address | fa:16:3e:ca:11:87 | | name | rjuly013 | | network_id| 26a0f22b-42b0-41d2-9b76-41270ce9b655 | | security_groups | b0ef012a-96b2-458f-bd28-c46306f063fa | | status| DOWN | | tenant_id | 2ebabf166ecd43dd8093b70a37f26be4 | +---+-+ $ And then create a VM with this port: $ nova boot --image 3c3a5387-7471-4e88-a19e-09e0c9a08707 --flavor 3 --nic port-id=2ec3b30e-e3cf-4a8f-a7cb-68a910a59e9a rjuly013 Now we can see a VF configured: $ ip link|grep fa:16:3e:ca:11:87 vf 7 MAC fa:16:3e:ca:11:87, spoof checking on, link-state auto $ After deletion of VM, we can see that the VF is still configured: $ ip link|grep fa:16:3e:ca:11:87 vf 7 MAC fa:16:3e:ca:11:87, spoof checking on, link-state auto $ This situation could cause troubles, for example, if user would want to create a new port with the mac address of the removed port, and if a port would be allocated on the same PF, there would be 2 VFs with the same MAC address in result. This could cause an unexpected behavior, with 'ixgbe' at least. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1475256/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1463911] Re: IPV6 fragmentation and mtu issue
** No longer affects: nova -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1463911 Title: IPV6 fragmentation and mtu issue Status in neutron: Confirmed Status in linux package in Ubuntu: Confirmed Status in linux source package in Trusty: Fix Released Status in linux source package in Vivid: Fix Released Bug description: Fragmented IPv6 packets are REJECTED by ip6tables on compute nodes. The traffic is goign through an intra-VM network and the packet loss is hurting the system. There is a patch for this issue: http://patchwork.ozlabs.org/patch/434957/ I would like to know is there any bug report or official release date for this issue ? This is pretty critical for my deployment. Thanks in advance, BR, Gyula To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1463911/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1461734] Re: duplicate detach volume in nova
So I think cinder has fixed this now they have improved state handling. It moves into the detaching state, which causes the duplicates to fail. We need to double check this again on master, so marking as invalid for now, while we wait for a valid repo steps and logs capturing the error that occurs. It's possible this is backend specific (cinder and nova) so please give more details on the repo environment. ** Changed in: nova Status: Confirmed => Invalid ** Changed in: nova Status: Invalid => Incomplete -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1461734 Title: duplicate detach volume in nova Status in OpenStack Compute (nova): Incomplete Bug description: right now, there are risk that nova will process duplicate detach request. To recur this problem. You can 1) create a server 2) attach a volume 3) detach multi-times you can use the bash downside: for i in {1..20} do nova volume-detach server-id volume-id & done probably you will see the volume is in detaching for ever. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1461734/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1459542] Re: Move check whether instance is booted from volume from libvirt/driver.py to API
So this behaviour is correct. The volume size is inferred from the flavor, unless the user chooses otherwise. It's really not related. The quota and usage in Nova is only tracking local disk, cinder tracks all volumes. Given all that, marking this as invalid for now. ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1459542 Title: Move check whether instance is booted from volume from libvirt/driver.py to API Status in OpenStack Compute (nova): Invalid Bug description: In the current "resize instance" implementation, if a instance is booted from volume, it is allowed to be resize down. But there are two problems: 1. The current implementation allows resize down with a flavor with smaller 'root_gb' than the volume the instance booted from. This will cause differences between the actual 'root_gb' size and the one recorded in the data base. We shouldn't allow resize down with flavor has less 'root_gb' than the volume. 2. Check if the instance is booted from volume in driver.py is a bad implementation, we should perform it in API layer. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1459542/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1456474] Re: Nova-scheduler & Nova-cert stopped after server restart
So please report this issue with the package author. This is not an upstream bug, we have no packing upstream. ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1456474 Title: Nova-scheduler & Nova-cert stopped after server restart Status in OpenStack Compute (nova): Invalid Bug description: Hi All, 1. OpenStack Juno 2. Both nova-scheduler & Nova-cert services are in stop/waiting state after the server restart. root@osjuno:~# service nova-cert status nova-cert stop/waiting root@osjuno:~# service nova-scheduler status nova-scheduler stop/waiting root@osjuno:~# nova service-list +--+--+--+-+---++-+ | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +--+--+--+-+---++-+ | nova-cert| icehouse | internal | enabled | down | 2015-05-19T06:45:41.00 | - | | nova-consoleauth | icehouse | internal | enabled | up| 2015-05-19T06:47:39.00 | - | | nova-scheduler | icehouse | internal | enabled | down | 2015-05-19T06:45:49.00 | - | | nova-conductor | icehouse | internal | enabled | up| 2015-05-19T06:47:33.00 | - | | nova-compute | icehouse | nova | enabled | up| 2015-05-19T06:47:40.00 | - | | nova-network | icehouse | internal | enabled | up| 2015-05-19T06:47:42.00 | - | +--+--+--+-+---++-+ 3. This is the issue we are currently facing in Juno version. In order to resolve this issue, we are starting both the services 'nova-cert & nova-scheduler' in boot process by adding:- service nova-cert start service nova-scheduler restart in '/etc/rc.local' file. 4. Is it the correct procedure to do that & why both these services are not starting after reboot. or Is it a new bug in Juno version under nova? Thanks & Regards, Naga Phani. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1456474/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1452032] Re: Device descriptor not removed with different iqn and multipath enabled.
So to fix this on stable releases, the bug should be targeted for that stable release.this is still invalid for master. ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1452032 Title: Device descriptor not removed with different iqn and multipath enabled. Status in OpenStack Compute (nova): Invalid Status in Ubuntu: New Bug description: [Environment] OpenStack Kilo Trusty 14.04.4 [Description] if the attached multipath devices doesn't have same iqn like regular lvm+iscsi backend, in_use will be false. In that case,_disconnect_volume_multipath_iscsi() returns without calling _remove_multipath_device_descriptor(). [Reproduction] 1) Enable cinder LVM ISCSI on /etc/cinder/cinder.conf volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver 2) Enable iscsi_use_multipath on /etc/nova/nova.conf on your compute nodes: iscsi_use_multipath = True 3) Create 3 cinder volumes $ cinder create 1 $ cinder create 1 $ cinder create 1 $ cinder list ubuntu@niedbalski2-bastion:~/specs/1374999$ cinder list +--+--+--+--+-+--+--+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--+--+--+--+-+--+--+ | 10844be6-8f86-414f-a10e-e1a31e2ba6e7 | in-use | None | 1 | None| false | b0a14447-5740-408a-b96f-a1e904b229e5 | | 1648d24c-0d65-4377-9fa5-6d3aeb8b1291 | in-use | None | 1 | None| false | b0a14447-5740-408a-b96f-a1e904b229e5 | | 53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f | in-use | None | 1 | None| false | b0a14447-5740-408a-b96f-a1e904b229e5 | +--+--+--+--+-+--+--+ 4) Attach them to nova $ nova volume-attach instance_id 10844be6-8f86-414f-a10e-e1a31e2ba6e7 $ nova volume-attach instance_id 1648d24c-0d65-4377-9fa5-6d3aeb8b1291 $ nova volume-attach instance_id 53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f 5) Check on the nova-compute unit for the current multipath/session status tcp: [1] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-10844be6-8f86-414f-a10e-e1a31e2ba6e7 tcp: [2] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-1648d24c-0d65-4377-9fa5-6d3aeb8b1291 tcp: [3] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f Multipath: root@juju-1374999-machine-10:/home/ubuntu# multipath -ll 330030001 dm-2 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 10:0:0:1 sdb 8:16 active ready running 330010001 dm-0 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 8:0:0:1 sdg 8:96 active ready running 330020001 dm-1 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 9:0:0:1 sda 8:0active ready running 6) Detach the current volumes. First. ubuntu@niedbalski2-bastion:~/specs/1374999$ nova volume-detach b0a14447-5740-408a-b96f-a1e904b229e5 10844be6-8f86-414f-a10e- e1a31e2ba6e7 ubuntu@niedbalski2-bastion:~/specs/1374999$ juju ssh 10 "sudo multipath -ll" 330030001 dm-2 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 10:0:0:1 sdb 8:16 active ready running 330010001 dm-0 , size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- #:#:#:# - #:#active faulty running 330020001 dm-1 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 9:0:0:1 sda 8:0active ready running Second raises the faulty state 330030001 dm-2 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 10:0:0:1 sdb 8:16 active ready running 330010001 dm-0 , size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- #:#:#:# - #:#active faulty running 330020001 dm-1 , size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- #:#:#:# - #:#active faulty running Third, raises the faulty state also sudo: unable to resolve host juju-1374999-machine-10 330030001 dm-2 , size=1.
[Yahoo-eng-team] [Bug 1443796] Re: Missing dest type in block_device_mapping_v2
This wishlist bug has been open a year without any activity. I'm going to move it to "Opinion / Wishlist", which is an easily-obtainable queue of older requests that have come on. This bug can be reopened (set back to "New") if someone decides to work on this. ** Changed in: nova Status: Confirmed => Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1443796 Title: Missing dest type in block_device_mapping_v2 Status in OpenStack Compute (nova): Opinion Bug description: I use the command like follows to create vm nova boot --image 66c6aa5a-ce65-40ca-b0fc-1af18b2f744d --flavor m1.tiny --nic net-id=05233983-9bf6-4ab3-bc82-687d6ab9204d --block-device source=blank,device=vdb,bootindex=1,size=1 cirros After the vm created, virsh dumpxml vm i can found only one disk attached to the vm In my view, we need to check the params named dest in if it exsit in block_device_mappint_v2 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1443796/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1439861] Re: encrypted iSCSI volume attach fails to attach when multipath-tools installed
*** This bug is a duplicate of bug 1439869 *** https://bugs.launchpad.net/bugs/1439869 ** This bug has been marked a duplicate of bug 1439869 encrypted iSCSI volume attach fails when iscsi_use_multipath is enabled -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1439861 Title: encrypted iSCSI volume attach fails to attach when multipath-tools installed Status in OpenStack Compute (nova): Confirmed Bug description: An error was occurring in a devstack setup with nova version: commit ab25f5f34b6ee37e495aa338aeb90b914f622b9d Merge "instance termination with update_dns_entries set fails" A volume-type encrypted with CryptsetupEncryptor was being used. A volume was created using this volume-type and an attempt to attach it to an instance was made. This error also occurred when using the LuksEncryptor for the volume-type. The following error occurred in n-cpu during attachment: Stack Trace: 2015-04-02 13:39:54.397 ERROR nova.virt.block_device [req-a8220e7d-8d1e-459d-be1f-4ddd65b7ff66 admin admin] [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] Driver failed to attach volume 81c5f69a-9b01-4f c0-a105-be9d3c966aaf at /dev/vdb 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] Traceback (most recent call last): 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/opt/stack/nova/nova/virt/block_device.py", line 251, in attach 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] device_type=self['device_type'], encryption=encryption) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1064, in attach_volume 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] self._disconnect_volume(connection_info, disk_dev) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 85, in __exit__ 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] six.reraise(self.type_, self.value, self.tb) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 1051, in attach_volume 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] encryptor.attach_volume(context, **encryption) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/opt/stack/nova/nova/volume/encryptors/cryptsetup.py", line 93, in attach_volume 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] self._open_volume(passphrase, **kwargs) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/opt/stack/nova/nova/volume/encryptors/cryptsetup.py", line 78, in _open_volume 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] check_exit_code=True, run_as_root=True) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/opt/stack/nova/nova/utils.py", line 206, in execute 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] return processutils.execute(*cmd, **kwargs) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/processutils.py", line 233, in execute 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] cmd=sanitized_cmd) 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] ProcessExecutionError: Unexpected error while running command. 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf cryptsetup create --key-file=- ip-10.50.3.20:3260- iscsi-iqn.2003-10.com.lefthandnetworks:vsa-12-721:853:volume-81c5f69a-9b01-4fc0-a105-be9d3c966aaf-lun-0 /dev/sdb 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] Exit code: 5 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a1ce-45eb-a5ff-bcb96ec0d8e5] Stdout: u'' 2015-04-02 13:39:54.397 TRACE nova.virt.block_device [instance: 41d0c192-a
[Yahoo-eng-team] [Bug 1427522] Re: affinity server group is limited on one host
This wishlist bug has been open a year without any activity. I'm going to move it to "Opinion / Wishlist", which is an easily-obtainable queue of older requests that have come on. This bug can be reopened (set back to "New") if someone decides to work on this. ** Changed in: nova Status: Confirmed => Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1427522 Title: affinity server group is limited on one host Status in OpenStack Compute (nova): Opinion Bug description: In OpenStack Configuration Reference section scheduling, it states following: Are in a set of group hosts (if requested) (ServerGroupAffinityFilter). It looks like affinity server group can be placed on multiple hosts, but after some trials and investigation, the affinity server group is limited on one host. When the current host doesn't have enough resource to host the subsequent VMs, the nova scheduler returns "No Valid Host Was Found". I don't exactly know whether one affinity server group plans to support multiple hosts. If yes the current implementation needs to update. Otherwise, above document needs to reword. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1427522/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1571839] Re: NoSuchOptError: no such option in group neutron: auth_plugin
*** This bug is a duplicate of bug 1574988 *** https://bugs.launchpad.net/bugs/1574988 This but has already been fixed here: https://github.com/openstack/nova/commit/2647f91ae97844a73176fc1c8663d9b186bdec1a ** This bug has been marked a duplicate of bug 1574988 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1571839 Title: NoSuchOptError: no such option in group neutron: auth_plugin Status in OpenStack Compute (nova): In Progress Bug description: After this change[1], auth_type configuration value is used instead of auth_plugin, resulting in a deprecated value in the code[2] [1] https://github.com/openstack/keystoneauth/commit/a56ed4218aef5a2e528aa682cea967e767dca923 [2] https://github.com/openstack/nova/blob/2bda625935f04f03622ac24eb5ad67e81bc748a1/nova/network/neutronv2/api.py#L107 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1571839/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1569555] Re: Request is wrong for compute v2.1 os-flavor-access list
We should say Fix Released for this. ** Changed in: nova Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1569555 Title: Request is wrong for compute v2.1 os-flavor-access list Status in OpenStack Compute (nova): Fix Released Bug description: The docs say that it takes a tenant_id parameter for listing flavor- access: http://developer.openstack.org/api-ref- compute-v2.1.html#listFlavorAccess But the code doesn't take a tenant_id, only a flavor_id: https://github.com/openstack/nova/blob/2002120c459561d995eac4273befb42b3809d5bb/nova/api/openstack/compute/flavor_access.py#L51 The response in the docs is correct, only the request is wrong. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1569555/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1412285] Re: InstanceInfoCacheNotFound exception
Looks like this got fixed, marking it as invalid. ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1412285 Title: InstanceInfoCacheNotFound exception Status in OpenStack Compute (nova): Invalid Bug description: Operating steps: 1. nova boot a instance named 'A' on host named 'hostxx'. 2. use 'systemctl stop openstack-nova-compute' command to stop nova-compute service on hostxx. 3. use 'nova delete' command delete instance A, and the result is successful. 4. Then use 'systemctl start openstack-nova-compute' command to start nova-compute service on hostxx. 5. wait 30 minutes, the periodic task named '_cleanup_running_deleted_instances' will execute to delete the instance A on hostxx and the exception InstanceInfoCacheNotFound occurs. (You can modify the value of 'running_deleted_instance_poll_interval' in file /etc/nova/nova.conf to reduce the waiting time) the nova-compute.log on hostxx: root@lxlcompute1 ~]# tail -100 /var/log/nova/nova-compute.log 2015-03-17 15:55:36.764 17764 WARNING nova.compute.manager [-] Found 0 in the database and 1 on the hypervisor. 2015-03-17 15:56:00.963 17764 INFO nova.compute.manager [-] [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] Destroying instance with name label 'instance-0056' which is marked as DELETED but still present on host. 2015-03-17 15:56:00.964 17764 AUDIT nova.compute.manager [req-2d912481-6e91-4713-a62a-694232c4e58c None None] [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] Terminating instance 2015-03-17 15:56:01.263 17764 INFO nova.compute.manager [-] [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] VM Stopped (Lifecycle Event) 2015-03-17 15:56:01.272 17764 INFO nova.virt.libvirt.driver [-] [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] Instance destroyed successfully. 2015-03-17 15:56:01.854 17764 ERROR nova.network.api [-] [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] Failed storing info cache 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] Traceback (most recent call last): 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] File "/usr/lib/python2.7/site-packages/nova/network/api.py", line 81, in update_instance_cache_with_nw_info 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] ic.save(update_cells=update_cells) 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] File "/usr/lib/python2.7/site-packages/nova/objects/base.py", line 142, in wrapper 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] ctxt, self, fn.__name__, args, kwargs) 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] File "/usr/lib/python2.7/site-packages/nova/conductor/rpcapi.py", line 430, in object_action 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] objmethod=objmethod, args=args, kwargs=kwargs) 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] File "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/client.py", line 150, in call 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] wait_for_reply=True, timeout=timeout) 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] File "/usr/lib/python2.7/site-packages/oslo/messaging/transport.py", line 90, in _send 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] timeout=timeout) 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 412, in send 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] return self._send(target, ctxt, message, wait_for_reply, timeout) 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] File "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py", line 405, in _send 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] raise result 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-4e0c-84a6-3e4739bb6158] InstanceInfoCacheNotFound_Remote: Info cache for instance b4217413-f4ef-4e0c-84a6-3e4739bb6158 could not be found. 2015-03-17 15:56:01.854 17764 TRACE nova.network.api [instance: b4217413-f4ef-
[Yahoo-eng-team] [Bug 1561357] Re: VM deployed with availability-zone (force_hosts) cannot be live migrated to an untargeted host
** Also affects: nova/mitaka Importance: Undecided Status: New ** Changed in: nova/mitaka Milestone: None => mitaka-rc3 ** Changed in: nova/mitaka Status: New => Fix Committed ** Changed in: nova/mitaka Importance: Undecided => Critical ** Changed in: nova/mitaka Assignee: (unassigned) => Sylvain Bauza (sylvain-bauza) ** Changed in: nova/mitaka Importance: Critical => High -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1561357 Title: VM deployed with availability-zone (force_hosts) cannot be live migrated to an untargeted host Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) mitaka series: Fix Committed Bug description: Steps: 1) Deploy a VM to a specific host using availability zones (i.e., do a targeted deploy). 2) Attempt to live migrate the VM from (1) letting the scheduler decide what host to live migrate to (i.e., do an untargeted live migration). Outcome: The live migration will always fail. Version: mitaka This is happening because of the following recent change: https://github.com/openstack/nova/commit/111a852e79f0d9e54228d8e2724dc4183f737397. The recent change pulls the request spec from the originak deploy from the DB and uses it for the live migration. Since the initial deploy of the VM was targeted, the request spec object saved in the DB has the "force_hosts" field set to a specific host. Part of the live migrate flow will set the "ignore_hosts" field of said request spec object to said specific host since it doesn't make sense to live migrate to the source host. This results in unsolvable constraints for the scheduler. nova/compute/api.py::live_migrate(): ... try: request_spec = objects.RequestSpec.get_by_instance_uuid( <--- this fetches the request spec from the DB, which will have force_hosts set context, instance.uuid) ... self.compute_task_api.live_migrate_instance(context, instance, host_name, block_migration=block_migration, disk_over_commit=disk_over_commit, request_spec=request_spec) After a lot of API plumbing, the flow ends up in nova/conductor/tasks/live_migrate.py::_find_destination(): ... attempted_hosts = [self.source] ... host = None while host is None: ... request_spec.ignore_hosts = attempted_hosts <-- we're setting the source host to "ignore_hosts" field try: host = self.scheduler_client.select_destinations(self.context, request_spec)[0]['host'] < we're passing an unsolvable request_spec to the scheduler now, which will never find a valid host to migrate to Example on a multi-node (2) devstack environment: stack@controller:~/devstack$ nova boot tdp-server --image 13a9f724 -36ef-46ae-896d-f4f003ac1a10 --flavor m1.tiny --availability-zone nova:host613 stack@controller:~/devstack$ nova list --fields name,status,OS-EXT-SRV-ATTR:host +--+++---+ | ID | Name | Status | OS-EXT-SRV-ATTR: Host | +--+++---+ | a9fe19e4-5528-40f2-af08-031eaf4c33a6 | tdp-server | ACTIVE | host613 | +--+++---+ mysql> select spec from request_specs where instance_uuid="a9fe19e4-5528-40f2-af08-031eaf4c33a6"; { ... "nova_object.name":"RequestSpec", "nova_object.data":{ "instance_uuid":"a9fe19e4-5528-40f2-af08-031eaf4c33a6", ..., "availability_zone":"nova", "force_nodes":null, ..., "force_hosts":[ "host613" ], "ignore_hosts":null, ..., "scheduler_hints":{} }, ... } stack@controller:~/devstack$ nova live-migration tdp-server ERROR (BadRequest): No valid host was found. There are not enough hosts available. (HTTP 400) (Request-ID: req-78725630-e87b-426c-a4f6-dc31f9c08223) /opt/stack/logs/n-sch.log:2016-03-24 02:25:27.515 INFO nova.scheduler.host_manager [req-78725630-e87b-426c-a4f6-dc31f9c08223 admin admin] Host filter ignoring hosts: host613 ... /opt/stack/logs/n-sch.log:2016-03-24 02:25:27.515 INFO nova.scheduler.host_manager [req-78725630-e87b-426c-a4f6-dc31f9c08223 admin admin] No hosts matched due to not matching 'force_hosts' value of 'host613' This is breaking previous behavior - the force_hosts field was not "sticky" in that it did not prevent the scheduler from moving the VM to another host after initial deploy. It previously only forced t
[Yahoo-eng-team] [Bug 1561022] Re: Server group policies are not honored during live migration
Seems like a nasty regression, adding to mitaka rc2 ** Also affects: nova/mitaka Importance: Undecided Status: New ** Changed in: nova/mitaka Milestone: None => mitaka-rc2 ** Tags added: mitaka-rc-potential -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1561022 Title: Server group policies are not honored during live migration Status in OpenStack Compute (nova): Confirmed Status in OpenStack Compute (nova) mitaka series: New Bug description: Commit https://github.com/openstack/nova/commit/111a852e79f0d9e54228d8e2724dc4183f737397 introduced regression that causes affinity/anti-affinity policies to be omitted while live migrating an instance. This is because we don't pass instance_group here: https://github.com/openstack/nova/blob/111a852e79f0d9e54228d8e2724dc4183f737397/nova/conductor/tasks/live_migrate.py#L183 However, filters are expecting this information: https://github.com/openstack/nova/blob/111a852e79f0d9e54228d8e2724dc4183f737397/nova/scheduler/filters/affinity_filter.py#L86 Basically we should pass instance group so that filters can read this information later. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1561022/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1558343] Re: configdrive is lost after resize.(libvirt driver)
** Also affects: nova/mitaka Importance: Undecided Status: New ** Changed in: nova/mitaka Status: New => Confirmed ** Changed in: nova/mitaka Importance: Undecided => High ** Changed in: nova/mitaka Milestone: None => mitaka-rc2 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1558343 Title: configdrive is lost after resize.(libvirt driver) Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) kilo series: Confirmed Status in OpenStack Compute (nova) liberty series: Confirmed Status in OpenStack Compute (nova) mitaka series: Confirmed Bug description: Used the trunk code as of 2016/03/16 my environment disabled metadata agent and forced the use of config drive. console log before resize: http://paste.openstack.org/show/490825/ console log after resize: http://paste.openstack.org/show/490824/ qemu 18683 1 4 18:40 ?00:00:32 /usr/bin/qemu-system-x86_64 -name instance-0002 -S -machine pc-i440fx-2.0,accel=tcg,usb=off -m 128 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 018892c7-8144-49c0-93d2-79ee83efd6a9 -smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack Nova,version=13.0.0,serial=16c127e2-6369-4e19-a646-251a416a8dcd,uuid=018892c7-8144-49c0-93d2-79ee83efd6a9,family=Virtual Machine -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-instance-0002/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/opt/stack/data/nova/instances/018892c7-8144-49c0-93d2-79ee83efd6a9/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/opt/stack/da ta/nova/instances/018892c7-8144-49c0-93d2-79ee83efd6a9/disk.config,if=none,id=drive-ide0-1-1,readonly=on,format=raw,cache=none -device ide-cd,bus=ide.1,unit=1,drive=drive-ide0-1-1,id=ide0-1-1 -netdev tap,fd=23,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:34:d6:f3,bus=pci.0,addr=0x3 -chardev file,id=charserial0,path=/opt/stack/data/nova/instances/018892c7-8144-49c0-93d2-79ee83efd6a9/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -vnc 127.0.0.1:1 -k en-us -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on $ blkid /dev/vda1: LABEL="cirros-rootfs" UUID="d42bb4a4-04bb-49b0-8821-5b813116b17b" TYPE="ext3" $ another vm without resize: $ blkid /dev/vda1: LABEL="cirros-rootfs" UUID="d42bb4a4-04bb-49b0-8821-5b813116b17b" TYPE="ext3" /dev/sr0: LABEL="config-2" TYPE="iso9660" $ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1558343/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1557585] Re: Xenapi live-migration does not work at all now
** Also affects: nova/mitaka Importance: Undecided Status: New ** Changed in: nova/mitaka Importance: Undecided => High ** Changed in: nova/mitaka Status: New => Confirmed ** Changed in: nova/mitaka Milestone: None => mitaka-rc2 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1557585 Title: Xenapi live-migration does not work at all now Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) mitaka series: Confirmed Bug description: In case Nova calculated live migration type by itself and it's a block live migration, it will not work if Xen is used because of invalid check in driver: https://github.com/openstack/nova/blob/dae13c5153a3aee25c8ded1cb154cc56a04cd7a2/nova/virt/xenapi/vmops.py#L2391 Basically because here block_migration will be None and real value will be stored in migrate_data.block_migration To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1557585/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1525196] Re: change default policy for password and resetstate
The owner of the instance should not be able to reset state, that is totally as intended. A policy issue will never make an instance go to error, I think you are hitting a different issue here. Please double check the logs, I suspect if you are using KVM, you are missing the qemu agent from your image, or similar. ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1525196 Title: change default policy for password and resetstate Status in OpenStack Compute (nova): Invalid Bug description: all cases from devstack, git head commit 4474dce9e6b847a7691fc3f01d0c594061570d73 I created an instance with admin tenant then use set-password with demo user, it make the instance ERROR this kind of operations should not disabled by default jichen@devstack1:~$ export OS_USERNAME=demo jichen@devstack1:~$ nova set-password t5 New password: Again: ERROR (Conflict): Failed to set admin password on d3485187-779c-441f-8394-4e3d31234a9f because error setting admin password (HTTP 409) (Request-ID: req-96b69164-e353-44f8-82a1-ecd20200173b) jichen@devstack1:~$ nova list +--+--+++-+---+ | ID | Name | Status | Task State | Power State | Networks | +--+--+++-+---+ | d3485187-779c-441f-8394-4e3d31234a9f | t5 | ERROR | - | Running | private=10.0.0.10 | +--+--+++-+---+ Also, after the instance become ERROR state, I can't change it to ACTIVE state even if I am the owner of the instance jichen@devstack1:~$ nova list +--++-++-+--+ | ID | Name | Status | Task State | Power State | Networks | +--++-++-+--+ | 380e55e3-9726-4928-a44c-a206bc656f48 | t2 | ERROR | - | Running | private=10.0.0.8 | | c426c3d0-a981-4839-969a-50d828e05459 | t4 é | SHUTOFF | - | Shutdown| private=10.0.0.2 || +--++-++-+--+ jichen@devstack1:~$ nova reset-state --active t2 Reset state for server t2 failed: Policy doesn't allow os_compute_api:os-admin-actions:reset_state to be performed. (HTTP 403) (Request-ID: req-e7669a3c-3075-4a63-a7a6-f646013a5428) ERROR (CommandError): Unable to reset the state for the specified server(s) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1525196/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1412993] Re: Nova resize for boot-from-volume instance does not resize volume
Marking this as invalid, as I think this is the correct behaviour. There was talk of adding the ability to resize a volume during resize using BDM, but thats a spec. ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1412993 Title: Nova resize for boot-from-volume instance does not resize volume Status in OpenStack Compute (nova): Invalid Bug description: Resizing an instance which booted from a volume to a new flavor with a bigger disk will not cause the volume to resize accordingly. This can cause confusion among the users, which will expect to have instances with bigger storage. Scenario: 1. Have a glance image. 2. Create a bootable volume from glance image. 3. Create instance using volume and flavor having 10GB disk. 4. Perform nova resize on instance to a new flavor having 20GB disk. 5. After resize, see that the instance still has 10GB storage. Cinder volume still has the same size. This issue has been discussed on #openstack-nova and it was agreed upon to fail the resize operation, if the given instance is booted from volume and the given new flavor has a different disk size. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1412993/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1259292] Re: Some tests use assertEqual(observed, expected) , the argument order is wrong
** Changed in: nova Status: In Progress => Won't Fix ** Changed in: nova Assignee: Rajesh Tailor (rajesh-tailor) => (unassigned) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1259292 Title: Some tests use assertEqual(observed, expected) , the argument order is wrong Status in Barbican: In Progress Status in Ceilometer: Invalid Status in Cinder: Fix Released Status in congress: In Progress Status in Designate: In Progress Status in Glance: Fix Committed Status in heat: Fix Released Status in OpenStack Dashboard (Horizon): In Progress Status in Keystone: Fix Committed Status in Manila: In Progress Status in Mistral: In Progress Status in murano: Fix Committed Status in OpenStack Compute (nova): Won't Fix Status in python-ceilometerclient: Invalid Status in python-cinderclient: Fix Released Status in python-designateclient: In Progress Status in python-mistralclient: In Progress Status in Python client library for Zaqar: In Progress Status in Sahara: Fix Released Status in zaqar: In Progress Bug description: The test cases will produce a confusing error message if the tests ever fail, so this is worth fixing. To manage notifications about this bug go to: https://bugs.launchpad.net/barbican/+bug/1259292/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1498075] [NEW] Filter leading/trailing spaces for name field in v2.1 compat mode
Public bug reported: This has spun out of: https://bugs.launchpad.net/nova/+bug/1491511 v2_legacy allows trailing whitespace, so v2.0 compat needs to also accept those request. To make it simpler, best to strip all the trailing whitespace in v2.0. ** Affects: nova Importance: High Assignee: Alex Xu (xuhj) Status: In Progress ** Tags: liberty-rc-potential ** Changed in: nova Importance: Undecided => High ** Tags added: liberty-rc-potential -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1498075 Title: Filter leading/trailing spaces for name field in v2.1 compat mode Status in OpenStack Compute (nova): In Progress Bug description: This has spun out of: https://bugs.launchpad.net/nova/+bug/1491511 v2_legacy allows trailing whitespace, so v2.0 compat needs to also accept those request. To make it simpler, best to strip all the trailing whitespace in v2.0. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1498075/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1427098] Re: Container servers are wrongly scheduled to non-docker hypervisor
docker is not in the nova tree any more, so this is no longer in scope for nova. ** Changed in: nova Status: New => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1427098 Title: Container servers are wrongly scheduled to non-docker hypervisor Status in OpenStack Compute (nova): Won't Fix Bug description: In an environment with more than one hypervisor type including QEMU and docker, when creating a container server, the request can be wrongly scheduled to a QEMU hypervisor. I test it in Juno and there is the same bug in Kilo. For example, there is two compute node, one hypervisor is docker (node A-172), anther one hypervisor is QEMU(node B-168). I can create a docker server which uses docker image "tutum/wordpress" in a QEMU node(node B-168). In this case it ignores the parameter “image" whether is match the hypervisor type, finally this server boot failed with "no bootable device". I'm not sure is there any other parameter for QEMU, docker and Xen hypervisor should check. My step is following: [root@ ~(keystone_admin)]# glance image-show tutum/wordpress +--+--+ | Property | Value| +--+--+ | checksum | bab44a59a74878dd953c4ae5242f7c7c | | container_format | docker | | created_at | 2015-02-02T07:00:47 | | deleted | False| | disk_format | raw | | id | b8e12702-3fd1-4847-b018-ac8ba6edead7 | | is_public| True | | min_disk | 0| | min_ram | 0| | name | tutum/wordpress | | owner| 09291698b9ff44728493252e67fc6ee5 | | protected| False| | size | 517639680| | status | active | | updated_at | 2015-02-02T07:02:15 | +--+--+ [root@ ~(keystone_admin)]# nova boot --flavor 2 --image tutum/wordpress --key-name key1 --nic net- id=2510a249-1665-4184-afc8-62a2eccf6c3b --availability-zone xxx:B-168 test-image-168 2015-03-02 01:55:12.239 2857 WARNING nova.virt.disk.vfs.guestfs [-] Failed to close augeas aug_close: do_aug_close: y ou must call 'aug-init' first to initialize Augeas 2015-03-02 01:55:12.274 2857 DEBUG nova.virt.disk.api [-] Unable to mount image /var/lib/nova/instances/e799cc70-2e9f -44da-9ebd-0f55ddc7cd13/disk with error Error mounting /var/lib/nova/instances/e799cc70-2e9f-44da-9ebd-0f55ddc7cd13/d isk with libguestfs (mount_options: /dev/sda on / (options: ''): mount: /dev/sda is write-protected, mounting read-on ly mount: unknown filesystem type '(null)'). Cannot resize. is_image_partitionless /usr/lib/python2.7/site-packages/nova /virt/disk/api.py:218 2015-03-02 01:55:12.276 2857 DEBUG nova.virt.libvirt.driver [-] [instance: e799cc70-2e9f-44da-9ebd-0f55ddc7cd13] Star t _get_guest_xml network_info=[VIF({'profile': {}, 'ovs_interfaceid': u'563e9d58-9d41-4700-b4f2-a56a6ecfcefe', 'netwo rk': Network({'bridge': 'br-int', 'subnets': [Subnet({'ips': [FixedIP({'meta': {}, 'version': 4, 'type': 'fixed', 'fl oating_ips': [], 'address': u'10.0.0.45'})], 'version': 4, 'meta': {'dhcp_server': u'10.0.0.3'}, 'dns': [], 'routes': [], 'cidr': u'10.0.0.0/24', 'gateway': IP({'meta': {}, 'version': 4, 'type': 'gateway', 'address': u'10.0.0.1'})})], 'meta': {'injected': False, 'tenant_id': u'3cf2410b5f554653a93796982657984b'}, 'id': u'2510a249-1665-4184-afc8-62a2e ccf6c3b', 'label': u'private'}), 'devname': u'tap563e9d58-9d', 'vnic_type': u'normal', 'qbh_params': None, 'meta': {} , 'details': {u'port_filter': True, u'ovs_hybrid_plug': True}, 'address': u'fa:16:3e:d9:b3:1f', 'active': False, 'typ e': u'ovs', 'id': u'563e9d58-9d41-4700-b4f2-a56a6ecfcefe', 'qbg_params': None})] disk_info={'disk_bus': 'virtio', 'cd rom_bus': 'ide', 'mapping': {'disk': {'bus': 'virtio', 'boot_index': '1', 'type': 'disk', 'dev': u'vda'}, 'root': {'b us': 'virtio', 'boot_index': '1', 'type': 'disk', 'dev': u'vda'}}} image_meta={u'status': u'active', u'deleted': Fals e, u'container_format': u'docker', u'min_ram': 0, u'updated_at': u'2015-02-02T07:02:15.00', u'min_disk': 0, u'own er': u'09291698b9ff44728493252e67fc6ee5', u'is_public': True, u'deleted_at': None, u'properties': {}, u'size': 517639 680, u'name': u'tutum/wordpress', u'checksum': u'bab44a59a74878dd953c4ae5242f7c7c', u'created_at': u'201
[Yahoo-eng-team] [Bug 1296414] Re: quotas not updated when periodic tasks or startup finish deletes
Apparently this is a two part fix, the second part is here: https://review.openstack.org/#/c/170118 ** Changed in: nova Status: Fix Released => In Progress ** Changed in: nova Milestone: liberty-1 => None -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1296414 Title: quotas not updated when periodic tasks or startup finish deletes Status in OpenStack Compute (nova): In Progress Status in OpenStack Compute (nova) kilo series: Fix Released Bug description: There are a couple of cases in the compute manager where we don't pass reservations to _delete_instance(). For example, one of them is cleaning up when we see a delete that is stuck in DELETING. The only place we ever update quotas as part of delete should be when the instance DB record is removed. If something is stuck in DELETING, it means that the quota was not updated. We should make sure we're always updating the quota when the instance DB record is removed. Soft delete kinda throws a wrench in this, though, because I think you want soft deleted instances to not count against quotas -- yet their DB records will still exist. In this case, it seems we may have a race condition in _delete_instance() -> _complete_deletion() where if the instance somehow was SOFT_DELETED, quotas would have updated twice (once in soft_delete and once in _complete_deletion). To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1296414/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1297375] Re: All nova apis relying on Instance.save(expected_*_state) for safety contain a race condition
This was reverted, but now has a patch open for review again ** Changed in: nova Milestone: liberty-2 => None ** Changed in: nova Status: Fix Released => In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1297375 Title: All nova apis relying on Instance.save(expected_*_state) for safety contain a race condition Status in OpenStack Compute (nova): In Progress Bug description: Take, for example, resize_instance(). In manager.py, we assert that the instance is in RESIZE_PREP state with: instance.save(expected_task_state=task_states.RESIZE_PREP) This should mean that the first resize will succeed, and any subsequent will fail. However, the underlying db implementation does not lock the instance during the update, and therefore doesn't guarantee this. Specifically, _instance_update() in db/sqlalchemy/apy.py starts a session, and reads task_state from the instance. However, it does not use a 'select ... for update', meaning the row is not locked. 2 concurrent calls to this method can both read the same state, then race to the update. The last writer will win. Without 'select ... for update', the db transaction is only ensuring that all writes are atomic, not reads with dependent writes. SQLAlchemy seems to support select ... for update, as do MySQL and PostgreSQL, although MySQL will fall back to whole table locks for non-InnoDB tables, which would likely be a significant performance hit. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1297375/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1452032] Re: Device descriptor not removed with different iqn and multipath enabled.
Looks like this is invalid now os-brick has merged. ** Tags added: volumes ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1452032 Title: Device descriptor not removed with different iqn and multipath enabled. Status in OpenStack Compute (nova): Invalid Status in Ubuntu: New Bug description: [Environment] OpenStack Kilo Trusty 14.04.4 [Description] if the attached multipath devices doesn't have same iqn like regular lvm+iscsi backend, in_use will be false. In that case,_disconnect_volume_multipath_iscsi() returns without calling _remove_multipath_device_descriptor(). [Reproduction] 1) Enable cinder LVM ISCSI on /etc/cinder/cinder.conf volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver 2) Enable iscsi_use_multipath on /etc/nova/nova.conf on your compute nodes: iscsi_use_multipath = True 3) Create 3 cinder volumes $ cinder create 1 $ cinder create 1 $ cinder create 1 $ cinder list ubuntu@niedbalski2-bastion:~/specs/1374999$ cinder list +--+--+--+--+-+--+--+ | ID | Status | Display Name | Size | Volume Type | Bootable | Attached to | +--+--+--+--+-+--+--+ | 10844be6-8f86-414f-a10e-e1a31e2ba6e7 | in-use | None | 1 | None| false | b0a14447-5740-408a-b96f-a1e904b229e5 | | 1648d24c-0d65-4377-9fa5-6d3aeb8b1291 | in-use | None | 1 | None| false | b0a14447-5740-408a-b96f-a1e904b229e5 | | 53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f | in-use | None | 1 | None| false | b0a14447-5740-408a-b96f-a1e904b229e5 | +--+--+--+--+-+--+--+ 4) Attach them to nova $ nova volume-attach instance_id 10844be6-8f86-414f-a10e-e1a31e2ba6e7 $ nova volume-attach instance_id 1648d24c-0d65-4377-9fa5-6d3aeb8b1291 $ nova volume-attach instance_id 53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f 5) Check on the nova-compute unit for the current multipath/session status tcp: [1] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-10844be6-8f86-414f-a10e-e1a31e2ba6e7 tcp: [2] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-1648d24c-0d65-4377-9fa5-6d3aeb8b1291 tcp: [3] 10.5.1.43:3260,1 iqn.2010-10.org.openstack:volume-53d6bb4e-2ca2-45ab-9ed1-887b1df2ff8f Multipath: root@juju-1374999-machine-10:/home/ubuntu# multipath -ll 330030001 dm-2 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 10:0:0:1 sdb 8:16 active ready running 330010001 dm-0 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 8:0:0:1 sdg 8:96 active ready running 330020001 dm-1 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 9:0:0:1 sda 8:0active ready running 6) Detach the current volumes. First. ubuntu@niedbalski2-bastion:~/specs/1374999$ nova volume-detach b0a14447-5740-408a-b96f-a1e904b229e5 10844be6-8f86-414f-a10e- e1a31e2ba6e7 ubuntu@niedbalski2-bastion:~/specs/1374999$ juju ssh 10 "sudo multipath -ll" 330030001 dm-2 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 10:0:0:1 sdb 8:16 active ready running 330010001 dm-0 , size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- #:#:#:# - #:#active faulty running 330020001 dm-1 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 9:0:0:1 sda 8:0active ready running Second raises the faulty state 330030001 dm-2 IET,VIRTUAL-DISK size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=1 status=active `- 10:0:0:1 sdb 8:16 active ready running 330010001 dm-0 , size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- #:#:#:# - #:#active faulty running 330020001 dm-1 , size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=0 status=active `- #:#:#:# - #:#active faulty running Third, raises the faulty state also sudo: unable to resolve host juju-1374999-machine-10 330030001 dm-2 , size=1.0G features='0' hwhandler='0' wp=rw `-+-
[Yahoo-eng-team] [Bug 1447342] Re: libvirtError: XML error: Missing CPU model name lead to compute service fail to start
patch is abandoned, saying this is a valid failure as things are misconfigured. ** Tags added: libvirt ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1447342 Title: libvirtError: XML error: Missing CPU model name lead to compute service fail to start Status in OpenStack Compute (nova): Invalid Bug description: got following error and failed to start a compute service not sure if we should disallow compute service to start if 'libvirtError: XML error: Missing CPU model name' or not 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup result = function(*args, **kwargs) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/openstack/common/service.py", line 497, in run_service 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup service.start() 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/service.py", line 164, in start 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup self.manager.init_host() 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/compute/manager.py", line 1258, in init_host 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup self.driver.init_host(host=self.host) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 529, in init_host 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup self._do_quality_warnings() 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 507, in _do_quality_warnings 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup caps = self._host.get_capabilities() 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/opt/stack/nova/nova/virt/libvirt/host.py", line 753, in get_capabilities 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in doit 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup result = proxy_call(self._autowrap, f, *args, **kwargs) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in proxy_call 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup rv = execute(f, *args, **kwargs) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in execute 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup six.reraise(c, e, tb) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in tworker 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup rv = meth(*args, **kwargs) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 3153, in baselineCPU 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self) 2015-04-20 14:06:57.351 TRACE nova.openstack.common.threadgroup libvirtError: XML error: Missing CPU model name To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1447342/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1380742] Re: Mulitpath scsi devices are not removed if there is an error in multipath command stdout
Apparently the move to os_brick means we don't need this fixing any more. ** Tags added: volumes ** Changed in: nova Status: In Progress => Invalid ** Changed in: nova Importance: Undecided => Medium -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1380742 Title: Mulitpath scsi devices are not removed if there is an error in multipath command stdout Status in Cinder: Fix Released Status in OpenStack Compute (nova): Invalid Bug description: In cinder/brick/initiator/linuxscsi.py remove_multipath_devce() we are skipping over the remove_scsi_device() calls because find_multipath_device() fails to get information about the device. The problem is that there is a device connected and this then leaves it in a bad state afterwards. Inside of find_multipath_device() we do a call to 'mulitpath -l ' and then parse the stdout which we expect to be in this format: 3624a9370590474d16e6708fd00012dc0 dm-0 PURE,FlashArray size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=-1 status=active |- 34:0:0:1 sdc 8:32 active undef running `- 33:0:0:1 sdb 8:16 active undef running But with a slight misconfiguration you can get a string back that looks like: Oct 13 10:24:01 | /lib/udev/scsi_id exitted with 1 3624a9370590474d16e6708fd00012dc0 dm-0 PURE,FlashArray size=1.0G features='0' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=-1 status=active |- 34:0:0:1 sdc 8:32 active undef running `- 33:0:0:1 sdb 8:16 active undef running Which is then unable to be parsed with the code in there currently, and we just bail out of disconnecting 'dm-0'. We probably should support being able to pick out the device from the string regardless of the extra line in stdout. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1380742/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1460176] Re: Reschedules sometimes do not allocate networks
its not really released yet, move back to fix committed. ** Changed in: nova Status: Fix Released => Fix Committed ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Assignee: (unassigned) => Jim Rollenhagen (jim-rollenhagen) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1460176 Title: Reschedules sometimes do not allocate networks Status in OpenStack Compute (Nova): Fix Committed Status in OpenStack Compute (nova) kilo series: Fix Committed Bug description: https://gist.github.com/jimrollenhagen/b6b45aa43878cdc89d89 Fixed by https://review.openstack.org/#/c/177470/ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1460176/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1462305] [NEW] multi-node test causes nova-compute to lockup
Public bug reported: Its not very clear whats going on here, but here is the symptom. One of the nova-compute nodes appears to lock up: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/screen-n-cpu.txt.gz#_2015-05-29_23_27_48_296 It was just completing the termination of an instance: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/screen-n-cpu.txt.gz#_2015-05-29_23_27_48_153 This is also seen in the scheduler reporting the node as down: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/screen-n-sch.txt.gz#_2015-05-29_23_31_02_711 On further inspection it seems like the other nova compute node had just started a migration: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/subnode-2/screen-n-cpu.txt.gz#_2015-05-29_23_27_48_079 We have had issues in the past where olso.locks can lead to deadlocks, its not totally clear if thats happening here. all the periodic tasks happen in the same greenlet, so you can stop them happening if you hold a lock in an RPC call thats being processed, etc. No idea if thats happening here though. ** Affects: nova Importance: Undecided Assignee: Joe Gordon (jogo) Status: Incomplete ** Tags: testing ** Changed in: nova Status: New => Incomplete ** Changed in: nova Assignee: (unassigned) => Joe Gordon (jogo) ** Tags added: testing -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1462305 Title: multi-node test causes nova-compute to lockup Status in OpenStack Compute (Nova): Incomplete Bug description: Its not very clear whats going on here, but here is the symptom. One of the nova-compute nodes appears to lock up: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/screen-n-cpu.txt.gz#_2015-05-29_23_27_48_296 It was just completing the termination of an instance: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/screen-n-cpu.txt.gz#_2015-05-29_23_27_48_153 This is also seen in the scheduler reporting the node as down: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/screen-n-sch.txt.gz#_2015-05-29_23_31_02_711 On further inspection it seems like the other nova compute node had just started a migration: http://logs.openstack.org/67/175067/2/check/check-tempest-dsvm-multinode-full/7a95fb0/logs/subnode-2/screen-n-cpu.txt.gz#_2015-05-29_23_27_48_079 We have had issues in the past where olso.locks can lead to deadlocks, its not totally clear if thats happening here. all the periodic tasks happen in the same greenlet, so you can stop them happening if you hold a lock in an RPC call thats being processed, etc. No idea if thats happening here though. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1462305/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1460673] [NEW] nova-manage flavor convert fails if instance has no flavor in sys_meta
Public bug reported: nova-manage fails if instance has no flavor in sys_meta when trying to move them all to instance_extra. But mostly the instance_type table includes the correct information, so it should be possible to copy it from there. ** Affects: nova Importance: Medium Assignee: Dan Smith (danms) Status: In Progress ** Tags: unified-objects ** Tags added: unified-objects ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Status: New => Triaged ** Description changed: nova-manage fails if instance has no flavor in sys_meta when trying to - move them all to instance_extra + move them all to instance_extra. + + But mostly the instance_type table includes the correct information, so + it should be possible to copy it from there. ** Summary changed: - nova-manage fails if instance has no flavor in sys_meta when trying to move them all to instance_extra + nova-manage flavor convert fails if instance has no flavor in sys_meta -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1460673 Title: nova-manage flavor convert fails if instance has no flavor in sys_meta Status in OpenStack Compute (Nova): In Progress Bug description: nova-manage fails if instance has no flavor in sys_meta when trying to move them all to instance_extra. But mostly the instance_type table includes the correct information, so it should be possible to copy it from there. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1460673/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1459758] [NEW] inline flavor migration fails with pre-kilo instances
Public bug reported: If instance.save() is used to migrate the flavor from sys_meta to instance_extra, this only works in that instance already has an instance_extra row. In the case where its missing, the update call silently fails to make any changes to the database, and you get lots of OrphanedInstanceErrors when listing instances, because the instance no longer has any flavors. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Affects: nova/kilo Importance: Undecided Assignee: Dan Smith (danms) Status: In Progress ** Tags: db kilo-backport-potential unified-objects ** Tags added: db ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Status: New => Triaged ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1459758 Title: inline flavor migration fails with pre-kilo instances Status in OpenStack Compute (Nova): In Progress Status in OpenStack Compute (nova) kilo series: In Progress Bug description: If instance.save() is used to migrate the flavor from sys_meta to instance_extra, this only works in that instance already has an instance_extra row. In the case where its missing, the update call silently fails to make any changes to the database, and you get lots of OrphanedInstanceErrors when listing instances, because the instance no longer has any flavors. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1459758/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1447132] Re: nova-manage db migrate_flavor_data doesn't do instances not in instance_extra
** Also affects: nova/kilo Importance: Undecided Status: New ** Changed in: nova/kilo Milestone: None => kilo-rc3 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1447132 Title: nova-manage db migrate_flavor_data doesn't do instances not in instance_extra Status in OpenStack Compute (Nova): Fix Committed Status in OpenStack Compute (nova) kilo series: New Bug description: nova-manage db migrate_flavor_data selects all of the instances by joining them to the instance_extra table and then checks which ones have flavor information in the metadata table or the extras table. However, if an instance isn't in instance_extra (for example, it hasn't been written to since the creation of the extras table) then it won't be migrated (even if it isn't deleted AND has flavor info in the metadata table). migrate_flavor_data should select all of the instances in the metadata table with flavor information and migrate those. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1447132/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1446638] [NEW] api has issues when Sorting and pagination params used as filters
Public bug reported: While retrieving servers, the sort and pagination query string parameters are treated as search options. These parameters are passed down to the DB layer and eventually filtered out when an AttributeError is caught because they do not exist on the Instance model. This is taken from: https://review.openstack.org/#/c/147298/4 ** Affects: nova Importance: Medium Assignee: Steven Kaufer (kaufer) Status: In Progress ** Tags: api -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1446638 Title: api has issues when Sorting and pagination params used as filters Status in OpenStack Compute (Nova): In Progress Bug description: While retrieving servers, the sort and pagination query string parameters are treated as search options. These parameters are passed down to the DB layer and eventually filtered out when an AttributeError is caught because they do not exist on the Instance model. This is taken from: https://review.openstack.org/#/c/147298/4 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1446638/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1443816] [NEW] cells: config drive doesn't work with cells when injecting an ssh key
Public bug reported: To reproduce the problem, build an instance with a config drive attached, and keypair selected, when the deployment is using cells. This is the change that caused this issue: https://github.com/openstack/nova/commit/80aae8fcf45fdc38fcb6c9fea503cecbe42e42b6#diff-567f52edc17aff6c473d69c341a4cb0cR313 The addition of reading the key from the database doesn't work for cells, where the key is stored in the api cell database. Ideally we might want to: * add keypair_type into the instance object, along side keypair_name, etc * consider sending a message to the parent cell to fetch the keypair I prefer the first idea. ** Affects: nova Importance: High Status: Confirmed ** Tags: cells ** Tags added: cells ** Changed in: nova Status: New => Confirmed ** Changed in: nova Importance: Undecided => High ** Summary changed: - cells: config drive doesn't work with cells + cells: config drive doesn't work with cells when injecting an ssh key ** Description changed: + To reproduce the problem, build an instance with a config drive + attached, and keypair selected, when the deployment is using cells. + + This is the change that caused this issue: + https://github.com/openstack/nova/commit/80aae8fcf45fdc38fcb6c9fea503cecbe42e42b6#diff-567f52edc17aff6c473d69c341a4cb0cR313 + The addition of reading the key from the database doesn't work for cells, where the key is stored in the api cell database. - This is the change that caused this issue: - https://github.com/openstack/nova/commit/80aae8fcf45fdc38fcb6c9fea503cecbe42e42b6#diff-567f52edc17aff6c473d69c341a4cb0cR313 + Ideally we might want to: + * add keypair_type into the instance object, along side keypair_name, etc + * consider sending a message to the parent cell to fetch the keypair + I prefer the first idea. -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1443816 Title: cells: config drive doesn't work with cells when injecting an ssh key Status in OpenStack Compute (Nova): Confirmed Bug description: To reproduce the problem, build an instance with a config drive attached, and keypair selected, when the deployment is using cells. This is the change that caused this issue: https://github.com/openstack/nova/commit/80aae8fcf45fdc38fcb6c9fea503cecbe42e42b6#diff-567f52edc17aff6c473d69c341a4cb0cR313 The addition of reading the key from the database doesn't work for cells, where the key is stored in the api cell database. Ideally we might want to: * add keypair_type into the instance object, along side keypair_name, etc * consider sending a message to the parent cell to fetch the keypair I prefer the first idea. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1443816/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1392773] Re: Live migration of volume backed instances broken after upgrade to Juno
** Changed in: nova Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1392773 Title: Live migration of volume backed instances broken after upgrade to Juno Status in OpenStack Compute (Nova): Fix Released Status in OpenStack Compute (nova) juno series: Fix Committed Bug description: I'm running nova in a virtualenv with a checkout of stable/juno: root@compute1:/opt/openstack/src/nova# git branch stable/icehouse * stable/juno root@compute1:/opt/openstack/src/nova# git rev-list stable/juno | head -n 1 54330ce33ee31bbd84162f0af3a6c74003d57329 Since upgrading from icehouse, our iscsi backed instances are no longer able to live migrate, throwing exceptions like: Traceback (most recent call last): File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply incoming.message)) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch return self._do_dispatch(endpoint, method, ctxt, args) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch result = getattr(endpoint, method)(ctxt, **new_args) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/exception.py", line 88, in wrapped payload) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/exception.py", line 71, in wrapped return f(self, context, *args, **kw) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 326, in decorated_function kwargs['instance'], e, sys.exc_info()) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/openstack/common/excutils.py", line 82, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 314, in decorated_function return function(self, context, *args, **kwargs) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/compute/manager.py", line 4882, in check_can_live_migrate_source dest_check_data) File "/opt/openstack/venv/nova/local/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5040, in check_can_live_migrate_source raise exception.InvalidSharedStorage(reason=reason, path=source) InvalidSharedStorage: compute2 is not on shared storage: Live migration can not be used without shared storage. Looking back through the code, given dest_check_data like this: {u'disk_over_commit': False, u'disk_available_mb': None, u'image_type': u'default', u'filename': u'tmpyrUUg1', u'block_migration': False, 'is_volume_backed': True} In Icehouse the code to validate the request skipped this[0]: elif not shared and (not is_volume_backed or has_local_disks): In Juno, it matches this[1]: if (dest_check_data.get('is_volume_backed') and not bool(jsonutils.loads( self.get_instance_disk_info(instance['name']: In Juno at least, get_instance_disk_info returns something like this: [{u'disk_size': 10737418240, u'type': u'raw', u'virt_disk_size': 10737418240, u'path': u'/dev/disk/by-path/ip-10.0.0.1:3260-iscsi- iqn.2010-10.org.openstack:volume-10f2302c-26b6-44e0-a3ea- 7033d1091470-lun-1', u'backing_file': u'', u'over_committed_disk_size': 0}] I wonder if that was previously an empty return value in Icehouse, I'm unable to test right now, but if it returned the same then I'm not sure how it ever worked before. This is a lab environment, the volume storage is an LVM+ISCSI cinder service. nova.conf and cinder.conf here[2] [0]: https://github.com/openstack/nova/blob/stable/icehouse/nova/virt/libvirt/driver.py#L4299 [1]: https://github.com/openstack/nova/blob/stable/juno/nova/virt/libvirt/driver.py#L5073 [2]: https://gist.github.com/DazWorrall/b1b1e906a6dc2338f6c1 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1392773/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1434033] [NEW] api: need to clean up keypairs after initial work
Public bug reported: Place holder for the clean up after the initial work on: https://blueprints.launchpad.net/nova/+spec/keypair-x509-certificates ** Affects: nova Importance: High Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: api ** Changed in: nova Importance: Undecided => High ** Changed in: nova Milestone: None => kilo-rc1 ** Changed in: nova Assignee: (unassigned) => Claudiu Belu (cbelu) ** Changed in: nova Status: New => In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1434033 Title: api: need to clean up keypairs after initial work Status in OpenStack Compute (Nova): In Progress Bug description: Place holder for the clean up after the initial work on: https://blueprints.launchpad.net/nova/+spec/keypair-x509-certificates To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1434033/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1422342] [NEW] xenapi: soft reboot should attempt hard reboot on failure
Public bug reported: When we issue a soft reboot. If it fails we should then do a hard reboot. This is for consistency with libvirt: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1977 ** Affects: nova Importance: Medium Status: Triaged ** Tags: xenserver ** Changed in: nova Assignee: John Garbutt (johngarbutt) => (unassigned) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1422342 Title: xenapi: soft reboot should attempt hard reboot on failure Status in OpenStack Compute (Nova): Triaged Bug description: When we issue a soft reboot. If it fails we should then do a hard reboot. This is for consistency with libvirt: https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L1977 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1422342/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1398826] [NEW] xenapi: upload failures causing image to go active
Public bug reported: There was an attempt to stop some glance errors when we have upload failures here: https://github.com/openstack/nova/commit/e039b036b5e9dbaff8b37f7ab22c209b71bdc182 However, sending the chunk terminator makes glance thing that the failed upload has completed. We need to make sure when the upload fails, glance puts the image into the failed state, not the active state. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: xenserver ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) ** Changed in: nova Status: New => In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1398826 Title: xenapi: upload failures causing image to go active Status in OpenStack Compute (Nova): In Progress Bug description: There was an attempt to stop some glance errors when we have upload failures here: https://github.com/openstack/nova/commit/e039b036b5e9dbaff8b37f7ab22c209b71bdc182 However, sending the chunk terminator makes glance thing that the failed upload has completed. We need to make sure when the upload fails, glance puts the image into the failed state, not the active state. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1398826/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1376681] Re: cells: CONF.cells.reserve_percent should help reserve whole hosts
Actually this is rubbish, its there for a reason. ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1376681 Title: cells: CONF.cells.reserve_percent should help reserve whole hosts Status in OpenStack Compute (Nova): Invalid Bug description: The current cells state updates only reserve an amount of capacity. Really it should be trying to keep a number of hosts free, at least that was the original intention. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1376681/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1376681] [NEW] cells: CONF.cells.reserve_percent should help reserve whole hosts
Public bug reported: The current cells state updates only reserve an amount of capacity. Really it should be trying to keep a number of hosts free, at least that was the original intention. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: cells ** Changed in: nova Status: New => Triaged ** Changed in: nova Importance: Undecided => Medium -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1376681 Title: cells: CONF.cells.reserve_percent should help reserve whole hosts Status in OpenStack Compute (Nova): In Progress Bug description: The current cells state updates only reserve an amount of capacity. Really it should be trying to keep a number of hosts free, at least that was the original intention. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1376681/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1371072] [NEW] xenapi: should clean up old snapshots before creating a new one
Public bug reported: When nova-compute gets forcably restarted, or fails, we get left over snapshots. We have some clean up code for after nova-compute comes back up, but it would be good to clean up older snapshots, and generally try to minimize the size of the snapshot that goes to glance. ** Affects: nova Importance: Low Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: xenserver ** Changed in: nova Importance: Undecided => Low ** Changed in: nova Status: New => Triaged ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1371072 Title: xenapi: should clean up old snapshots before creating a new one Status in OpenStack Compute (Nova): In Progress Bug description: When nova-compute gets forcably restarted, or fails, we get left over snapshots. We have some clean up code for after nova-compute comes back up, but it would be good to clean up older snapshots, and generally try to minimize the size of the snapshot that goes to glance. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1371072/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1370999] [NEW] xenapi: windows agent unreliable due to reboots
Public bug reported: The windows nova-agent now can trigger a gust reboot during resetnetwork, so the hostname is correctly updated. Also there was always a reboot during the first stages of polling for the agent version that can cause the need to wait for a call to timeout, rather than detecting a reboot. Either way, we need to take more care to detect reboots while talking to the agent. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: xenserver ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1370999 Title: xenapi: windows agent unreliable due to reboots Status in OpenStack Compute (Nova): In Progress Bug description: The windows nova-agent now can trigger a gust reboot during resetnetwork, so the hostname is correctly updated. Also there was always a reboot during the first stages of polling for the agent version that can cause the need to wait for a call to timeout, rather than detecting a reboot. Either way, we need to take more care to detect reboots while talking to the agent. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1370999/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1366758] [NEW] notifications should include progress info and cell name
Public bug reported: The notifications are quite out of sync with some of the instance object changes, in particular these very useful details are not included: * progress * cell_name ** Affects: nova Importance: Wishlist Assignee: John Garbutt (johngarbutt) Status: In Progress ** Changed in: nova Status: New => Confirmed ** Changed in: nova Importance: Undecided => Wishlist ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) ** Changed in: nova Status: Confirmed => In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1366758 Title: notifications should include progress info and cell name Status in OpenStack Compute (Nova): In Progress Bug description: The notifications are quite out of sync with some of the instance object changes, in particular these very useful details are not included: * progress * cell_name To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1366758/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1359835] [NEW] select_destinations should send start/end notifications
Public bug reported: In the filter scheduler, schedule_run_instance sends notifications, but select_destinations does not. This is inconsistent, and we should send start/end notifications from both code paths. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: Triaged ** Tags: scheduler -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1359835 Title: select_destinations should send start/end notifications Status in OpenStack Compute (Nova): Triaged Bug description: In the filter scheduler, schedule_run_instance sends notifications, but select_destinations does not. This is inconsistent, and we should send start/end notifications from both code paths. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1359835/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1359651] [NEW] xenapi: still get MAP_DUPLICATE_KEY in some edge cases
Public bug reported: Older version of XenServer require us to keep the live copy of xenstore updated in sync with the copy of xenstore recorded in the xenapi metadata for that VM. Code inspection has shown that we don't consistently keep those two copies up to date. While its hard to reproduce this errors, (add_ip_address_to_vm seems particuarly likely to hit issues), it seems best to tidy up the xenstore writing code so we consistently add/remove keys from the live copy and the copy in xenapi. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: Triaged ** Tags: xenserver -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1359651 Title: xenapi: still get MAP_DUPLICATE_KEY in some edge cases Status in OpenStack Compute (Nova): Triaged Bug description: Older version of XenServer require us to keep the live copy of xenstore updated in sync with the copy of xenstore recorded in the xenapi metadata for that VM. Code inspection has shown that we don't consistently keep those two copies up to date. While its hard to reproduce this errors, (add_ip_address_to_vm seems particuarly likely to hit issues), it seems best to tidy up the xenstore writing code so we consistently add/remove keys from the live copy and the copy in xenapi. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1359651/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1340596] Re: Tests fail due to novaclient 2.18 update
** Also affects: python-novaclient Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1340596 Title: Tests fail due to novaclient 2.18 update Status in Orchestration API (Heat): Invalid Status in heat havana series: Confirmed Status in heat icehouse series: New Status in OpenStack Dashboard (Horizon): In Progress Status in Python client library for Nova: New Bug description: tests currently fail on stable branches: 2014-07-11 07:14:28.737 | == 2014-07-11 07:14:28.738 | ERROR: test_index (openstack_dashboard.dashboards.admin.aggregates.tests.AggregatesViewTests) 2014-07-11 07:14:28.774 | -- 2014-07-11 07:14:28.775 | Traceback (most recent call last): 2014-07-11 07:14:28.775 | File "/home/jenkins/workspace/gate-horizon-python26/openstack_dashboard/test/helpers.py", line 124, in setUp 2014-07-11 07:14:28.775 | test_utils.load_test_data(self) 2014-07-11 07:14:28.775 | File "/home/jenkins/workspace/gate-horizon-python26/openstack_dashboard/test/test_data/utils.py", line 43, in load_test_data 2014-07-11 07:14:28.775 | data_func(load_onto) 2014-07-11 07:14:28.775 | File "/home/jenkins/workspace/gate-horizon-python26/openstack_dashboard/test/test_data/exceptions.py", line 60, in data 2014-07-11 07:14:28.776 | TEST.exceptions.nova_unauthorized = create_stubbed_exception(nova_unauth) 2014-07-11 07:14:28.776 | File "/home/jenkins/workspace/gate-horizon-python26/openstack_dashboard/test/test_data/exceptions.py", line 44, in create_stubbed_exception 2014-07-11 07:14:28.776 | return cls(status_code, msg) 2014-07-11 07:14:28.776 | File "/home/jenkins/workspace/gate-horizon-python26/openstack_dashboard/test/test_data/exceptions.py", line 31, in fake_init_exception 2014-07-11 07:14:28.776 | self.code = code 2014-07-11 07:14:28.776 | AttributeError: can't set attribute 2014-07-11 07:14:28.777 | To manage notifications about this bug go to: https://bugs.launchpad.net/heat/+bug/1340596/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1331440] [NEW] xenapi: failed snapshots never deleted
Public bug reported: When uploading a snapshot, its possible that nova-compute process is killed. When this happens, currently, that snapshot is never deleted, and the VDI chain can grown a lot. To fix this, we should remove any snapshots from the chain, before taking the next snapshot. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: Triaged ** Tags: xenserver ** Tags added: xenserver ** Changed in: nova Status: New => Triaged ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1331440 Title: xenapi: failed snapshots never deleted Status in OpenStack Compute (Nova): Triaged Bug description: When uploading a snapshot, its possible that nova-compute process is killed. When this happens, currently, that snapshot is never deleted, and the VDI chain can grown a lot. To fix this, we should remove any snapshots from the chain, before taking the next snapshot. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1331440/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1312132] [NEW] live migrate of stopped VM goes to error, not back to task_state=None
Public bug reported: When the InstanceNotRunning exception is raised during the initial live- migrate, it should really just revert to task_state=None in the same way as if there was NoValidHost, or the compute service was down. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: compute ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Status: New => Confirmed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1312132 Title: live migrate of stopped VM goes to error, not back to task_state=None Status in OpenStack Compute (Nova): In Progress Bug description: When the InstanceNotRunning exception is raised during the initial live-migrate, it should really just revert to task_state=None in the same way as if there was NoValidHost, or the compute service was down. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1312132/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1301515] [NEW] reduce logging in the scheduler to improve performance
Public bug reported: The current debug logs in the scheduler are at critical points in the code, and are causing performance issues. After the DB, the scheduler is spending more time doing logging, than anything else. This was discovered using the test_performance_check_select_destination unit test, and modifying it to look at when there are around 200 hosts, which is still quite a modest size. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: scheduler ** Changed in: nova Status: New => Triaged ** Changed in: nova Importance: Undecided => Medium -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1301515 Title: reduce logging in the scheduler to improve performance Status in OpenStack Compute (Nova): In Progress Bug description: The current debug logs in the scheduler are at critical points in the code, and are causing performance issues. After the DB, the scheduler is spending more time doing logging, than anything else. This was discovered using the test_performance_check_select_destination unit test, and modifying it to look at when there are around 200 hosts, which is still quite a modest size. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1301515/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1293720] Re: xenapi: errors when trying to kill tar process
** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1293720 Title: xenapi: errors when trying to kill tar process Status in OpenStack Compute (Nova): Invalid Bug description: For the following bug fix, we introduced a broken fix: 1284596 We see errors like this in the log: ['XENAPI_PLUGIN_FAILURE', 'upload_vhd', 'TypeError', 'sequence item 2: expected string, int found'] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1293720/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1293720] [NEW] xenapi: errors when trying to kill tar process
Public bug reported: For the following bug fix, we introduced a broken fix: 1284596 We see errors like this in the log: ['XENAPI_PLUGIN_FAILURE', 'upload_vhd', 'TypeError', 'sequence item 2: expected string, int found'] ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: Invalid ** Tags: xenserver ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Status: New => In Progress ** Tags added: xenserver -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1293720 Title: xenapi: errors when trying to kill tar process Status in OpenStack Compute (Nova): Invalid Bug description: For the following bug fix, we introduced a broken fix: 1284596 We see errors like this in the log: ['XENAPI_PLUGIN_FAILURE', 'upload_vhd', 'TypeError', 'sequence item 2: expected string, int found'] To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1293720/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1262601] Re: Can't do "resize" on shared storage
Sorry, I don't see this as something we should support. If you have a fix, I am cool with that, but its a very complicated bit of code, I would rather make simpler not more complex, for just an edge case. ** Changed in: nova Status: Triaged => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1262601 Title: Can't do "resize" on shared storage Status in OpenStack Compute (Nova): Won't Fix Bug description: Can't do "resize" to instance (at least to bigger flavor) on shared storage using XenServer as hypervisor. Additionaly, if fails to "resize", VM is crashed too, I don't get a way to restore it in OpenStack. In XenServer VM is running, but by different name: Before - instance-0031 After - instance-0031-orig 2013-12-19 12:41:20.549 7888 ERROR nova.compute.manager [req-618c5498-3a38-4a34-a763-07e2c0c2cdeb 9ffbc08b941540c69a804ecfa4d3dc4e 76619aca43f841e0bed1a3b89ce8b426] [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] Setting instance vm_state to ERROR 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] Traceback (most recent call last): 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4967, in _error_out_instance_on_exception 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] yield 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3021, in resize_instance 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] block_device_info) 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/driver.py", line 284, in migrate_disk_and_power_off 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] dest, instance_type, block_device_info) 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 939, in migrate_disk_and_power_off 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] context, instance, dest, vm_ref, sr_path) 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 882, in _migrate_disk_resizing_up 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] self._migrate_vhd(instance, vdi_uuid, dest, sr_path, seq_num) 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] File "/usr/lib/python2.7/dist-packages/nova/virt/xenapi/vmops.py", line 770, in _migrate_vhd 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] raise exception.MigrationError(reason=msg) 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] MigrationError: Migration error: Failed to transfer vhd to new host 2013-12-19 12:41:20.549 7888 TRACE nova.compute.manager [instance: e6903021-bdee-4f59-ad31-6cc686851eaa] 2013-12-19 12:41:20.956 7888 ERROR nova.openstack.common.rpc.amqp [req-618c5498-3a38-4a34-a763-07e2c0c2cdeb 9ffbc08b941540c69a804ecfa4d3dc4e 76619aca43f841e0bed1a3b89ce8b426] Exception during message handling 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line 461, in _process_data 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp **args) 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py", line 172, in dispatch 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp result = getattr(proxyobj, method)(ctxt, **kwargs) 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 353, in decorated_function 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp return function(self, context, *args, **kwargs) 2013-12-19 12:41:20.956 7888 TRACE nova.openstack.common.rpc.amqp File "/usr/lib/pyth
[Yahoo-eng-team] [Bug 1030108] Re: xenapi: bad handling of "volume in use" errors
This now raises: Reached maximum number of retries trying to unplug VBD OpaqueRef:50be57f5-7c5a-0e72-7659-38b37608ea2a Instance goes into the error state. If the user calls reboot, everything will return to normal. ** Changed in: nova Status: Triaged => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1030108 Title: xenapi: bad handling of "volume in use" errors Status in OpenStack Compute (Nova): Invalid Bug description: We have noticed several circumstances (like for example if a volume is in a raid, or in use) and detach call is issued, the volume stays in the in-use state, and Xen holds on to that volume. When the volume becomes not in use any longer (for example if you remove it from the raid, unmount it), Xen will then still have the detach command queued, and detach the device. This may lead to confusing behavior for the user. If a detach fails because it is in use, it should remove the queued detach command from the instance so that it isn't detached when not in use any more. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1030108/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1024786] Re: xenapi:agent tries to update before networking is setup
*** This bug is a duplicate of bug 1250162 *** https://bugs.launchpad.net/bugs/1250162 Fixed here: https://github.com/openstack/nova/commit/9393ed0a8546ce393e354614e87056c795abd22b ** This bug has been marked a duplicate of bug 1250162 xenapi: Agent update ordering -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1024786 Title: xenapi:agent tries to update before networking is setup Status in OpenStack Compute (Nova): Triaged Bug description: Most of the time, the server might not have network connectivity to download the new agent (if not using DHCP) therefore failing the update. It should attempt to update, should it fail, it should try again after the network is configured on the host. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1024786/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1182934] Re: xenapi: set hostname is failing with latest XS 6.1 PV tools
This was fixed in the nova agent, and is now fixed. Clearly had another bug or blueprint open for the fix. ** Changed in: nova Status: Triaged => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1182934 Title: xenapi: set hostname is failing with latest XS 6.1 PV tools Status in OpenStack Compute (Nova): Invalid Bug description: XenServer 6.1 PV tools no longer set the hostname in the way expected by nova's XenAPI driver. Need to look at using the guest agent, or in the case of cloud-init not setting the hostname flag. Also, other network related xenstore values are not really relevant when the agent is not being used. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1182934/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1188141] Re: Deleting images breaks rescue mode for servers built from said images
There is a blueprint for the fix for this: https://blueprints.launchpad.net/nova/+spec/allow-image-to-be-specified-during-rescue ** Changed in: nova Status: Triaged => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1188141 Title: Deleting images breaks rescue mode for servers built from said images Status in OpenStack Compute (Nova): Invalid Bug description: Rescue image is considered to be the image from which the instance is built from. If that image is deleted, the rescue fails Consider this scenario, The customer has a snapshot of an instance. He build another instance from the snapshot and deletes the snapshot. Now the customer tries to rescue the instance that was built from the snapshot, it fails because the rescue image is not available. Possible solutions - 1. Recursively find the image that's available: The system_metadata of an instance has the "image_base_image_ref" as property set on it. This points to image from which it was built. Image has instance_uuid, if it's a snapshot. We'll have to recursively find the base_install and use it as rescue image if the snapshot is deleted. 2. Store the corresponding original image reference in all images. So that it's easier to find the rescue image. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1188141/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1289361] [NEW] xenapi: unable to create instances with ephemeral disks
Public bug reported: The resize ephemeral disk blueprint has regressed the ability to spawn instances with ephemeral disks. ** Affects: nova Importance: High Assignee: John Garbutt (johngarbutt) Status: Triaged ** Tags: xenserver ** Tags added: xenserver ** Changed in: nova Milestone: None => icehouse-rc1 ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) ** Changed in: nova Importance: Undecided => High ** Changed in: nova Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1289361 Title: xenapi: unable to create instances with ephemeral disks Status in OpenStack Compute (Nova): Triaged Bug description: The resize ephemeral disk blueprint has regressed the ability to spawn instances with ephemeral disks. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1289361/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Blueprint auto-backup-terminate] Autobackup When Deprovisioning
Blueprint changed by John Garbutt: Definition Status: Approved => Obsolete -- Autobackup When Deprovisioning https://blueprints.launchpad.net/nova/+spec/auto-backup-terminate -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1024944] Re: XenServer: If file system is directly is on root device, auto_disk_configure does not work
This seems unlikely at this point, marking as will not fix. ** Changed in: nova Status: Triaged => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1024944 Title: XenServer: If file system is directly is on root device, auto_disk_configure does not work Status in OpenStack Compute (Nova): Won't Fix Bug description: When using XenServer and auto_disk_configure set to true, it only currently works with a disk with a partition table and an OS on the first partition. There are possibilities that it could be directly on the root of the drive (filesystem). To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1024944/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1284596] [NEW] xenapi: must cleanup tar process on glance download errors
Public bug reported: We recently ensured there is a socket timeout on glance download errors. However this now leaves the tar processes behind. We need to kill the tar process when these kinds of errors occur. The code is also inconsistent, we should really do something similar for the upload code path, to ensure both behave in a similar way to any network errors. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: xenserver ** Changed in: nova Status: New => In Progress ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Assignee: (unassigned) => John Garbutt (johngarbutt) ** Summary changed: - xenapi: must cleanup tar process on glance errors + xenapi: must cleanup tar process on glance download errors ** Description changed: We recently ensured there is a socket timeout on glance download errors. However this now leaves the tar processes behind. We need to kill the tar process when these kinds of errors occur. + + The code is also inconsistent, we should really do something similar for + the upload code path, to ensure both behave in a similar way to any + network errors. -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1284596 Title: xenapi: must cleanup tar process on glance download errors Status in OpenStack Compute (Nova): In Progress Bug description: We recently ensured there is a socket timeout on glance download errors. However this now leaves the tar processes behind. We need to kill the tar process when these kinds of errors occur. The code is also inconsistent, we should really do something similar for the upload code path, to ensure both behave in a similar way to any network errors. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1284596/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1261964] Re: should handle extra parameters as invalid via update_host API
We might have to live with this one, because its an incompatible API change. ** Tags added: api ** Changed in: nova Status: New => Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1261964 Title: should handle extra parameters as invalid via update_host API Status in OpenStack Compute (Nova): Opinion Bug description: On update_host API(update() in nova/api/openstack/compute/contrib/hosts.py), there is a different behavior between JSON request and XML request. If JSON request has extra API parameters(not "status", "maintenance_mode"), Nova returns a BadRequest response with message "Invalid update setting: ". But if XML request, Nova does not return a BadRequest response and continues its work. This behavior seems wrong from the viewpoint of API consistency. Note: This is due to HostUpdateDeserializer class, the class' method ignores extra API parameters and deserializes right parameters only. This difference happens in Nova v2 API, and it has been fixed in Nova v3 API. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1261964/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1262914] Re: Unnecessary data copy during cold snapshot
Personally, the VM could be turned on at any time, so this seems like the safest thing to do. I will let the libvirt experts take a look at this. ** Tags added: libvirt ** Changed in: nova Status: New => Opinion ** Changed in: nova Importance: Undecided => Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1262914 Title: Unnecessary data copy during cold snapshot Status in OpenStack Compute (Nova): Opinion Bug description: When creating a cold snapshot, LibvirtDriver.snapshot() creates a local copy of the VM image before uploading from that copy into a new image in Glance. In case of snapshotting a local file backed VM to Swift, that's one copy too many: if the target format matches the source format, the local file can be uploaded directly, halving the time it takes to create a snapshot. In case of snapshotting an RBD backed VM to RBD backed Glance, that's two copies too many: a copy-on-write clone of the VM drive could obviate the need to copy any data at all. I think that instead of passing the target location as a temporary file path under snapshots_directory, LibvirtDriver.snapshot() should pass image metadata to Image.snapshot_extract() and let the image backend figure out and return the target location. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1262914/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1264925] Re: Setting up the configuration rpc_zmq_topic_backlog causing zmq receiver to silently ignore all messages
Sounds like an oslo bug, rather than nova, but needs an experts eye ** Tags added: zmq ** Also affects: oslo Importance: Undecided Status: New ** Changed in: nova Status: New => Incomplete -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1264925 Title: Setting up the configuration rpc_zmq_topic_backlog causing zmq receiver to silently ignore all messages Status in OpenStack Compute (Nova): Incomplete Status in Oslo - a Library of Common OpenStack Code: New Bug description: Setting up the configuration parameter rpc_zmq_topic_backlog causing zmq receiver to silently ignore all messages - I had run strace on nova-rpc-zmq-receiver, from where I see that, the issue was with one configuration option “rpc_zmq_topic_backlog”, due to which the code was silently returning without processing that message – there was no logs or no trace in the zmq receiver log even after enabling debug. What I see is that this option is set in zmq_opts array in impl_zmq.py, but when a message come in, the class ZmqProxy check this config item , and the function __getattr__ in the class ConfigOpts (oslo/config/cfg.py) is raising error saying no such option and then return it without processing that message further. If I just comment out the configuration entry, it just works fine. Please see attachment for Strace output. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1264925/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1257756] Re: multiple services running causes failure
This isn't a nova issue, please raise on the ML or raise a devstack bug. ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1257756 Title: multiple services running causes failure Status in OpenStack Compute (Nova): Invalid Bug description: Scenario : when stack.sh(from devstack) script is run for the first time and if for any reason the user unstack the openstack by using unstack.sh and clean.sh. And then again install it Issue : during 2nd run of stack.sh , few nova services specially n-cpu and n-api fails running. It happens because of unstacking the last stack these services was not killed. During 2nd run, the log file says : multiple services running on the same address (same URI). there would be multiple duplicate services runs which causes current install fails. the above message can be view. on running the services for example. just run nova-api on terminal Traceback : it will show address already in use. ERROR To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1257756/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1274924] Re: Settings the availability zone deletes all attached metadata
I think that is as expected, it updates all the metadata to that supplied, however, its a bit confusing. ** Tags added: api ** Changed in: nova Status: New => Triaged ** Tags added: nova-client ** Tags removed: nova-client ** Tags added: novaclient ** Also affects: python-novaclient Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1274924 Title: Settings the availability zone deletes all attached metadata Status in OpenStack Compute (Nova): Triaged Status in Python client library for Nova: New Bug description: I'm using latest git with devstack on Ubuntu 12.04. When I update the availability zone the attached metadata gets deleted. Steps to reproduce the problem: 1) nova aggregate-create testagg #assuming that this creates a new metadata entry with the Id 26 2) nova aggregate-set-metadata 26 x=y 3) nova aggregate-update 26 testagg zone1 Now the availability zone is set, but the x=y metadata is lost. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1274924/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1276050] Re: when i run nova ./run_test -N ; Errors ocurred by "ImportError: No module named config"
You need to update your virtual environment, it should then install oslo.config. ./run_tests -u Please re-open if this doesn't fix your issue. Its probably better to ask about this in IRC or the ML. ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1276050 Title: when i run nova ./run_test -N ;Errors ocurred by "ImportError: No module named config" Status in OpenStack Compute (Nova): Invalid Bug description: 1.cd nova/ 2.'pip install -r requirements.txt' and 'pip install -r test-requirements.txt' 3.erros appeard .and logs as: Running ` python -m nova.openstack.common.lockutils python setup.py testr --testr-args='--subunit --concurrency 0 '` Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/opt/stack/nova/nova/openstack/common/lockutils.py", line 29, in from oslo.config import cfg ImportError: No module named config Ran 0 tests in 0.001s OK cat: .testrepository/next-stream: No such file or directory To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1276050/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1253186] Re: instance delete requests occasionally lost after nova-api says OK
*** This bug is a duplicate of bug 1251920 *** https://bugs.launchpad.net/bugs/1251920 Already fixed, marking as invalid. ** Changed in: nova Status: New => Invalid ** This bug has been marked a duplicate of bug 1251920 Tempest failures due to failure to return console logs from an instance -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1253186 Title: instance delete requests occasionally lost after nova-api says OK Status in OpenStack Compute (Nova): Invalid Bug description: Doing performance testing on the latest openstack code, instance delete requests that I'm issuing are occasionally "lost". By "lost", I mean that nova-api responds HTTP 200, but nothing seems to happen to the instance, even after several minutes. Hence the request seems to be lost after nova-api casts the RPC to nova-compute. A subsequent delete request for the instance usually works as expected. In a run where I boot 10 instances in parallel then delete each instances as soon as it goes ACTIVE, usually 2 of my delete requests are lost. To get a sense of timing, consider that a run of 5 instances currently takes ~8s on my system; I can't report on a run of 10 instances, because they never successfully finish. I'm using mysql and rabbitmq. I plan on digging into the nova logs to see what happens with these lost requests. I'll always include the req-XXX tag in the HTTP response, then grep for the req-XXX tags of the lost requests. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1253186/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1277339] Re: snapshot-failed-when-dependent-volume-exist
seem like a cinder thing, but throw it back if you were using the nova api. ** Project changed: nova => cinder -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1277339 Title: snapshot-failed-when-dependent-volume-exist Status in Cinder: New Bug description: When we use 'HpSanISCSIDriver ', there is some issues. 1. Create volume 2. Create snapshot from volume 3. Create new volume from snapshot 4. Try to delete snapshot. In this work flow, the status of snapshot should be 'error_deleting'. The error message says like this. \n\n \n\n To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1277339/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1277316] Re: Diconnecting a volume with multipath generates excesive multipath calls
Sounds more like a cinder issue, although the detach probably gets called through nova. ** Project changed: nova => cinder -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1277316 Title: Diconnecting a volume with multipath generates excesive multipath calls Status in Cinder: New Bug description: I have a compute node with 20 volumes attached using iscsi and multipath. Each multipath device has 4 iscsi devices. When I disconnect a volume it generates 779 multipath -ll calls. iscsiadm -m node --rescan iscsiadm -m session --rescan multipath - r multipath -ll /dev/sdch multipath -ll /dev/sdcg multipath -ll /dev/sdcf multipath -ll /dev/sdce multipath -ll /dev/sdcd multipath -ll /dev/sdcc multipath -ll /dev/sdcb multipath -ll /dev/sdca multipath -ll /dev/sdbz multipath -ll /dev/sdby multipath -ll /dev/sdbx multipath -ll /dev/sdbw multipath -ll /dev/sdbv multipath -ll /dev/sdbu multipath -ll /dev/sdbt multipath -ll /dev/sdbs multipath -ll /dev/sdbr multipath -ll /dev/sdbq multipath -ll /dev/sdbp multipath -ll /dev/sdbo multipath -ll /dev/sdbn multipath -ll /dev/sdbm multipath -ll /dev/sdbl multipath -ll /dev/sdbk multipath -ll /dev/sdbj multipath -ll /dev/sdbi multipath -ll /dev/sdbh multipath -ll /dev/sdbg multipath -ll /dev/sdbf multipath -ll /dev/sdbe multipath -ll /dev/sdbd multipath -ll /dev/sdbc multipath -ll /dev/sdbb multipath -ll /dev/sdba .. And so on for 779 times cp /dev/stdin /sys/block/sdcd/device/delete cp /dev/stdin /sys/block/sdcc/device/delete cp /dev/stdin /sys/block/sdcb/device/delete cp /dev/stdin /sys/block/sdca/device/delete multipath - r To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1277316/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1277054] [NEW] Poll rescued instances fails with key error
Public bug reported: After an instance has been in the rescue state for some time, a periodic task triggers to unrescue the instances: _poll_rescued_instances File nova/notifications.py info_from_instance instance_type = flavors.extract_flavor(instance_ref) File "nova/compute/flavors.py" in extract_flavor instance_type[key] = type_fn(sys_meta[type_key]) KeyError: 'instance_type_memory_mb' This then continues to happen on every run of the periodic task, and starts to fill up the DB with instance faults. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: compute ** Changed in: nova Importance: Undecided => Medium ** Changed in: nova Status: New => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1277054 Title: Poll rescued instances fails with key error Status in OpenStack Compute (Nova): In Progress Bug description: After an instance has been in the rescue state for some time, a periodic task triggers to unrescue the instances: _poll_rescued_instances File nova/notifications.py info_from_instance instance_type = flavors.extract_flavor(instance_ref) File "nova/compute/flavors.py" in extract_flavor instance_type[key] = type_fn(sys_meta[type_key]) KeyError: 'instance_type_memory_mb' This then continues to happen on every run of the periodic task, and starts to fill up the DB with instance faults. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1277054/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1265465] [NEW] xenapi: auto disk config drops boot partition flag
Public bug reported: When the XenAPI driver resizes a boot partition, it does not take care to add back the boot partition flag. With PV images, this is not really needed, because Xen doesn't worry about the partition being bootable, but for HVM images, it is stops the image from booting any more. ** Affects: nova Importance: Medium Status: New ** Tags: xenserver -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1265465 Title: xenapi: auto disk config drops boot partition flag Status in OpenStack Compute (Nova): New Bug description: When the XenAPI driver resizes a boot partition, it does not take care to add back the boot partition flag. With PV images, this is not really needed, because Xen doesn't worry about the partition being bootable, but for HVM images, it is stops the image from booting any more. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1265465/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1262206] [NEW] xenapi: server gets deleted under certain live-migrate error conditions
Public bug reported: Any errors after the XenServer migrate command completes currently can cause the users VM to be deleted. While there should be some cleanup performed, deleting the VM does not make sense for the XenAPI driver. ** Affects: nova Importance: Medium Assignee: John Garbutt (johngarbutt) Status: In Progress ** Tags: xenserver -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1262206 Title: xenapi: server gets deleted under certain live-migrate error conditions Status in OpenStack Compute (Nova): In Progress Bug description: Any errors after the XenServer migrate command completes currently can cause the users VM to be deleted. While there should be some cleanup performed, deleting the VM does not make sense for the XenAPI driver. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1262206/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1188141] Re: Deleting images breaks rescue mode for servers built from said images
This is still happening, lets bring this back ** Changed in: nova Status: Invalid => Triaged -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1188141 Title: Deleting images breaks rescue mode for servers built from said images Status in OpenStack Compute (Nova): Triaged Bug description: Rescue image is considered to be the image from which the instance is built from. If that image is deleted, the rescue fails Consider this scenario, The customer has a snapshot of an instance. He build another instance from the snapshot and deletes the snapshot. Now the customer tries to rescue the instance that was built from the snapshot, it fails because the rescue image is not available. Possible solutions - 1. Recursively find the image that's available: The system_metadata of an instance has the "image_base_image_ref" as property set on it. This points to image from which it was built. Image has instance_uuid, if it's a snapshot. We'll have to recursively find the base_install and use it as rescue image if the snapshot is deleted. 2. Store the corresponding original image reference in all images. So that it's easier to find the rescue image. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1188141/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1225900] Re: xenapi: race condition in get_all_vdis_in_sr
This is bogus ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1225900 Title: xenapi: race condition in get_all_vdis_in_sr Status in OpenStack Compute (Nova): Invalid Bug description: The code in get_all_vdis_in_sr, does not deal with the set of VDIs changing underneath the call. if a VDI gets deleted, it could lead to a "HANDLE_INVALID VDI" error, or similar. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1225900/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 747394] Re: XenServer port needs to clear out vm-data/networking before issuing resetnetwork command
Hmm, thinking about this, its related to issues with inject_network_info. Currently people call inject_network_info, and potentially alter xenstore, then call resetnetwork. So, sadly, this behaviour is now being used as a feature. ** Changed in: nova Status: Confirmed => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/747394 Title: XenServer port needs to clear out vm-data/networking before issuing resetnetwork command Status in OpenStack Compute (Nova): Invalid Bug description: The guest agent will pull networking information from vm- data/networking in xenstore. If Nova chooses a different name than what is already in that directory directory, the guest agent will apply all configurations, even if they are old or conflicting. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/747394/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1218541] Re: openvswith-nova init.d script turn off all networks except management
Information has dried up on this, marking as invalid for now. ** Changed in: nova Status: Incomplete => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1218541 Title: openvswith-nova init.d script turn off all networks except management Status in OpenStack Compute (Nova): Invalid Bug description: Hi! In that bug I comment the process exec by that init.d script => https://bugs.launchpad.net/nova/+bug/1218528 An other problem is: when the process create flows only create physical nic flows and management network flows but if XenServer have other networks (storage, monitoring, etc...)?. The traffic of that networks are drop. In my case I have: xapi1 (bond eth0 and eth1) xapi2 (storage vlan) xapi3 (monitoring vlan) Thanks. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1218541/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1200591] Re: virt xenapi driver does not retry upload_image on a socket error
I don't think we have seen this recently. Lets close this out for now, and keep watching. ** Changed in: nova Status: Incomplete => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1200591 Title: virt xenapi driver does not retry upload_image on a socket error Status in OpenStack Compute (Nova): Invalid Bug description: As of now only XenAPI failures are retried in the upload_image method (for the glance_num_retries ). We need to retry on "error: [Errno 104] Connection reset by peer". To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1200591/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp