[Yahoo-eng-team] [Bug 1655307] Re: Sequential API requests in instance list as a bottleneck
Reviewed: https://review.openstack.org/426496 Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=df194c8b4c25c6128f03a98fd0e628ba14b743ff Submitter: Jenkins Branch:master commit df194c8b4c25c6128f03a98fd0e628ba14b743ff Author: Mateusz KowalskiDate: Sat Jan 28 11:52:06 2017 +0100 Make API calls in Instances view parallel In order to increase rendering speed, make api.nova.server_list, api.network.servers_update_addresses, api.nova.flavor_list and api.glance.image_list_detailed parallel. For admin panel also api.keystone.tenant_list is parallelized. Closes-bug: #1655307 Change-Id: I83150d3963f1233edc9efbcdac299520dedbed3c ** Changed in: horizon Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1655307 Title: Sequential API requests in instance list as a bottleneck Status in OpenStack Dashboard (Horizon): Fix Released Bug description: When loading instances list, the following API calls are executed sequentially * nova server list * neutron update addresses * nova flavor list * glance image list Only the first one really needs to be the first one, three following ones can be run in parallel. Currently it acts as a big bottleneck in environments with a lot of servers, flavours and images. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1655307/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673637] [NEW] cloud-init - Hosts in softlayer receiving warning
Public bug reported: Newly provisioned Xenial hosts in softlayer bootstrapped via cloud-init are getting the following message. ** # A new feature in cloud-init identified possible datasources for# # this system as:# # ['Ec2', 'None'] # # However, the datasource used was: NoCloud # ## # In the future, cloud-init will only attempt to use datasources that# # are identified or specifically configured. # # For more information see # # https://bugs.launchpad.net/bugs/1669675 # ## # If you are seeing this message, please file a bug against # # cloud-init at # #https://bugs.launchpad.net/cloud-init/+filebug?field.tags=dsid # # Make sure to include the cloud provider your instance is # # running on.# ## # After you have filed a bug, you can disable this warning by launching # # your instance with the cloud-config below, or putting that content # # into /etc/cloud/cloud.cfg.d/99-warnings.cfg# ## # #cloud-config # # warnings: # # dsid_missing_source: off # ** I'm running what I believe is the latest version of cloud-init: $ dpkg -s cloud-init Package: cloud-init Status: install ok installed Priority: extra Section: admin Installed-Size: 1417 Maintainer: Scott MoserArchitecture: all Version: 0.7.9-48-g1c795b9-0ubuntu1~16.04.1 I'm able to get rid of the message following the instructions provided, but posting the bug report as instructed. ** Affects: cloud-init Importance: Undecided Status: New ** Tags: dsid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1673637 Title: cloud-init - Hosts in softlayer receiving warning Status in cloud-init: New Bug description: Newly provisioned Xenial hosts in softlayer bootstrapped via cloud-init are getting the following message. ** # A new feature in cloud-init identified possible datasources for# # this system as:# # ['Ec2', 'None'] # # However, the datasource used was: NoCloud # ## # In the future, cloud-init will only attempt to use datasources that# # are identified or specifically configured. # # For more information see # # https://bugs.launchpad.net/bugs/1669675 # ## # If you are seeing this message, please file a bug against # # cloud-init at # #https://bugs.launchpad.net/cloud-init/+filebug?field.tags=dsid # # Make sure to include the cloud provider your instance is # # running on.# ## # After you have filed a bug, you can disable this warning by launching # # your instance with the cloud-config below, or putting that content # # into /etc/cloud/cloud.cfg.d/99-warnings.cfg# ## # #cloud-config # # warnings: # # dsid_missing_source: off # ** I'm running what I believe is the latest version of cloud-init: $ dpkg -s cloud-init Package: cloud-init Status: install ok installed Priority: extra Section: admin
[Yahoo-eng-team] [Bug 1673628] [NEW] api-ref: 'tags' field is not in response parameters docs for "GET /servers/{server_id}"
Public bug reported: As of the 2.26 microversion, the 'tags' field is in the response for showing details about a given server, see the API sample here: https://github.com/openstack/nova/blob/master/doc/api_samples/os-server- tags/v2.26/server-tags-show-details-resp.json The tags field is not in the response table description for that API though: https://developer.openstack.org/api-ref/compute/?expanded=list-servers- detail,show-server-details-detail#show-server-details ** Affects: nova Importance: Medium Status: Confirmed ** Tags: api-ref low-hanging-fruit -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673628 Title: api-ref: 'tags' field is not in response parameters docs for "GET /servers/{server_id}" Status in OpenStack Compute (nova): Confirmed Bug description: As of the 2.26 microversion, the 'tags' field is in the response for showing details about a given server, see the API sample here: https://github.com/openstack/nova/blob/master/doc/api_samples/os- server-tags/v2.26/server-tags-show-details-resp.json The tags field is not in the response table description for that API though: https://developer.openstack.org/api-ref/compute/?expanded=list- servers-detail,show-server-details-detail#show-server-details To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673628/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1671648] Re: Instances are not rescheduled after deploy fails
Reviewed: https://review.openstack.org/444106 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=cb4ce72f5f092644aa9b84fa58bcb9fd89b6bedc Submitter: Jenkins Branch:master commit cb4ce72f5f092644aa9b84fa58bcb9fd89b6bedc Author: ShunliZhouDate: Fri Mar 10 14:05:57 2017 +0800 Add populate_retry to schedule_and_build_instances When boot an instance and failed on the compute node, nova will not retry to boot on other host. Since https://review.openstack.org/#/c/319379/ change the create instance workflow and called schedule_and_build_instances which not populate the retry into filter properties. So nova will not retry when boot on compute fail. This patch populate retry to instance properties when call schedule_and_build_instances. Change-Id: Ifdaddcd265a7fe8282499e27043936f8212610ad Closes-Bug: #1671648 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1671648 Title: Instances are not rescheduled after deploy fails Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) ocata series: In Progress Bug description: Steps to reproduce: Pre-step. Need to force the deploy to fail in such a way that it can be rescheduled. For testing I just forced it to fail by adding raise nova.exception.ComputeResourcesUnavailable('forced failure') during the instance spawn on the host. 1. Make sure environment is set to retry failed deploys. 2. Attempt to deploy VM and wait for it to fail. Expected result: Failed instance is rescheduled and attempted on another host. Actual result: Deploy fails but is not rescheduled. I am just beginning to experiment with ocata build from early March. I found that when an instance fails to deploy and throws a RescheduledException, it is not getting rescheduled as expected. The problem appears to be that the filter_properties['retry'] is not getting set during initial deploy. On initial deploy nova.conductor.manager.schedule_and_build_instances() schedules the build_request and creates the instance object. That method also creates the filter properties (filter_props) that is passed on to compute_rpcapi.build_and_run_instance(). The problem is that scheduler_utils.populate_retry() is not called before the filter_props is passed on to the build call. When the deploy later fails on the host nova.compute.manager._do_build_and_run_instance() catches the RescheduledException but does not try and reschedule it because filter_properties.get('retry') returns None. In the past it looks like populate_retry() was called in by nova.conductor.manager.build_instances() during the initial deploy. I'm not seeing build_instances() get called during initial deploy after switching to ocata. As an experiment I added scheduler_utils.populate_retry(filter_props, build_request.instance_uuid) immediately after filter_props is set in schedule_and_build_instances(). Afterward I do see the instances get rescheduled. I also noticed nova.conductor.manager.build_instances() gets called for each attempt after the first. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1671648/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673613] [NEW] cli command 'openstack server list' fails
Public bug reported: I installed ocata on a three node config (one controller, two compute). When executing openstack server list on the controller node, I get Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. (HTTP 500) (Request-ID: req-7544be6c-3302-4a94-94ac-834a50696ac6) openstack --debug server list START with options: [u'--debug', u'server', u'list'] options: Namespace(access_key='', access_secret='***', access_token='***', access_token_endpoint='', access_token_type='', aodh_endpoint='', auth_type='', auth_url='http://controller:35357/v3', cacert=None, cert='', client_id='', client_secret='***', cloud='', code='', consumer_key='', consumer_secret='***', debug=True, default_domain='default', default_domain_id='', default_domain_name='', deferred_help=False, discovery_endpoint='', domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, interface='', key='', log_file=None, old_profile=None, openid_scope='', os_alarming_api_version='2', os_beta_command=False, os_clustering_api_version='1', os_compute_api_version='', os_container_infra_api_version='1', os_data_processing_api_version='1.1', os_data_processing_url='', os_database_api_version='1', os_dns_api_version='2', os_identity_api_version='3', os_image_api_version='2', os_key_manager_api_version='1', os_network_api_version='', os_o bject_api_version='', os_orchestration_api_version='1', os_project_id=None, os_project_name=None, os_queues_api_version='2', os_volume_api_version='', os_workflow_api_version='2', passcode='', password='***', profile=None, project_domain_id='', project_domain_name='default', project_id='', project_name='admin', protocol='', redirect_uri='', region_name='RegionOne', roles='', timing=False, token='***', trust_id='', url='', user_domain_id='', user_domain_name='default', user_id='', username='admin', verbose_level=3, verify=None) Auth plugin password selected auth_config_hook(): {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:35357/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'clustering_api_version': '1', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'admin', u'container_infra_api_version': '1', 'verbose_level': 3, 'region_name': 'RegionOne', 'api_timeout': None, u'baremetal_api_version': u'1', 'queues_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', u'container_api_version': u'1', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': '1', 'timing': False, 'password': '***', u'application_catalog_api_version': u'1', 'cacert': None, u'key_mana ger_api_version': '1', u'metering_api_version': u'2', 'deferred_help': False, u'identity_api_version': '3', 'workflow_api_version': '2', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'status': u'active', 'alarming_api_version': '2', 'debug': True, u'interface': None, u'disable_vendor_agent': {}} defaults: {u'auth_type': 'password', u'status': u'active', u'compute_api_version': u'2', 'key': None, u'database_api_version': u'1.0', 'api_timeout': None, u'baremetal_api_version': u'1', u'image_api_version': u'2', u'container_infra_api_version': u'1', u'metering_api_version': u'2', u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', u'orchestration_api_version': u'1', 'cacert': None, u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', u'application_catalog_api_version': u'1', u'key_manager_api_version': u'v1', 'verify': True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', u'container_api_version': u'1', u'dns_api_version': u'2', u'object_store_api_version': u'1', u'interface': None, u'disable_vendor_agent': {}} cloud cfg: {'auth_type': 'password', 'beta_command': False, u'compute_api_version': u'2', 'key': None, u'database_api_version': '1', 'data_processing_api_version': '1.1', 'auth_url': 'http://controller:35357/v3', u'network_api_version': u'2', u'message': u'', u'image_format': u'qcow2', 'networks': [], u'image_api_version': '2', 'clustering_api_version': '1', 'verify': True, u'dns_api_version': '2', u'object_store_api_version': u'1', 'username': 'admin', u'container_infra_api_version': '1', 'verbose_level': 3, 'region_name': 'RegionOne', 'api_timeout': None, u'baremetal_api_version': u'1', 'queues_api_version': '2', 'auth': {'user_domain_name': 'default', 'project_name': 'admin', 'project_domain_name': 'default'}, 'default_domain': 'default', u'container_api_version': u'1',
[Yahoo-eng-team] [Bug 1667736] Re: gate-neutron-fwaas-dsvm-functional failure after recent localrc change
This bug was fixed in the package neutron-fwaas - 1:10.0.0-0ubuntu2 --- neutron-fwaas (1:10.0.0-0ubuntu2) zesty; urgency=medium * d/p/gate_hook-stop-including-localrc.patch: Cherry-pick patch from stable/ocata to fix tempest failure (LP: #1667736). -- Corey BryantThu, 16 Mar 2017 15:22:58 -0400 ** Changed in: neutron-fwaas (Ubuntu) Status: Triaged => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1667736 Title: gate-neutron-fwaas-dsvm-functional failure after recent localrc change Status in Ubuntu Cloud Archive: Triaged Status in neutron: Fix Released Status in neutron-fwaas package in Ubuntu: Fix Released Bug description: eg. http://logs.openstack.org/59/286059/1/check/gate-neutron-fwaas- dsvm-functional/a0f2285/console.html 2017-02-24 15:27:58.187720 | + /opt/stack/new/neutron-fwaas/neutron_fwaas/tests/contrib/gate_hook.sh:main:26 : source /opt/stack/new/devstack/localrc 2017-02-24 15:27:58.187833 | /opt/stack/new/neutron-fwaas/neutron_fwaas/tests/contrib/gate_hook.sh: line 26: /opt/stack/new/devstack/localrc: No such file or directory To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1667736/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1672921] Re: Functional gate job broken due to os-log-merger crashing on null-bytes
Reviewed: https://review.openstack.org/445884 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=75612f200b984c9f42558ed951ae19e17c3633d9 Submitter: Jenkins Branch:master commit 75612f200b984c9f42558ed951ae19e17c3633d9 Author: Miguel Angel AjoDate: Wed Mar 15 12:29:32 2017 +0100 Bump os-log-merger version in post_test_hook The 1.0.6 version of os-log-merger exploded in cases where log files contained \0 characters. 1.1.0 fixes such issue. Change-Id: I470d38dfd5dd7e9cf6317d9f2f32875f33f3a996 Closes-Bug: #1672921 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1672921 Title: Functional gate job broken due to os-log-merger crashing on null-bytes Status in neutron: Fix Released Status in os-log-merger: Fix Committed Bug description: gate-neutron-dsvm-functional-ubuntu-xenial seems to be failing even though tests are being passed. 1. http://logs.openstack.org/53/319353/3/check/gate-neutron-dsvm-functional-ubuntu-xenial/b36e01e/ 2. http://logs.openstack.org/25/338625/33/check/gate-neutron-dsvm-functional-ubuntu-xenial/fc680a7/ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1672921/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673602] [NEW] circular reference error in gate
Public bug reported: Spotted this failure in the gate: http://logs.openstack.org/45/446145/2/gate/gate-tempest-dsvm-neutron- full-ubuntu-xenial/5a3bb88/logs/screen-n-api.txt.gz?level=TRACE http://logs.openstack.org/45/446145/2/gate/gate-tempest-dsvm-neutron- full-ubuntu-xenial/5a3bb88/logs/testr_results.html.gz (Sorry about poor bug report quality, strapped for time) ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673602 Title: circular reference error in gate Status in OpenStack Compute (nova): New Bug description: Spotted this failure in the gate: http://logs.openstack.org/45/446145/2/gate/gate-tempest-dsvm-neutron- full-ubuntu-xenial/5a3bb88/logs/screen-n-api.txt.gz?level=TRACE http://logs.openstack.org/45/446145/2/gate/gate-tempest-dsvm-neutron- full-ubuntu-xenial/5a3bb88/logs/testr_results.html.gz (Sorry about poor bug report quality, strapped for time) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673602/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673430] Re: Launch Instance error ImageNotFound
Can you add steps, environment details etc? ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673430 Title: Launch Instance error ImageNotFound Status in OpenStack Compute (nova): Invalid Bug description: 2017-03-16 19:09:57.370 29479 INFO nova.compute.claims [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Claim successful 2017-03-16 19:09:57.988 29479 INFO nova.virt.osinfo [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] Cannot load Libosinfo: (No module named Libosinfo) 2017-03-16 19:09:58.505 29479 INFO nova.virt.libvirt.driver [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Creating image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Instance failed to spawn 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Traceback (most recent call last): 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2218, in _build_resources 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] yield resources 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2064, in _build_and_run_instance 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] block_device_info=block_device_info) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2772, in spawn 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] admin_pass=admin_password) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3190, in _create_image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] instance, size, fallback_from_host) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 6789, in _try_fetch_image_cache 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] size=size) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 251, in cache 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] *args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 593, in create_image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] prepare_template(target=base, max_size=size, *args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in inner 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] return f(*args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 241, in fetch_func_sync 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] fetch_func(target=target, *args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 429, in fetch_image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056]
[Yahoo-eng-team] [Bug 1673593] [NEW] api-ref: descriptions for the various host fields in server GET response are useless
Public bug reported: This came up in a review for the mistaken use of the hypervisor_hostname field in the server GET response: https://review.openstack.org/#/c/446685/2/nova/tests/functional/regressions/test_bug_1671648.py@149 Looking at the API ref: https://developer.openstack.org/api-ref/compute/?expanded=show-server- details-detail#id25 There are three 'host' related fields in the response: 1. hostId - The ID of the host. 2. OS-EXT-SRV-ATTR:host - The host name. Appears in the response for administrative users only. 3. OS-EXT-SRV-ATTR:hypervisor_hostname - The hypervisor host name. Appears in the response for administrative users only. The hostId is an obfuscated id for the hostname in the guest (server instance). The 'host' is the name of the compute host that the nova-compute service is running on. This comes from the 'host' configuration option value in nova.conf. The 'hypervisor_hostname' is the hostname of the hypervisor, provided by the compute driver, which in most cases is the same as 'host' *except* in the case of Ironic in which case it's the Ironic node name. ** Affects: nova Importance: Low Status: Confirmed ** Tags: api-ref -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673593 Title: api-ref: descriptions for the various host fields in server GET response are useless Status in OpenStack Compute (nova): Confirmed Bug description: This came up in a review for the mistaken use of the hypervisor_hostname field in the server GET response: https://review.openstack.org/#/c/446685/2/nova/tests/functional/regressions/test_bug_1671648.py@149 Looking at the API ref: https://developer.openstack.org/api-ref/compute/?expanded=show-server- details-detail#id25 There are three 'host' related fields in the response: 1. hostId - The ID of the host. 2. OS-EXT-SRV-ATTR:host - The host name. Appears in the response for administrative users only. 3. OS-EXT-SRV-ATTR:hypervisor_hostname - The hypervisor host name. Appears in the response for administrative users only. The hostId is an obfuscated id for the hostname in the guest (server instance). The 'host' is the name of the compute host that the nova-compute service is running on. This comes from the 'host' configuration option value in nova.conf. The 'hypervisor_hostname' is the hostname of the hypervisor, provided by the compute driver, which in most cases is the same as 'host' *except* in the case of Ironic in which case it's the Ironic node name. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673593/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1667736] Re: gate-neutron-fwaas-dsvm-functional failure after recent localrc change
** Also affects: neutron-fwaas (Ubuntu) Importance: Undecided Status: New ** Changed in: neutron-fwaas (Ubuntu) Assignee: (unassigned) => Corey Bryant (corey.bryant) ** Also affects: cloud-archive Importance: Undecided Status: New ** Changed in: neutron-fwaas (Ubuntu) Status: New => Triaged ** Changed in: cloud-archive Status: New => Triaged ** Changed in: cloud-archive Importance: Undecided => Medium ** Changed in: neutron-fwaas (Ubuntu) Importance: Undecided => High ** Changed in: cloud-archive Assignee: (unassigned) => Corey Bryant (corey.bryant) ** Changed in: cloud-archive Importance: Medium => High -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1667736 Title: gate-neutron-fwaas-dsvm-functional failure after recent localrc change Status in Ubuntu Cloud Archive: Triaged Status in neutron: Fix Released Status in neutron-fwaas package in Ubuntu: Triaged Bug description: eg. http://logs.openstack.org/59/286059/1/check/gate-neutron-fwaas- dsvm-functional/a0f2285/console.html 2017-02-24 15:27:58.187720 | + /opt/stack/new/neutron-fwaas/neutron_fwaas/tests/contrib/gate_hook.sh:main:26 : source /opt/stack/new/devstack/localrc 2017-02-24 15:27:58.187833 | /opt/stack/new/neutron-fwaas/neutron_fwaas/tests/contrib/gate_hook.sh: line 26: /opt/stack/new/devstack/localrc: No such file or directory To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1667736/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673562] [NEW] Incorrect Security Rules help in Launch Instance workflow
Public bug reported: When using Horizon workflow to launch a new instance, the "Security Group" help page needs to be updated to the correct location of the manage rules page. Currently, it says: "Security groups define a set of IP filter rules that determine how network traffic flows to and from an instance. Users can add additional rules to an existing security group to further define the access options for an instance. To create additional rules, go to the Compute | Access & Security view, then find the security group and click Manage Rules." However, "manage rules" is now under Network | Security Groups -> Manage Rule. The text should be updated to read: "Security groups define a set of IP filter rules that determine how network traffic flows to and from an instance. Users can add additional rules to an existing security group to further define the access options for an instance. To create additional rules, go to the Network | Security Groups view, then find the security group and click Manage Rules." [root@cloud ~]# nova-manage version 15.0.0-1.el7 [root@cloud ~]# rpm -qa | grep -i horizon python-django-horizon-11.0.0-1.el7.noarch puppet-horizon-10.3.0-1.el7.noarch ** Affects: horizon Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1673562 Title: Incorrect Security Rules help in Launch Instance workflow Status in OpenStack Dashboard (Horizon): New Bug description: When using Horizon workflow to launch a new instance, the "Security Group" help page needs to be updated to the correct location of the manage rules page. Currently, it says: "Security groups define a set of IP filter rules that determine how network traffic flows to and from an instance. Users can add additional rules to an existing security group to further define the access options for an instance. To create additional rules, go to the Compute | Access & Security view, then find the security group and click Manage Rules." However, "manage rules" is now under Network | Security Groups -> Manage Rule. The text should be updated to read: "Security groups define a set of IP filter rules that determine how network traffic flows to and from an instance. Users can add additional rules to an existing security group to further define the access options for an instance. To create additional rules, go to the Network | Security Groups view, then find the security group and click Manage Rules." [root@cloud ~]# nova-manage version 15.0.0-1.el7 [root@cloud ~]# rpm -qa | grep -i horizon python-django-horizon-11.0.0-1.el7.noarch puppet-horizon-10.3.0-1.el7.noarch To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1673562/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673375] Re: "ValueError: Circular reference detected" in send_notification
Looking at logstash: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22_send_notification%5C%22%20AND%20message%3A%5C%22ValueError%3A%20Circular%20reference%20detected%5C%22=7d This seems to only show up on that aggregate negative test, and it's probably this error notification wrapper: https://github.com/openstack/nova/blob/2380659e358770a3f36253b93a112b9779a23958/nova/compute/api.py#L4602 The AggregateHostExists duplicate error comes out of the call to "aggregate.add_host(host_name)". I'm not sure what's breaking in the actual notification driver. It also looks like this is happening on non-py35 jobs too. ** Summary changed: - Duplicate entry for key 'uniq_aggregate_hosts0host0aggregate_id + "ValueError: Circular reference detected" in send_notification ** Also affects: oslo.messaging Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673375 Title: "ValueError: Circular reference detected" in send_notification Status in OpenStack Compute (nova): New Status in oslo.messaging: New Bug description: I guess something is wrong with nova, causing a tempest failure. http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35 -ubuntu- xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging context) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging cursor.execute(statement, parameters) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 166, in execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging result = self._query(query) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 322, in _query 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging conn.query(q) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 852, in query 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1053, in _read_query_result 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging result.read() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1336, in read 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging first_packet = self.connection._read_packet() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1010, in _read_packet 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging packet.check_error() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 393, in check_error 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging err.raise_mysql_exception(self._data) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging raise errorclass(errno, errval) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging pymysql.err.IntegrityError: (1062, "Duplicate entry 'ubuntu-xenial-rax-iad-7906449-6' for key 'uniq_aggregate_hosts0host0aggregate_id'") To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673375/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673528] Re: Enable MySQL Cluster Support
Nova uses blueprints and specs for RFEs, not bugs. Since you already have a blueprint and a spec, I'm going to invalid this bug. https://blueprints.launchpad.net/nova/+spec/mysql-cluster-support ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673528 Title: Enable MySQL Cluster Support Status in OpenStack Compute (nova): Invalid Bug description: Implements minor fixes to the SQL Alchemy and Alembic scripts in Nova to support the MySQL Cluster DB. This includes the usage of the boolean mysql_enable_ndb setting from oslo.db from bug 1564110. This allows operators to select from MySQL (InnoDB) or MySQL Cluster (NDB) as the storage engine backend. Additionally, this fixes the api.py and database.py to support this enhancement. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673528/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1649248] Re: Glance image upload wizard does not restrict invalid image files
After discussing this in the OSSP meeting, I will mark this as won't fix for the OSSN, as we already have covered this the recommended actions in several previous OSSNs. There is also a good amount of info in the security guide around protecting end points and access controls available for glance. ** Changed in: ossn Status: New => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1649248 Title: Glance image upload wizard does not restrict invalid image files Status in OpenStack Dashboard (Horizon): New Status in OpenStack Security Advisory: Won't Fix Status in OpenStack Security Notes: Won't Fix Bug description: An unrestricted file upload exists when an application allows users to upload files without proper validation. glance fails to properly validate image files across four key factors including file extension, mime-type, size, and upload frequency. In addition, glance does not appear to scan uploaded files for known malware. Failing to restrict file uploads affects the security of the OpenStack environment in a number of ways. Attacker may commonly use file upload functionality to upload viruses or malware onto trusted servers. In addition to spreading malware, attacker can upload source code files (.aspx and .jsp for example) which may be rendered as valid application pages to end users. Additionally, if users are able to upload files of any size or at any frequency, an attacker may abuse this functionality to exhaust the server’s disk space. Steps To Reproduce: 1. Login to the OpenStack as an admin 2. Click on Images tab and create a new image by uploading a EICAR text file with anti-malware string (EICAR anti-malware test file can be downloaded from http://www.eicar.org/ ) 3. Observe that file is uploaded successfully without any pre-checks being done. The application should validate uploaded files for type and size, and limit how often the user is able to perform uploads. The following validation can be performed: a) If the application requires uploaded files to be of a specific type such as img, vmdk, the application should validate the extension. b) The first four bytes of the file i.e. Magic Numbers can be validated. These first few bytes are known as the file’s ‘Magic Number’ and will uniquely identify the file type. For example all PDF files start with the byte-sequence ‘%PDF’. c) An upper limit on file size can be enforced. In addition to the primary criteria above, all uploaded files should be scanned for known malware/viruses. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1649248/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673528] [NEW] Enable MySQL Cluster Support
Public bug reported: Implements minor fixes to the SQL Alchemy and Alembic scripts in Nova to support the MySQL Cluster DB. This includes the usage of the boolean mysql_enable_ndb setting from oslo.db from bug 1564110. This allows operators to select from MySQL (InnoDB) or MySQL Cluster (NDB) as the storage engine backend. Additionally, this fixes the api.py and database.py to support this enhancement. ** Affects: nova Importance: Undecided Assignee: Octave Orgeron (octave-orgeron) Status: New ** Changed in: nova Assignee: (unassigned) => Octave Orgeron (octave-orgeron) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673528 Title: Enable MySQL Cluster Support Status in OpenStack Compute (nova): New Bug description: Implements minor fixes to the SQL Alchemy and Alembic scripts in Nova to support the MySQL Cluster DB. This includes the usage of the boolean mysql_enable_ndb setting from oslo.db from bug 1564110. This allows operators to select from MySQL (InnoDB) or MySQL Cluster (NDB) as the storage engine backend. Additionally, this fixes the api.py and database.py to support this enhancement. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673528/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673531] [NEW] fullstack test_controller_timeout_does_not_break_connectivity_sigkill(GRE and l2pop, openflow-native_ovsdb-cli) failure
Public bug reported: Logs for failure: http://logs.openstack.org/98/446598/1/check/gate- neutron-dsvm-fullstack-ubuntu-xenial/2e0f93e/logs/dsvm-fullstack- logs/TestOvsConnectivitySameNetworkOnOvsBridgeControllerStop .test_controller_timeout_does_not_break_connectivity_sigkill_GRE-and- l2pop,openflow-native_ovsdb-cli_/ logstash query: http://logstash.openstack.org/#dashboard/file/logstash.json?query=messge%3A%5C%22in%20test_controller_timeout_does_not_break_connectivity_sigkill%5C%22%20AND%20tags%3Aconsole%20AND%20build_name %3Agate-neutron-dsvm-fullstack-ubuntu-xenial ** Affects: neutron Importance: Undecided Status: New ** Tags: fullstack -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1673531 Title: fullstack test_controller_timeout_does_not_break_connectivity_sigkill(GRE and l2pop,openflow-native_ovsdb-cli) failure Status in neutron: New Bug description: Logs for failure: http://logs.openstack.org/98/446598/1/check/gate- neutron-dsvm-fullstack-ubuntu-xenial/2e0f93e/logs/dsvm-fullstack- logs/TestOvsConnectivitySameNetworkOnOvsBridgeControllerStop .test_controller_timeout_does_not_break_connectivity_sigkill_GRE-and- l2pop,openflow-native_ovsdb-cli_/ logstash query: http://logstash.openstack.org/#dashboard/file/logstash.json?query=messge%3A%5C%22in%20test_controller_timeout_does_not_break_connectivity_sigkill%5C%22%20AND%20tags%3Aconsole%20AND%20build_name %3Agate-neutron-dsvm-fullstack-ubuntu-xenial To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1673531/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1638312] Re: EC2 credentials are cached on disk
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.04.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.04.1) xenial-proposed; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-xenial.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:34:10 -0500 ** Changed in: cloud-init (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1638312 Title: EC2 credentials are cached on disk Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] On EC2, instance metadata can include credentials that remain valid for as much as 6 hours. Reading these and allowing them to be pickled represents a potential vulnerability if a snapshot of the disk is taken and shared as part of an AMI. The fix applied was simply to avoid reading the security credentials in cloud-init. [Test Case] 1. Launch an instance on Ec2. 2. Verify broken-ness by verifying 'security-credentials' exists in the pickled object in /var/lib/cloud/instance/obj.pkl 3. enable proposed, update, upgrade 4. clean instance rm -Rf /var/lib/cloud /var/log/cloud-init* 5. reboot 6. go back in and verify no 'security-credentials' are present. [Regression Potential] Low, but possible if someone was using the obj.pkl and expecting to find credentials there. No one should be doing that. Second possibility is if someone was using cloud-init's get_instance_metadata and expected to have the security-credentials there. === End SRU Template === On EC2, instance metadata can include credentials that remain valid for as much as 6 hours. Reading these and allowing them to be pickled represents a potential vulnerability if a snapshot of the disk is taken and shared as part of an AMI. Note that: a.) the simple fact of storing the credentials in a file that is readable only by root is not a serious problem as any attacker on the system has access to the network available data. b.) General care needs to be taken for anyone "capturing" an ami and then making it public. the suggested fix is to skip security-credentials when walking the meta-data tree. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1638312/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe
[Yahoo-eng-team] [Bug 1638312] Re: EC2 credentials are cached on disk
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.10.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.10.1) yakkety; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-yakkety.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:37:28 -0500 ** Changed in: cloud-init (Ubuntu Yakkety) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1638312 Title: EC2 credentials are cached on disk Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] On EC2, instance metadata can include credentials that remain valid for as much as 6 hours. Reading these and allowing them to be pickled represents a potential vulnerability if a snapshot of the disk is taken and shared as part of an AMI. The fix applied was simply to avoid reading the security credentials in cloud-init. [Test Case] 1. Launch an instance on Ec2. 2. Verify broken-ness by verifying 'security-credentials' exists in the pickled object in /var/lib/cloud/instance/obj.pkl 3. enable proposed, update, upgrade 4. clean instance rm -Rf /var/lib/cloud /var/log/cloud-init* 5. reboot 6. go back in and verify no 'security-credentials' are present. [Regression Potential] Low, but possible if someone was using the obj.pkl and expecting to find credentials there. No one should be doing that. Second possibility is if someone was using cloud-init's get_instance_metadata and expected to have the security-credentials there. === End SRU Template === On EC2, instance metadata can include credentials that remain valid for as much as 6 hours. Reading these and allowing them to be pickled represents a potential vulnerability if a snapshot of the disk is taken and shared as part of an AMI. Note that: a.) the simple fact of storing the credentials in a file that is readable only by root is not a serious problem as any attacker on the system has access to the network available data. b.) General care needs to be taken for anyone "capturing" an ami and then making it public. the suggested fix is to skip security-credentials when walking the meta-data tree. To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1638312/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe :
[Yahoo-eng-team] [Bug 1657130] Re: get_data in DataSourceOpenStack.py can time out if metadata service is slow
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.10.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.10.1) yakkety; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-yakkety.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:37:28 -0500 ** Changed in: cloud-init (Ubuntu Yakkety) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1657130 Title: get_data in DataSourceOpenStack.py can time out if metadata service is slow Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] On heavily loaded openstack metadata services, cloud-init may hit a timeout and not properly retry when waiting longer or retring would allow it to succeed. cloud-init contained a setting to configure this but it was not used in all cases. The change here enabled usage of timeout and retry for. [Test Case] 1. Launch an instance on openstack. 2. Verify inconsistent use of 'timeout' in /var/log/cloud-init.log $ grep http://169.254.169.254/openstack /var/log/cloud-init.log | grep 0/ | head -n 2 2017-03-03 16:51:23,824 - url_helper.py[DEBUG]: [0/1] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'method': 'GET', 'timeout': 10.0} configuration 2017-03-03 16:51:24,384 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'method': 'GET', 'timeout': 5.0} configuration 3. enable proposed, update, upgrade 4. clean rm -Rf /var/lib/cloud /var/log/cloud-init* 5. reboot 6. re-check step 2, expect see 'timeout' is consistent. [Regression Potential] low chance for regression. Slower boot times but more reliable on a non-perform ant metadata service. === End SRU Template === cloud-init sometimes times out and fails to fetch metadata in the OpenStack environment when the Controller node is under high workload. The default timeout value is 5 seconds and it may be too small in some cases where the Controller node is too busy to respond to the metadata request from the instance in time. There is a 'timeout' configuration setting, as in... datasource: OpenStack: timeout: 30 ...but
[Yahoo-eng-team] [Bug 1657130] Re: get_data in DataSourceOpenStack.py can time out if metadata service is slow
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.04.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.04.1) xenial-proposed; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-xenial.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:34:10 -0500 ** Changed in: cloud-init (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1657130 Title: get_data in DataSourceOpenStack.py can time out if metadata service is slow Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] On heavily loaded openstack metadata services, cloud-init may hit a timeout and not properly retry when waiting longer or retring would allow it to succeed. cloud-init contained a setting to configure this but it was not used in all cases. The change here enabled usage of timeout and retry for. [Test Case] 1. Launch an instance on openstack. 2. Verify inconsistent use of 'timeout' in /var/log/cloud-init.log $ grep http://169.254.169.254/openstack /var/log/cloud-init.log | grep 0/ | head -n 2 2017-03-03 16:51:23,824 - url_helper.py[DEBUG]: [0/1] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'method': 'GET', 'timeout': 10.0} configuration 2017-03-03 16:51:24,384 - url_helper.py[DEBUG]: [0/6] open 'http://169.254.169.254/openstack' with {'url': 'http://169.254.169.254/openstack', 'allow_redirects': True, 'headers': {'User-Agent': 'Cloud-Init/0.7.9'}, 'method': 'GET', 'timeout': 5.0} configuration 3. enable proposed, update, upgrade 4. clean rm -Rf /var/lib/cloud /var/log/cloud-init* 5. reboot 6. re-check step 2, expect see 'timeout' is consistent. [Regression Potential] low chance for regression. Slower boot times but more reliable on a non-perform ant metadata service. === End SRU Template === cloud-init sometimes times out and fails to fetch metadata in the OpenStack environment when the Controller node is under high workload. The default timeout value is 5 seconds and it may be too small in some cases where the Controller node is too busy to respond to the metadata request from the instance in time. There is a 'timeout' configuration setting, as in... datasource: OpenStack: timeout: 30
[Yahoo-eng-team] [Bug 1657940] Re: eni rendering dhcp6 writes aliases fails to bring up dhcp6
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.10.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.10.1) yakkety; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-yakkety.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:37:28 -0500 ** Changed in: cloud-init (Ubuntu Yakkety) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1657940 Title: eni rendering dhcp6 writes aliases fails to bring up dhcp6 Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] cloud-init rendered ENI (/etc/network/interfaces) configuration using "aliases" (eth0:1). That fails in some scenarios to bring up all interfaces on boot. Most specifically problematic is dhcp with ipv4 and dhcp with ipv6. In that case ipv6 dhcp just would not come up on boot. [Test Case] There are unit tests to verify that the new network configuration is written as expected. Additionally, this is now using the same ENI rendering format that curtin uses, which has good test cases in its 'vmtest'. [Regression Potential] Users who had done: ifup eth0:1 will no longer be able to do that. However now this will reliably work: ifup eth0 to bring up all ip addresses on eth0. === End SRU Template === Currently, a config like this: | version: 1 | config: |- 'type': 'physical' | 'name': 'iface0' | 'subnets': |- {'type': 'dhcp4'} |- {'type': 'dhcp6'} Will render: | auto lo | iface lo inet loopback | | auto iface0 | iface iface0 inet dhcp | post-up ifup iface0:1 | | | auto iface0:1 | iface iface0:1 inet6 dhcp Below is an example test case that shows the output. Heres the problem: $ sudo sh -c 'ifdown eth0; ifup eth0' $ sudo sh -c 'ifdown eth0; ifup eth0' Killed old client process Internet Systems Consortium DHCP Client 4.3.3 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on LPF/eth0/06:b3:0a:3a:2d:e3 Sending on LPF/eth0/06:b3:0a:3a:2d:e3 Sending on Socket/fallback DHCPRELEASE on eth0 to 172.31.16.1 port 67 (xid=0x32b625f1) Internet Systems Consortium DHCP Client 4.3.3 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit
[Yahoo-eng-team] [Bug 1660385] Re: Alert user of Ec2 Datasource on lookalike cloud
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.04.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.04.1) xenial-proposed; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-xenial.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:34:10 -0500 ** Changed in: cloud-init (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1660385 Title: Alert user of Ec2 Datasource on lookalike cloud Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] Opportunistic polling of the Ec2 Metadata service, which lives at 169.254.169.254 can be problematic for numerous reasons including timeouts. In this first phase of SRU, the code that has been added will be set to a warn-only mode. In 16.04, if cloud-init finds it is using a EC2 Metadata Service but not running on Amazon AWS, it will warn the user. In 16.10, it will warn the user and sleep 10 seconds to increase the likelyhood of being noticed. [Test Case] a.) check warnings are seen on openstack configured to use ec2 - launch instance on openstack (it will use OpenStack MD) - enable proposed upgrade - rm -Rf /var/lib/cloud /var/log/cloud-init* - dpkg-reconfigure cloud-init # select 'Ec2' and 'None' only - sudo reboot - ssh in. you should see a warning. The warning instructs you to silence the warning by putting the following in /etc/cloud/cloud.cfg.d/99-ec2-datasource.cfg. Do that. | datasource: | Ec2: | strict_id: false - rm -Rf /var/lib/cloud/ /var/log/cloud* - reboot - ssh in. you should not see a warning. [Regression Potential] There is real regression potential here. That is why we have announced this fairly widely and also are putting this into place with warnings only first. After some time is passed, further SRUs will put more strict behavior in place. [Other Info] We've announced this fairly widely on mailing lists https://lists.ubuntu.com/archives/ubuntu-devel/2017-February/039697.html === End SRU Template === Many cloud providers mimic the EC2 Metadata service [1] in order to provide a level of EC2 compatibility for images. This is quite useful and allows image portability. Because this is a network based metadata service, cloud-init opportunistically poll an IPv4 link local address
[Yahoo-eng-team] [Bug 1657940] Re: eni rendering dhcp6 writes aliases fails to bring up dhcp6
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.04.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.04.1) xenial-proposed; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-xenial.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:34:10 -0500 ** Changed in: cloud-init (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1657940 Title: eni rendering dhcp6 writes aliases fails to bring up dhcp6 Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] cloud-init rendered ENI (/etc/network/interfaces) configuration using "aliases" (eth0:1). That fails in some scenarios to bring up all interfaces on boot. Most specifically problematic is dhcp with ipv4 and dhcp with ipv6. In that case ipv6 dhcp just would not come up on boot. [Test Case] There are unit tests to verify that the new network configuration is written as expected. Additionally, this is now using the same ENI rendering format that curtin uses, which has good test cases in its 'vmtest'. [Regression Potential] Users who had done: ifup eth0:1 will no longer be able to do that. However now this will reliably work: ifup eth0 to bring up all ip addresses on eth0. === End SRU Template === Currently, a config like this: | version: 1 | config: |- 'type': 'physical' | 'name': 'iface0' | 'subnets': |- {'type': 'dhcp4'} |- {'type': 'dhcp6'} Will render: | auto lo | iface lo inet loopback | | auto iface0 | iface iface0 inet dhcp | post-up ifup iface0:1 | | | auto iface0:1 | iface iface0:1 inet6 dhcp Below is an example test case that shows the output. Heres the problem: $ sudo sh -c 'ifdown eth0; ifup eth0' $ sudo sh -c 'ifdown eth0; ifup eth0' Killed old client process Internet Systems Consortium DHCP Client 4.3.3 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/ Listening on LPF/eth0/06:b3:0a:3a:2d:e3 Sending on LPF/eth0/06:b3:0a:3a:2d:e3 Sending on Socket/fallback DHCPRELEASE on eth0 to 172.31.16.1 port 67 (xid=0x32b625f1) Internet Systems Consortium DHCP Client 4.3.3 Copyright 2004-2015 Internet Systems Consortium. All rights reserved. For info, please visit
[Yahoo-eng-team] [Bug 1660385] Re: Alert user of Ec2 Datasource on lookalike cloud
This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.10.1 --- cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.10.1) yakkety; urgency=medium * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d * debian/patches/ds-identify-behavior-yakkety.patch: adjust default behavior of ds-identify for SRU (LP: #1669675, #1660385). * New upstream snapshot. - Support warning if the used datasource is not in ds-identify's list (LP: #1669675). - DatasourceEc2: add warning message when not on AWS. (LP: #1660385) - Z99-cloudinit-warnings: Add profile.d script for showing warnings on - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary execute bit in permissions. - (RedHat) net: correct errors in cloudinit/net/sysconfig.py [Lars Kellogg-Stedman] - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty - Fix eni rendering of multiple IPs per interface [Ryan Harper] (LP: #1657940) - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized [Lars Kellogg-Stedman] - EC2: Do not cache security credentials on disk [Andrew Jorgensen] (LP: #1638312) - OpenStack: Use timeout and retries from config in get_data. [Lars Kellogg-Stedman] (LP: #1657130) - Fixed Misc issues related to VMware customization. [Sankar Tanguturi] - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman] - Get early logging logged, including failures of cmdline url. - test / doc / build environment changes - Remove style checking during build and add latest style checks to tox [Joshua Powers] - code-style: make master pass pycodestyle (2.3.1) cleanly, currently [Joshua Powers] - Fix small typo and change iso-filename for consistency - tools/mock-meta: support python2 or python3 and ipv6 in both. - tests: remove executable bit on test_net, so it runs, and fix it. - tests: No longer monkey patch httpretty for python 3.4.2 - reset httppretty for each test [Lars Kellogg-Stedman] - build: fix running Make on a branch with tags other than master - doc: Fix typos and clarify some aspects of the part-handler [Erik M. Bray] - doc: add some documentation on OpenStack datasource. - Fix minor docs typo: perserve > preserve [Jeremy Bicha] - validate-yaml: use python rather than explicitly python3 -- Scott MoserMon, 06 Mar 2017 16:37:28 -0500 ** Changed in: cloud-init (Ubuntu Yakkety) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1660385 Title: Alert user of Ec2 Datasource on lookalike cloud Status in cloud-init: Fix Committed Status in cloud-init package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Fix Released Status in cloud-init source package in Yakkety: Fix Released Bug description: === Begin SRU Template === [Impact] Opportunistic polling of the Ec2 Metadata service, which lives at 169.254.169.254 can be problematic for numerous reasons including timeouts. In this first phase of SRU, the code that has been added will be set to a warn-only mode. In 16.04, if cloud-init finds it is using a EC2 Metadata Service but not running on Amazon AWS, it will warn the user. In 16.10, it will warn the user and sleep 10 seconds to increase the likelyhood of being noticed. [Test Case] a.) check warnings are seen on openstack configured to use ec2 - launch instance on openstack (it will use OpenStack MD) - enable proposed upgrade - rm -Rf /var/lib/cloud /var/log/cloud-init* - dpkg-reconfigure cloud-init # select 'Ec2' and 'None' only - sudo reboot - ssh in. you should see a warning. The warning instructs you to silence the warning by putting the following in /etc/cloud/cloud.cfg.d/99-ec2-datasource.cfg. Do that. | datasource: | Ec2: | strict_id: false - rm -Rf /var/lib/cloud/ /var/log/cloud* - reboot - ssh in. you should not see a warning. [Regression Potential] There is real regression potential here. That is why we have announced this fairly widely and also are putting this into place with warnings only first. After some time is passed, further SRUs will put more strict behavior in place. [Other Info] We've announced this fairly widely on mailing lists https://lists.ubuntu.com/archives/ubuntu-devel/2017-February/039697.html === End SRU Template === Many cloud providers mimic the EC2 Metadata service [1] in order to provide a level of EC2 compatibility for images. This is quite useful and allows image portability. Because this is a network based metadata service, cloud-init opportunistically poll an IPv4 link local address
[Yahoo-eng-team] [Bug 1673427] Re: mysql OOM ( Out of sort memory, consider increasing server sort buffer size )
Fixed by https://review.openstack.org/#/c/446196/ ** Changed in: devstack Status: New => Fix Released ** Changed in: neutron Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1673427 Title: mysql OOM ( Out of sort memory, consider increasing server sort buffer size ) Status in devstack: Fix Released Status in neutron: Invalid Bug description: Neutron fails with: http://logs.openstack.org/84/445884/2/gate/gate-neutron-dsvm-api- ubuntu-xenial/800e806/logs/screen-q-svc.txt.gz 2017-03-15 21:02:19.418 30853 ERROR neutron.api.v2.resource DBError: (pymysql.err.InternalError) (1038, u'Out of sort memory, consider increasing server sort buffer size') [SQL: u'SELECT networks.id AS network_id, subnets.id AS subnet_id, subnets.cidr AS subnets_cidr, subnets.ip_version AS subnets_ip_version, networks.name AS network_name, networks.project_id AS networks_project_id, subnets.name AS subnet_name, count(ipallocations.subnet_id) AS used_ips \nFROM networks LEFT OUTER JOIN subnets ON networks.id = subnets.network_id LEFT OUTER JOIN ipallocations ON subnets.id = ipallocations.subnet_id GROUP BY networks.id, subnets.id, subnets.cidr, subnets.ip_version, networks.name, networks.project_id, subnets.name'] 2017-03-15 21:02:19.418 30853 ERROR neutron.api.v2.resource So may we consider changing the sort buffer size? To manage notifications about this bug go to: https://bugs.launchpad.net/devstack/+bug/1673427/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1602057] Re: (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list
** No longer affects: ubuntu ** No longer affects: Ubuntu Xenial -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1602057 Title: (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list Status in Ubuntu Cloud Archive: Fix Released Status in Ubuntu Cloud Archive mitaka series: Triaged Status in Ubuntu Cloud Archive newton series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) mitaka series: Won't Fix Status in OpenStack Compute (nova) newton series: Fix Committed Status in nova package in Ubuntu: New Status in nova source package in Xenial: New Bug description: 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager [req-d5d5d486-b488-4429-bbb5-24c9f19ff2c0 - - - - -] Error updating resources for node controller. 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager Traceback (most recent call last): 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6726, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager rt.update_available_resource(context) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 500, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager resources = self.driver.get_available_resource(self.nodename) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5728, in get_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager disk_over_committed = self._get_disk_over_committed_size_total() 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7397, in _get_disk_over_committed_size_total 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager local_instances[guest.uuid], bdms[guest.uuid]) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager KeyError: '0a5c5743-9555-4dfd-b26e-198449ebeee5' 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1602057/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1602057] Re: (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list
** Also affects: nova (Ubuntu) Importance: Undecided Status: New ** Also affects: nova (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: Ubuntu Xenial Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1602057 Title: (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list Status in Ubuntu Cloud Archive: Fix Released Status in Ubuntu Cloud Archive mitaka series: Triaged Status in Ubuntu Cloud Archive newton series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) mitaka series: Won't Fix Status in OpenStack Compute (nova) newton series: Fix Committed Status in nova package in Ubuntu: New Status in nova source package in Xenial: New Bug description: 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager [req-d5d5d486-b488-4429-bbb5-24c9f19ff2c0 - - - - -] Error updating resources for node controller. 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager Traceback (most recent call last): 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6726, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager rt.update_available_resource(context) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 500, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager resources = self.driver.get_available_resource(self.nodename) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5728, in get_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager disk_over_committed = self._get_disk_over_committed_size_total() 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7397, in _get_disk_over_committed_size_total 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager local_instances[guest.uuid], bdms[guest.uuid]) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager KeyError: '0a5c5743-9555-4dfd-b26e-198449ebeee5' 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1602057/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1602057] Re: (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list
** Changed in: cloud-archive/newton Status: Triaged => Fix Released ** Changed in: cloud-archive/mitaka Assignee: (unassigned) => Edward Hope-Morley (hopem) ** Also affects: ubuntu Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1602057 Title: (libvirt) KeyError updating resources for some node, guest.uuid is not in BDM list Status in Ubuntu Cloud Archive: Fix Released Status in Ubuntu Cloud Archive mitaka series: Triaged Status in Ubuntu Cloud Archive newton series: Fix Released Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) mitaka series: Won't Fix Status in OpenStack Compute (nova) newton series: Fix Committed Status in nova package in Ubuntu: New Status in nova source package in Xenial: New Bug description: 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager [req-d5d5d486-b488-4429-bbb5-24c9f19ff2c0 - - - - -] Error updating resources for node controller. 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager Traceback (most recent call last): 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 6726, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager rt.update_available_resource(context) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 500, in update_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager resources = self.driver.get_available_resource(self.nodename) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5728, in get_available_resource 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager disk_over_committed = self._get_disk_over_committed_size_total() 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7397, in _get_disk_over_committed_size_total 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager local_instances[guest.uuid], bdms[guest.uuid]) 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager KeyError: '0a5c5743-9555-4dfd-b26e-198449ebeee5' 2016-07-12 09:54:36.021 10056 ERROR nova.compute.manager To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1602057/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673483] [NEW] libvirt: test_attach_volume_shelved_or_offload_server times out waiting for device detach (which fails)
Public bug reported: http://logs.openstack.org/75/446175/1/gate/gate-tempest-dsvm-neutron- full-ubuntu-xenial/479c7bf/console.html#_2017-03-16_01_31_33_688563 The test teardown times out waiting for the volume to be detached from the server because the volume detach fails due to a libvirt i/o error during detach: http://logs.openstack.org/75/446175/1/gate/gate-tempest-dsvm-neutron- full-ubuntu- xenial/479c7bf/logs/screen-n-cpu.txt.gz?#_2017-03-16_01_01_32_897 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [req-ac93339e-9eb1-4104-8743-5e5626253ab8 tempest-AttachVolumeShelveTestJSON-116820638 tempest-AttachVolumeShelveTestJSON-116820638] [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] Failed to detach volume dc49318a-e51f-4812-97a3-64bf93e522eb from /dev/vdc 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] Traceback (most recent call last): 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/opt/stack/new/nova/nova/compute/manager.py", line 4833, in _driver_detach_volume 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] encryption=encryption) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1334, in detach_volume 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] live=live) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 414, in detach_device_with_retry 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] _try_detach_device(conf, persistent, live) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 408, in _try_detach_device 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] raise exception.DeviceNotFound(device=device) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] self.force_reraise() 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] six.reraise(self.type_, self.value, self.tb) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 393, in _try_detach_device 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] self.detach_device(conf, persistent=persistent, live=live) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 450, in detach_device 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] self._domain.detachDeviceFlags(device_xml, flags=flags) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] result = proxy_call(self._autowrap, f, *args, **kwargs) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] rv = execute(f, *args, **kwargs) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] six.reraise(c, e, tb) 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance: 5f67c47b-f7d7-4063-88f1-8f7b6177fcb9] File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker 2017-03-16 01:01:32.897 7447 ERROR nova.compute.manager [instance:
[Yahoo-eng-team] [Bug 1673411] Re: config-drive support is broken
** Also affects: cloud-init Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1673411 Title: config-drive support is broken Status in cloud-init: New Status in nova-lxd: Triaged Bug description: After reviewing https://review.openstack.org/#/c/445579/ and doing some testing, it would appear that the config-drive support in the nova-lxd driver is not functional. cloud-init ignores the data presented in /var/lib/cloud/data and reads from the network accessible metadata-service. To test this effectively you have to have a fully offline instance (i.e. no metadata service access). To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1673411/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673467] [NEW] [ocata] unsupported configuration: CPU mode 'host-model' for aarch64 kvm domain on aarch64 host is not supported by hypervisor
Public bug reported: We hit this error in Ocata while trying to launch an arm64 instance: 2017-03-16 08:01:42.329 144245 ERROR nova.virt.libvirt.guest [req-2ad2d5d9-696d-4baa-a071-756e460ca3de 8f431f83f7e44ef1a084e7e27b40a685 a904dd389c5d4817a4d95b8f3268cf4d - - -] Error launching a defined domain with XML: instance-0001 220bec1b-8907-4da9-9862-9cc2354abf39 http://openstack.org/xmlns/libvirt/nova/1.0;> guestOS-test-arm64-kvm-xenial-ci_oil_slave14_0 2017-03-16 08:01:38 2048 20 0 0 1 admin admin 2097152 2097152 1 1024 hvm /usr/share/AAVMF/AAVMF_CODE.fd /var/lib/libvirt/qemu/nvram/instance-0001_VARS.fd destroy restart destroy /usr/bin/kvm 2017-03-16 08:01:42.333 144245 ERROR nova.virt.libvirt.driver [req-2ad2d5d9-696d-4baa-a071-756e460ca3de 8f431f83f7e44ef1a084e7e27b40a685 a904dd389c5d4817a4d95b8f3268cf4d - - -] [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] Failed to start libvirt guest 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1930, in _build_and_run_instance 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] block_device_info=block_device_info) 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2688, in spawn 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] destroy_disks_on_failure=True) 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5099, in _create_domain_and_network 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] destroy_disks_on_failure) 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] self.force_reraise() 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] six.reraise(self.type_, self.value, self.tb) 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5071, in _create_domain_and_network 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] post_xml_callback=post_xml_callback) 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4989, in _create_domain 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] guest.launch(pause=pause) 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] self._encoded_xml, errors='ignore') 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] self.force_reraise() 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] six.reraise(self.type_, self.value, self.tb) 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 140, in launch 2017-03-16 08:01:43.522 144245 ERROR nova.compute.manager [instance: 220bec1b-8907-4da9-9862-9cc2354abf39] return
[Yahoo-eng-team] [Bug 1667468] Re: fullstack: test_securitygroup fails with linuxbridge-iptables configuration
*** This bug is a duplicate of bug 1657260 *** https://bugs.launchpad.net/bugs/1657260 ** This bug is no longer a duplicate of bug 1667469 fullstack: test_securitygroup fails with linuxbridge-iptables configuration ** This bug has been marked a duplicate of bug 1657260 Established connection don't stops when rule is removed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1667468 Title: fullstack: test_securitygroup fails with linuxbridge-iptables configuration Status in neutron: New Bug description: Example of failure: http://logs.openstack.org/14/437514/1/check/gate- neutron-dsvm-fullstack-ubuntu-xenial/2a7467e/testr_results.html.gz e-r-q; http://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name %3Agate-neutron-dsvm-fullstack-ubuntu- xenial%20AND%20message%3A%5C%22AssertionError%3A%20Still%20can%20connect%20to%20the%20VM%20from%20different%20host.%5C%22%20AND%20tags%3Aconsole 2017-02-23 19:12:37.483868 | 2017-02-23 19:12:37.483 | Captured traceback: 2017-02-23 19:12:37.485131 | 2017-02-23 19:12:37.484 | ~~~ 2017-02-23 19:12:37.486372 | 2017-02-23 19:12:37.486 | Traceback (most recent call last): 2017-02-23 19:12:37.487624 | 2017-02-23 19:12:37.487 | File "neutron/tests/base.py", line 116, in func 2017-02-23 19:12:37.488854 | 2017-02-23 19:12:37.488 | return f(self, *args, **kwargs) 2017-02-23 19:12:37.490167 | 2017-02-23 19:12:37.489 | File "neutron/tests/fullstack/test_securitygroup.py", line 182, in test_securitygroup 2017-02-23 19:12:37.491433 | 2017-02-23 19:12:37.491 | "Still can connect to the VM from different host.") 2017-02-23 19:12:37.492632 | 2017-02-23 19:12:37.492 | File "neutron/common/utils.py", line 754, in wait_until_true 2017-02-23 19:12:37.493913 | 2017-02-23 19:12:37.493 | raise exception 2017-02-23 19:12:37.495085 | 2017-02-23 19:12:37.494 | AssertionError: Still can connect to the VM from different host. 2017-02-23 19:12:37.496297 | 2017-02-23 19:12:37.495 | To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1667468/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673261] Re: Help docs for disk_weight_multiplier option are wrong
Reviewed: https://review.openstack.org/446177 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3397c7582e3f6e1a7030e29e0d7df5ef26ed0c4f Submitter: Jenkins Branch:master commit 3397c7582e3f6e1a7030e29e0d7df5ef26ed0c4f Author: Matt RiedemannDate: Wed Mar 15 17:14:25 2017 -0400 Fix the help for the disk_weight_multiplier option There was what looks to be a copy/paste error in the help text for the "disk_weight_multiplier" config option. It should say the disk weigher needs to be enabled, not the ram weigher. Change-Id: Ic6e3601e18adce2a01f36e956fbce3e366b08b80 Closes-Bug: #1673261 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673261 Title: Help docs for disk_weight_multiplier option are wrong Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) ocata series: Confirmed Bug description: The help for the "disk_weight_multiplier" config option says: "Also note that this setting only affects scheduling if the 'ram' weigher is enabled." That's a copy/paste error from the ram_weight_multiplier, it should say 'disk' weigher. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1673261/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673430] [NEW] Launch Instance error ImageNotFound
Public bug reported: 2017-03-16 19:09:57.370 29479 INFO nova.compute.claims [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Claim successful 2017-03-16 19:09:57.988 29479 INFO nova.virt.osinfo [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] Cannot load Libosinfo: (No module named Libosinfo) 2017-03-16 19:09:58.505 29479 INFO nova.virt.libvirt.driver [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Creating image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [req-bff9ab69-2110-41e5-bf34-4a13574f4076 0c8c68d359a249388def4b4ec5d6c507 9b8d0fa36b2d425a9dd6edf3dc5d2344 - - -] [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Instance failed to spawn 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] Traceback (most recent call last): 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2218, in _build_resources 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] yield resources 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2064, in _build_and_run_instance 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] block_device_info=block_device_info) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2772, in spawn 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] admin_pass=admin_password) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3190, in _create_image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] instance, size, fallback_from_host) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 6789, in _try_fetch_image_cache 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] size=size) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 251, in cache 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] *args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 593, in create_image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] prepare_template(target=base, max_size=size, *args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 274, in inner 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] return f(*args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagebackend.py", line 241, in fetch_func_sync 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] fetch_func(target=target, *args, **kwargs) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 429, in fetch_image 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] max_size=max_size) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 120, in fetch_to_raw 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager [instance: 1d8734a3-cd0e-4ee8-ad2c-20b46c98e056] max_size=max_size) 2017-03-16 19:09:58.609 29479 ERROR nova.compute.manager
[Yahoo-eng-team] [Bug 1635182] Re: [placement] in api code repeated need to restate json_error_formatter is error prone
Reviewed: https://review.openstack.org/433301 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=db5b1603282b4bb7665670c539d224c42dee8a9d Submitter: Jenkins Branch:master commit db5b1603282b4bb7665670c539d224c42dee8a9d Author: Pushkar UmaranikarDate: Mon Feb 13 22:22:24 2017 + Placement api: set custom json_error_formatter in root Change decorator for resource based actions in root handler to wsgi_wrapper.PlacementWsgify This is a newly introduced wrapper class around webob.dec.wsgify to set json formatter in case of webob exceptions. Change-Id: I847345f7f15a46930faf8793baf801beba98d368 Closes-Bug: #1635182 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1635182 Title: [placement] in api code repeated need to restate json_error_formatter is error prone Status in OpenStack Compute (nova): Fix Released Bug description: Any time placement API (v1.0) code wants to raise a webob.exc of some kind it needs to add a json_formatter attribute to the call. This is easy to forget. We can probably figure out a way to make this a default, perhaps by making subclasses of the existing exceptions or setting some defaults somewhere. Not a huge deal but worth looking into. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1635182/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673429] [NEW] nova removes ports not owned by "compute" in deallocate_for_instance
Public bug reported: Hit this on ocata when aborting a deployment through nova (nova boot, then nova delete while instance is still in BUILD), using ironic virt driver. Relevant bits of n-cpu log: 2017-03-16 10:06:16.780 ERROR nova.compute.manager [req-37b74c46-5553-484b-b397-6efbede9d962 admin admin] [instance: 888b7c5a-6f7d-400d-a61f -f367441e7a91] Build of instance 888b7c5a-6f7d-400d-a61f-f367441e7a91 aborted: Instance 888b7c5a-6f7d-400d-a61f-f367441e7a91 provisioning wa s aborted 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] Traceback (most recent call last): 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] File "/opt/stack/nova/nova/compute/man ager.py", line 1780, in _do_build_and_run_instance 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] filter_properties) 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] File "/opt/stack/nova/nova/compute/man ager.py", line 1961, in _build_and_run_instance 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] phase=fields.NotificationPhase.ERROR , exception=e) 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] File "/usr/local/lib/python2.7/dist-pa ckages/oslo_utils/excutils.py", line 220, in __exit__ 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] self.force_reraise() 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] File "/usr/local/lib/python2.7/dist-pa ckages/oslo_utils/excutils.py", line 196, in force_reraise 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] six.reraise(self.type_, self.value, self.tb) 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] File "/opt/stack/nova/nova/compute/man ager.py", line 1933, in _build_and_run_instance 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] instance=instance) 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] File "/usr/lib/python2.7/contextlib.py ", line 35, in __exit__ 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] self.gen.throw(type, value, tracebac k) 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] File "/opt/stack/nova/nova/compute/manager.py", line 2152, in _build_resources 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] reason=six.text_type(exc)) 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] BuildAbortException: Build of instance 888b7c5a-6f7d-400d-a61f-f367441e7a91 aborted: Instance 888b7c5a-6f7d-400d-a61f-f367441e7a91 provisioning was aborted 2017-03-16 10:06:16.780 TRACE nova.compute.manager [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] 2017-03-16 10:06:16.781 DEBUG nova.compute.manager [req-37b74c46-5553-484b-b397-6efbede9d962 admin admin] [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] Deallocating network for instance from (pid=25464) _deallocate_network /opt/stack/nova/nova/compute/manager.py:1661 2017-03-16 10:06:16.781 DEBUG nova.network.neutronv2.api [req-37b74c46-5553-484b-b397-6efbede9d962 admin admin] [instance: 888b7c5a-6f7d-400d-a61f-f367441e7a91] deallocate_for_instance() from (pid=25464) deallocate_for_instance /opt/stack/nova/nova/network/neutronv2/api.py:1156 2017-03-16 10:06:16.901 DEBUG neutronclient.v2_0.client [req-37b74c46-5553-484b-b397-6efbede9d962 admin admin] GET call to neutron for http://192.168.122.22:9696/v2.0/ports.json?device_id=888b7c5a-6f7d-400d-a61f-f367441e7a91 used request id req-be903c33-c4da-433a-8e52-9bd5d14ed018 from (pid=25464) _append_request_id /usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128 2017-03-16 10:06:17.587 DEBUG neutronclient.v2_0.client [req-37b74c46-5553-484b-b397-6efbede9d962 admin admin] DELETE call to neutron for http://192.168.122.22:9696/v2.0/ports/1dee64d3-4e81-4ce5-b428-ab90700051dd.json used request id req-a5d90558-2b04-46fa-833e-d65446146a16 from (pid=25464) _append_request_id /usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py:128 2017-03-16 10:06:19.055 DEBUG oslo_service.periodic_task [req-f990991a-f1ee-4e3e-a1e6-82a33df8f6d3 None None] Running periodic task ComputeManager._heal_instance_info_cache from (pid=25464) run_periodic_tasks /usr/local/lib/python2.7/dist-packages/oslo_service/periodic_task.py:215 2017-03-16 10:06:19.055 DEBUG nova.compute.manager [req-f990991a-f1ee-4e3e-a1e6-82a33df8f6d3 None None]
[Yahoo-eng-team] [Bug 1673427] [NEW] mysql OOM ( Out of sort memory, consider increasing server sort buffer size )
Public bug reported: Neutron fails with: http://logs.openstack.org/84/445884/2/gate/gate-neutron-dsvm-api-ubuntu- xenial/800e806/logs/screen-q-svc.txt.gz 2017-03-15 21:02:19.418 30853 ERROR neutron.api.v2.resource DBError: (pymysql.err.InternalError) (1038, u'Out of sort memory, consider increasing server sort buffer size') [SQL: u'SELECT networks.id AS network_id, subnets.id AS subnet_id, subnets.cidr AS subnets_cidr, subnets.ip_version AS subnets_ip_version, networks.name AS network_name, networks.project_id AS networks_project_id, subnets.name AS subnet_name, count(ipallocations.subnet_id) AS used_ips \nFROM networks LEFT OUTER JOIN subnets ON networks.id = subnets.network_id LEFT OUTER JOIN ipallocations ON subnets.id = ipallocations.subnet_id GROUP BY networks.id, subnets.id, subnets.cidr, subnets.ip_version, networks.name, networks.project_id, subnets.name'] 2017-03-15 21:02:19.418 30853 ERROR neutron.api.v2.resource So may we consider changing the sort buffer size? ** Affects: devstack Importance: Undecided Status: New ** Affects: neutron Importance: Medium Status: New ** Also affects: devstack Importance: Undecided Status: New ** Changed in: neutron Importance: Undecided => Medium -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1673427 Title: mysql OOM ( Out of sort memory, consider increasing server sort buffer size ) Status in devstack: New Status in neutron: New Bug description: Neutron fails with: http://logs.openstack.org/84/445884/2/gate/gate-neutron-dsvm-api- ubuntu-xenial/800e806/logs/screen-q-svc.txt.gz 2017-03-15 21:02:19.418 30853 ERROR neutron.api.v2.resource DBError: (pymysql.err.InternalError) (1038, u'Out of sort memory, consider increasing server sort buffer size') [SQL: u'SELECT networks.id AS network_id, subnets.id AS subnet_id, subnets.cidr AS subnets_cidr, subnets.ip_version AS subnets_ip_version, networks.name AS network_name, networks.project_id AS networks_project_id, subnets.name AS subnet_name, count(ipallocations.subnet_id) AS used_ips \nFROM networks LEFT OUTER JOIN subnets ON networks.id = subnets.network_id LEFT OUTER JOIN ipallocations ON subnets.id = ipallocations.subnet_id GROUP BY networks.id, subnets.id, subnets.cidr, subnets.ip_version, networks.name, networks.project_id, subnets.name'] 2017-03-15 21:02:19.418 30853 ERROR neutron.api.v2.resource So may we consider changing the sort buffer size? To manage notifications about this bug go to: https://bugs.launchpad.net/devstack/+bug/1673427/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1606495] Re: copy_from in api v1 allows network port scan
** Changed in: ossn Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1606495 Title: copy_from in api v1 allows network port scan Status in Glance: New Status in OpenStack Security Advisory: Opinion Status in OpenStack Security Notes: Fix Released Bug description: This issue is being treated as a potential security risk under embargo. Please do not make any public mention of embargoed (private) security vulnerabilities before their coordinated publication by the OpenStack Vulnerability Management Team in the form of an official OpenStack Security Advisory. This includes discussion of the bug or associated fixes in public forums such as mailing lists, code review systems and bug trackers. Please also avoid private disclosure to other individuals not already approved for access to this information, and provide this same reminder to those who are made aware of the issue prior to publication. All discussion should remain confined to this private bug report, and any proposed fixes should be added to the bug as attachments. copy_from allows to create Images with an url like http://localhost:22 The remote content gets copied unverified in the defined glance store. E.g. after downloading the image with copy_from url http://localhost:22, you see the OpenSSH banner. This is a security issue, as it allows users to do network "scans" for open ports and it copies remote (potentially malicious) content unverified to your configured glance store. glance api v1 is still the default in horizon. To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1606495/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1661797] Re: identify lxd-nova platform to enable Openstack datasource
Marking nova-lxd task as Fix Released; changes landed and released, need to work through SRU process. ** Changed in: nova-lxd Importance: Undecided => High ** Changed in: nova-lxd Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1661797 Title: identify lxd-nova platform to enable Openstack datasource Status in cloud-init: Fix Committed Status in nova-lxd: Fix Released Status in cloud-init package in Ubuntu: Fix Released Status in nova-lxd package in Ubuntu: Fix Released Status in cloud-init source package in Xenial: Confirmed Status in nova-lxd source package in Xenial: In Progress Status in cloud-init source package in Yakkety: Confirmed Status in nova-lxd source package in Yakkety: In Progress Status in cloud-init source package in Zesty: Fix Released Status in nova-lxd source package in Zesty: Fix Released Bug description: nova-lxd uses the Openstack Network metadata service. In an effort to avoid polling metadata services in cloud-init we will disable attempts to reach the MD without positive identification of the cloud. We need to be able to positively identify that the container we are running inside should have access to an openstack metadata service so we can safely assume it will be there. How can we positively identify that a container is running in nova-lxd? Is there anything in the environment (possibly pid 1 environ?) that we can look at? One way I could see doing t his would be for lxd-nova to put CLOUD_PLATFORM='openstack-nova' inside the pid 1 environment. then cloud-init can look at /proc/1/environ and pick that out. Open to other ideas, and would love it if there was something we could do. Related bugs bug 1660385: Alert user of Ec2 Datasource on lookalike cloud bug 1661797: identify lxd-nova platform to enable Openstack datasource bug 1661693: identify brightbox platform to enable Ec2 datasource bug 1663304: identify openstack kvm platform on arm64 bug 1668313: [SRU] mitaka point release To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1661797/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673371] [NEW] correct typo in signature doc
Public bug reported: word “url” should be capitalized in doc https://docs.openstack.org/developer/glance/signature.html#configuration ** Affects: glance Importance: Undecided Assignee: Dou Rui Yuan (xsfhxh) Status: New ** Changed in: glance Assignee: (unassigned) => Dou Rui Yuan (xsfhxh) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1673371 Title: correct typo in signature doc Status in Glance: New Bug description: word “url” should be capitalized in doc https://docs.openstack.org/developer/glance/signature.html#configuration To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1673371/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1673375] [NEW] Duplicate entry for key 'uniq_aggregate_hosts0host0aggregate_id
Public bug reported: I guess something is wrong with nova, causing a tempest failure. http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35 -ubuntu-xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging context) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging cursor.execute(statement, parameters) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 166, in execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging result = self._query(query) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 322, in _query 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging conn.query(q) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 852, in query 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging self._affected_rows = self._read_query_result(unbuffered=unbuffered) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1053, in _read_query_result 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging result.read() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1336, in read 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging first_packet = self.connection._read_packet() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1010, in _read_packet 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging packet.check_error() 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 393, in check_error 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging err.raise_mysql_exception(self._data) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging raise errorclass(errno, errval) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging pymysql.err.IntegrityError: (1062, "Duplicate entry 'ubuntu-xenial-rax-iad-7906449-6' for key 'uniq_aggregate_hosts0host0aggregate_id'") ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1673375 Title: Duplicate entry for key 'uniq_aggregate_hosts0host0aggregate_id Status in OpenStack Compute (nova): New Bug description: I guess something is wrong with nova, causing a tempest failure. http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35 -ubuntu- xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last): 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging context) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging cursor.execute(statement, parameters) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 166, in execute 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging result = self._query(query) 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 322, in _query 2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging
[Yahoo-eng-team] [Bug 1673086] Re: test_filter_router_tags fails with testtools.matchers._impl.MismatchError: set(['tag-res2', 'tag-res3']) != set([u'tag-res3'])
Reviewed: https://review.openstack.org/446005 Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=0cb8fd637472c9aeb3ad162bf74d96f5a678eacd Submitter: Jenkins Branch:master commit 0cb8fd637472c9aeb3ad162bf74d96f5a678eacd Author: Ihar HrachyshkaDate: Wed Mar 15 15:08:59 2017 + On update_tags, clean up tags from the requested resource only There was a bug in I0179a3616689f39d93c337c730283e21a01677cf that made the service plugin to clean up removed tags from all resources, not just the one requested. Change-Id: Ifbe13102ebd903e01eb28a00d97e7e94d1a0e0b1 Closes-Bug: #1673086 ** Changed in: neutron Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1673086 Title: test_filter_router_tags fails with testtools.matchers._impl.MismatchError: set(['tag-res2', 'tag-res3']) != set([u'tag-res3']) Status in neutron: Fix Released Bug description: http://logs.openstack.org/53/319353/4/check/gate-neutron-dsvm-api- ubuntu-xenial/bafaa60/testr_results.html.gz ft1.1: neutron.tests.tempest.api.test_tag.TagFilterRouterTestJSON.test_filter_router_tags[id-cdd3f3ea-073d-4435-a6cb-826a4064193d,smoke]_StringException: Empty attachments: stderr stdout pythonlogging:'': {{{ 2017-03-15 05:06:28,543 8715 INFO [tempest.lib.common.rest_client] Request (TagFilterRouterTestJSON:test_filter_router_tags): 200 GET http://15.184.66.234:9696/v2.0/routers?tags=red 0.300s 2017-03-15 05:06:28,543 8715 DEBUG[tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'X-Auth-Token': '', 'Accept': 'application/json'} Body: None Response - Headers: {'content-location': 'http://15.184.66.234:9696/v2.0/routers?tags=red', 'status': '200', u'x-openstack-request-id': 'req-dd2c112c-7e0b-4c1f-a3e9-67a993cb3765', u'content-type': 'application/json', u'content-length': '1431', u'date': 'Wed, 15 Mar 2017 05:06:28 GMT', u'connection': 'close'} Body: {"routers": [{"status": "ACTIVE", "external_gateway_info": null, "availability_zone_hints": [], "availability_zones": [], "description": "", "tags": ["blue", "red", "green"], "tenant_id": "a74304310e6a4a669dae814a4f644503", "created_at": "2017-03-15T05:06:25Z", "admin_state_up": false, "updated_at": "2017-03-15T05:06:27Z", "flavor_id": null, "revision_number": 6, "routes": [], "project_id": "a74304310e6a4a669dae814a4f644503", "id": "388e0638-9e6d-4658-b9b3-ac664fbbc96b", "name": "tag-res3"}, {"status": "ACTIVE", "external_gateway_info": null, "availability_zone_hints": [], "availability_zones": [], "description": "", "tags": ["red"], "tenant_id": "a74304310e6a4a669dae814a4f644503", "created_at": "2017-03-15T05:06:25Z", "admin_state_up": false, "updated_at": "2017-03-15T05:06:27Z", "flavor_id": null, "revision_number": 6, "routes": [], "project_id": "a74304310e6a4a669dae814a4f644503", "id": "74dca3b4-9cf1-493e-8acb-1845da14f2f5", "name": "tag-res2"}, {"status": "ACTIVE ", "external_gateway_info": null, "availability_zone_hints": [], "availability_zones": [], "description": "", "tags": ["red"], "tenant_id": "a74304310e6a4a669dae814a4f644503", "created_at": "2017-03-15T05:06:24Z", "admin_state_up": false, "updated_at": "2017-03-15T05:06:27Z", "flavor_id": null, "revision_number": 4, "routes": [], "project_id": "a74304310e6a4a669dae814a4f644503", "id": "baf41f14-fac6-4c3e-98db-10dd6de963ca", "name": "tag-res1"}]} 2017-03-15 05:06:28,728 8715 INFO [tempest.lib.common.rest_client] Request (TagFilterRouterTestJSON:test_filter_router_tags): 200 GET http://15.184.66.234:9696/v2.0/routers?tags=red%2Cblue 0.184s 2017-03-15 05:06:28,729 8715 DEBUG[tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'X-Auth-Token': '', 'Accept': 'application/json'} Body: None Response - Headers: {'content-location': 'http://15.184.66.234:9696/v2.0/routers?tags=red%2Cblue', 'status': '200', u'x-openstack-request-id': 'req-c17048b2-6a89-438a-8ba9-98cabf92dc6d', u'content-type': 'application/json', u'content-length': '497', u'date': 'Wed, 15 Mar 2017 05:06:28 GMT', u'connection': 'close'} Body: {"routers": [{"status": "ACTIVE", "external_gateway_info": null, "availability_zone_hints": [], "availability_zones": [], "description": "", "tags": ["green", "blue", "red"], "tenant_id": "a74304310e6a4a669dae814a4f644503", "created_at": "2017-03-15T05:06:25Z", "admin_state_up": false, "updated_at": "2017-03-15T05:06:27Z", "flavor_id": null, "revision_number": 6, "routes": [], "project_id": "a74304310e6a4a669dae814a4f644503", "id": "388e0638-9e6d-4658-b9b3-ac664fbbc96b", "name": "tag-res3"}]} }}} Traceback (most recent call last): File "tempest/test.py", line 121, in wrapper return func(*func_args,
[Yahoo-eng-team] [Bug 1668513] Re: KeyError on loadbalancer schedule
** Changed in: neutron Status: New => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1668513 Title: KeyError on loadbalancer schedule Status in neutron: Fix Released Bug description: When multi neutron-server rescheduled the same loadbalancer from a dead agent almost simultaneously, the function "get_agent_hosting_loadbalancer" might return a dict, which would cause KeyError since the dict format is such like {'agent': {'id': xxx, ...}}. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1668513/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp