[Yahoo-eng-team] [Bug 1724613] Re: AllocationCandidates.get_by_filters ignores shared RPs when the RC exists in both places

2018-03-31 Thread OpenStack Infra 
Reviewed:  https://review.openstack.org/533396
Committed: 
https://git.openstack.org/cgit/openstack/nova/commit/?id=02e357e7c2d2ccbab0a1f6b5a807d11f1ef72d46
Submitter: Zuul
Branch:master

commit 02e357e7c2d2ccbab0a1f6b5a807d11f1ef72d46
Author: Tetsuro Nakamura 
Date:   Sun Jan 14 17:09:05 2018 +0900

Fix allocation_candidates not to ignore shared RPs

When both the compute node resource provider and the shared
resource provider have inventory in the same resource class,
AllocationCandidates.get_by_filters didn't return an
AllocationRequest including the shared resource provider.

To fix the bug, this patch changes the function of
_alloc_candidates_with_shared() to consider resources from
non-sharing providers and resources from sharing providers
at the same time.

Change-Id: Iaf23f35f2f9a5d27a814ef5b94abed1a8b365bc3
Closes-Bug: #1724613
Related-Bug: #1731072


** Changed in: nova
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1724613

Title:
  AllocationCandidates.get_by_filters ignores shared RPs when the RC
  exists in both places

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  When both the compute node resource provider and the shared resource
  provider have inventory in the same resource class,
  AllocationCandidates.get_by_filters will not return an
  AllocationRequest including the shared resource provider.

  Example:

   cnrp { VCPU: 24,
  MEMORY_MB: 2048,
  DISK_GB: 16 }
   ssrp { DISK_GB: 32 }

   AllocationCandidates.get_by_filters(
   resources={ VCPU: 1,
   MEMORY_MB: 512,
   DISK_GB: 2 } )

  Expected:

   allocation_requests: [
   { cnrp: { VCPU: 1,
 MEMORY_MB: 512,
 DISK_GB: 2 } },
   { cnrp: { VCPU: 1,
 MEMORY_MB: 512 }
 ssrp: { DISK_GB: 2 } },
   ]

  Actual:

   allocation_requests: [
   { cnrp: { VCPU: 1,
 MEMORY_MB: 512,
 DISK_GB: 2 } }
   ]

  I will post a review shortly that demonstrates this.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1724613/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1760322] [NEW] Traits not synced if first retrieval fails

2018-03-31 Thread Eric Fried 
Public bug reported:

If the first trait you try to retrieve from placement doesn't exist,
traits are not synced from os_traits into the database, so it winds up
empty.

try:
rp_obj.Trait.get_by_name(self.ctx, 'CUSTOM_GOLD')
except exception.TraitNotFound:
pass
rp_obj.Trait.get_by_name(self.ctx, os_traits.HW_CPU_X86_AVX2)  # <== 
raises TraitNotFound

I *think* what's happening is this:

1@staticmethod
2@db_api.api_context_manager.writer  # trait sync can cause a write
3def _get_by_name_from_db(context, name):
4_ensure_trait_sync(context)
5result = context.session.query(models.Trait).filter_by(
6name=name).first()
7if not result:
8raise exception.TraitNotFound(names=name)
9return result

Line 4 "succeeds" and sets _TRAITS_SYNCED = True.
But because line 8 raises, the transaction is rolled back.  Database stays 
empty.
Subsequent retrieval attempts see _TRAITS_SYNCED == True so don't try to resync.

** Affects: nova
 Importance: Undecided
 Status: New


** Tags: placement

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1760322

Title:
  Traits not synced if first retrieval fails

Status in OpenStack Compute (nova):
  New

Bug description:
  If the first trait you try to retrieve from placement doesn't exist,
  traits are not synced from os_traits into the database, so it winds up
  empty.

  try:
  rp_obj.Trait.get_by_name(self.ctx, 'CUSTOM_GOLD')
  except exception.TraitNotFound:
  pass
  rp_obj.Trait.get_by_name(self.ctx, os_traits.HW_CPU_X86_AVX2)  # <== 
raises TraitNotFound

  I *think* what's happening is this:

  1@staticmethod
  2@db_api.api_context_manager.writer  # trait sync can cause a write
  3def _get_by_name_from_db(context, name):
  4_ensure_trait_sync(context)
  5result = context.session.query(models.Trait).filter_by(
  6name=name).first()
  7if not result:
  8raise exception.TraitNotFound(names=name)
  9return result

  Line 4 "succeeds" and sets _TRAITS_SYNCED = True.
  But because line 8 raises, the transaction is rolled back.  Database stays 
empty.
  Subsequent retrieval attempts see _TRAITS_SYNCED == True so don't try to 
resync.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1760322/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1760320] [NEW] vpnaas unit test is broken after https://review.openstack.org/#/c/524406/ is merged

2018-03-31 Thread Akihiro Motoki 
Public bug reported:

After https://review.openstack.org/#/c/524406/ is merged, neutron-vpnaas unit 
test is broken.
http://logs.openstack.org/94/543394/12/check/openstack-tox-py27/61b9f31/

example failure:

ft5.41: 
neutron_vpnaas.tests.unit.services.vpn.device_drivers.test_ipsec.TestOpenSwanProcess.test_sync_StringException:
 Traceback (most recent call last):
  File "neutron_vpnaas/tests/unit/services/vpn/device_drivers/test_ipsec.py", 
line 1117, in setUp
super(TestOpenSwanProcess, self).setUp(driver, ipsec_process)
  File "neutron_vpnaas/tests/unit/services/vpn/device_drivers/test_ipsec.py", 
line 419, in setUp
self._make_router_info_for_test()
  File "neutron_vpnaas/tests/unit/services/vpn/device_drivers/test_ipsec.py", 
line 424, in _make_router_info_for_test
**self.ri_kwargs)
  File 
"/home/zuul/src/git.openstack.org/openstack/neutron-vpnaas/.tox/py27/local/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
 line 77, in __init__
self.initialize_metadata_iptables()
  File 
"/home/zuul/src/git.openstack.org/openstack/neutron-vpnaas/.tox/py27/local/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
 line 1006, in initialize_metadata_iptables
'value': self.agent_conf.metadata_access_mark,
  File 
"/home/zuul/src/git.openstack.org/openstack/neutron-vpnaas/.tox/py27/local/lib/python2.7/site-packages/oslo_config/cfg.py",
 line 2519, in __getattr__
raise NoSuchOptError(name)
oslo_config.cfg.NoSuchOptError: no such option metadata_access_mark in group 
[DEFAULT]

** Affects: neutron
 Importance: Critical
 Status: New


** Tags: vpnaas

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1760320

Title:
  vpnaas unit test is broken after
  https://review.openstack.org/#/c/524406/ is merged

Status in neutron:
  New

Bug description:
  After https://review.openstack.org/#/c/524406/ is merged, neutron-vpnaas unit 
test is broken.
  http://logs.openstack.org/94/543394/12/check/openstack-tox-py27/61b9f31/

  example failure:

  ft5.41: 
neutron_vpnaas.tests.unit.services.vpn.device_drivers.test_ipsec.TestOpenSwanProcess.test_sync_StringException:
 Traceback (most recent call last):
File "neutron_vpnaas/tests/unit/services/vpn/device_drivers/test_ipsec.py", 
line 1117, in setUp
  super(TestOpenSwanProcess, self).setUp(driver, ipsec_process)
File "neutron_vpnaas/tests/unit/services/vpn/device_drivers/test_ipsec.py", 
line 419, in setUp
  self._make_router_info_for_test()
File "neutron_vpnaas/tests/unit/services/vpn/device_drivers/test_ipsec.py", 
line 424, in _make_router_info_for_test
  **self.ri_kwargs)
File 
"/home/zuul/src/git.openstack.org/openstack/neutron-vpnaas/.tox/py27/local/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
 line 77, in __init__
  self.initialize_metadata_iptables()
File 
"/home/zuul/src/git.openstack.org/openstack/neutron-vpnaas/.tox/py27/local/lib/python2.7/site-packages/neutron/agent/l3/router_info.py",
 line 1006, in initialize_metadata_iptables
  'value': self.agent_conf.metadata_access_mark,
File 
"/home/zuul/src/git.openstack.org/openstack/neutron-vpnaas/.tox/py27/local/lib/python2.7/site-packages/oslo_config/cfg.py",
 line 2519, in __getattr__
  raise NoSuchOptError(name)
  oslo_config.cfg.NoSuchOptError: no such option metadata_access_mark in group 
[DEFAULT]

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1760320/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1760303] Re: ComputeManager.cleanup_host fails when there are waiting events: ValueError: Field value network-vif-plugged-ce531f90-199f-48c0-816c is invalid

2018-03-31 Thread Matt Riedemann 
** Also affects: nova/ocata
   Importance: Undecided
   Status: New

** Also affects: nova/pike
   Importance: Undecided
   Status: New

** Also affects: nova/queens
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1760303

Title:
  ComputeManager.cleanup_host fails when there are waiting events:
  ValueError: Field value network-vif-plugged-ce531f90-199f-48c0-816c is
  invalid

Status in OpenStack Compute (nova):
  Triaged
Status in OpenStack Compute (nova) ocata series:
  New
Status in OpenStack Compute (nova) pike series:
  New
Status in OpenStack Compute (nova) queens series:
  New

Bug description:
  The parsing in cancel_all_events() fails to account for names such as:

  network-vif-plugged-ce531f90-199f-48c0-816c

  
https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/compute/manager.py#L406

  Which can be constructed when registering the event using a tuple of
  the event name and tag:

  
https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/compute/manager.py#L460-L463

  
https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/objects/external_event.py#L51

  So we almost need something like that make_key function but more like
  parse_key where using the known EVENT_NAMES we can split the event
  name to get the name and tag.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1760303/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1760303] [NEW] ComputeManager.cleanup_host fails when there are waiting events: ValueError: Field value network-vif-plugged-ce531f90-199f-48c0-816c is invalid

2018-03-31 Thread Matt Riedemann 
Public bug reported:

The parsing in cancel_all_events() fails to account for names such as:

network-vif-plugged-ce531f90-199f-48c0-816c

https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/compute/manager.py#L406

Which can be constructed when registering the event using a tuple of the
event name and tag:

https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/compute/manager.py#L460-L463

https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/objects/external_event.py#L51

So we almost need something like that make_key function but more like
parse_key where using the known EVENT_NAMES we can split the event name
to get the name and tag.

** Affects: nova
 Importance: Medium
 Status: Triaged


** Tags: compute

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1760303

Title:
  ComputeManager.cleanup_host fails when there are waiting events:
  ValueError: Field value network-vif-plugged-ce531f90-199f-48c0-816c is
  invalid

Status in OpenStack Compute (nova):
  Triaged

Bug description:
  The parsing in cancel_all_events() fails to account for names such as:

  network-vif-plugged-ce531f90-199f-48c0-816c

  
https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/compute/manager.py#L406

  Which can be constructed when registering the event using a tuple of
  the event name and tag:

  
https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/compute/manager.py#L460-L463

  
https://github.com/openstack/nova/blob/167023b5074c7cb241b33767d76a116ec95d652c/nova/objects/external_event.py#L51

  So we almost need something like that make_key function but more like
  parse_key where using the known EVENT_NAMES we can split the event
  name to get the name and tag.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1760303/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1760276] [NEW] "provider_summaries" doesn't include resources that are not requested

2018-03-31 Thread Tetsuro Nakamura 
Public bug reported:

Description


In ``GET /allocation_candidates`` API, ``provider_summaries`` should show all 
the inventories for all the resource classes in all the resource providers.
However, ``provider_summaries`` doesn't contain resources that aren't requested.

Steps to reproduce
==
Here's one example:

CN1 has inventory in VCPU, MEMORY_MB, and DISK_GB.
I make a request for only VCPU resource.

Expected result
===

In API response, 
* "allocation_requests" shows "allocation" of VCPU resource of CN1.
* "provider_summaries" shows "resources" of VCPU, MEMORY_MB, and DISK_GB  of 
CN1.

Actual result
=

In API response, 
* "allocation_requests" shows "allocation" of VCPU resource of CN1.
* "provider_summaries" shows "resources" of only VCPU of CN1.

** Affects: nova
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1760276

Title:
  "provider_summaries" doesn't include resources that are not requested

Status in OpenStack Compute (nova):
  New

Bug description:
  Description
  

  In ``GET /allocation_candidates`` API, ``provider_summaries`` should show all 
the inventories for all the resource classes in all the resource providers.
  However, ``provider_summaries`` doesn't contain resources that aren't 
requested.

  Steps to reproduce
  ==
  Here's one example:

  CN1 has inventory in VCPU, MEMORY_MB, and DISK_GB.
  I make a request for only VCPU resource.

  Expected result
  ===

  In API response, 
  * "allocation_requests" shows "allocation" of VCPU resource of CN1.
  * "provider_summaries" shows "resources" of VCPU, MEMORY_MB, and DISK_GB  of 
CN1.

  Actual result
  =

  In API response, 
  * "allocation_requests" shows "allocation" of VCPU resource of CN1.
  * "provider_summaries" shows "resources" of only VCPU of CN1.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1760276/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp