[Yahoo-eng-team] [Bug 1702635] Re: SR-IOV: sometimes a port may hang in BUILD state

[OpenStack Infra]

Reviewed:  https://review.openstack.org/480997
Committed: 
https://git.openstack.org/cgit/openstack/neutron/commit/?id=6402cd37c9fdc1b21b0320f5c04a509112e783a8
Submitter: Zuul
Branch:master

commit 6402cd37c9fdc1b21b0320f5c04a509112e783a8
Author: Oleg Bondarev 
Date:   Thu Jul 6 15:43:15 2017 +0400

SR-IOV agent should specify host when requesting devices info

Otherwise neutron server may set BUILD status for ports not bound
to agent's host.

Closes-Bug: #1702635
Change-Id: Ic0aa2b5d8fb5ad682293ce2b8e44606ef862a62d


** Changed in: neutron
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1702635

Title:
  SR-IOV: sometimes a port may hang in BUILD state

Status in Mirantis OpenStack:
  Fix Released
Status in neutron:
  Fix Released

Bug description:
  Scenario:

  1) vfio-pci driver is used for VFs
  2) 2 ports are created in neutron with binding type 'direct'
  3) VMs are spawned and deleted on 2 compute nodes using pre-created ports
  4) one neutron port may be bound to different compute nodes at different
     moments
  5) for some reason (probably a bug, but current bug is not about it)
     vfio-pci is not properly handling VF reset after VM deletion and for
     sriov agent it looks like some port's MAC is still mapped to some PCI
     slot though the port is not bound to the node
  6) sriov agent requests port info from server with
     get_devices_details_list() but doesn't specify 'host' in parameters
  7) in this case neutron server sets this port to BUILD, though it may be
     bound to another host:

  def _get_new_status(self, host, port_context):
  port = port_context.current
  if not host or host == port_context.host:
  new_status = (n_const.PORT_STATUS_BUILD if port['admin_state_up']
    else n_const.PORT_STATUS_DOWN)
  if port['status'] != new_status:
  return new_status

  8) after processing, the agent notifies server with update_device_list() and 
this time specifies 'host' parameter
  9) server detects port's and agent's host mismatch and doesn't update status 
of the port
  10) port stays in BUILD state

  A simple fix would be to specify host at step 6 - in this case neutron
  server won't set port's status to BUILD because of host mismatch.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mos/+bug/1702635/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1702635] Re: SR-IOV: sometimes a port may hang in BUILD state

[Ilya Bumarskov]

Can't reproduce the bug on a test environment due to lack of appropriate HW. In 
accordance with our policy, fix should be verified on customer side.
Fix is present in snapshots/9.0-2017-10-16-142324

** Changed in: mos
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1702635

Title:
  SR-IOV: sometimes a port may hang in BUILD state

Status in Mirantis OpenStack:
  Fix Released
Status in neutron:
  In Progress

Bug description:
  Scenario:

  1) vfio-pci driver is used for VFs
  2) 2 ports are created in neutron with binding type 'direct'
  3) VMs are spawned and deleted on 2 compute nodes using pre-created ports
  4) one neutron port may be bound to different compute nodes at different
     moments
  5) for some reason (probably a bug, but current bug is not about it)
     vfio-pci is not properly handling VF reset after VM deletion and for
     sriov agent it looks like some port's MAC is still mapped to some PCI
     slot though the port is not bound to the node
  6) sriov agent requests port info from server with
     get_devices_details_list() but doesn't specify 'host' in parameters
  7) in this case neutron server sets this port to BUILD, though it may be
     bound to another host:

  def _get_new_status(self, host, port_context):
  port = port_context.current
  if not host or host == port_context.host:
  new_status = (n_const.PORT_STATUS_BUILD if port['admin_state_up']
    else n_const.PORT_STATUS_DOWN)
  if port['status'] != new_status:
  return new_status

  8) after processing, the agent notifies server with update_device_list() and 
this time specifies 'host' parameter
  9) server detects port's and agent's host mismatch and doesn't update status 
of the port
  10) port stays in BUILD state

  A simple fix would be to specify host at step 6 - in this case neutron
  server won't set port's status to BUILD because of host mismatch.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mos/+bug/1702635/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

[Yahoo-eng-team] [Bug 1702635] Re: SR-IOV: sometimes a port may hang in BUILD state

[Denis Meltsaykin]

** Also affects: mos
   Importance: Undecided
   Status: New

** Changed in: mos
Milestone: None => 9.2-mu-3

** Changed in: mos
 Assignee: (unassigned) => Oleg Bondarev (obondarev)

** Changed in: mos
   Importance: Undecided => Medium

** Changed in: mos
   Status: New => In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1702635

Title:
  SR-IOV: sometimes a port may hang in BUILD state

Status in Mirantis OpenStack:
  In Progress
Status in neutron:
  In Progress

Bug description:
  Scenario:

  1) vfio-pci driver is used for VFs
  2) 2 ports are created in neutron with binding type 'direct'
  3) VMs are spawned and deleted on 2 compute nodes using pre-created ports
  4) one neutron port may be bound to different compute nodes at different
     moments
  5) for some reason (probably a bug, but current bug is not about it)
     vfio-pci is not properly handling VF reset after VM deletion and for
     sriov agent it looks like some port's MAC is still mapped to some PCI
     slot though the port is not bound to the node
  6) sriov agent requests port info from server with
     get_devices_details_list() but doesn't specify 'host' in parameters
  7) in this case neutron server sets this port to BUILD, though it may be
     bound to another host:

  def _get_new_status(self, host, port_context):
  port = port_context.current
  if not host or host == port_context.host:
  new_status = (n_const.PORT_STATUS_BUILD if port['admin_state_up']
    else n_const.PORT_STATUS_DOWN)
  if port['status'] != new_status:
  return new_status

  8) after processing, the agent notifies server with update_device_list() and 
this time specifies 'host' parameter
  9) server detects port's and agent's host mismatch and doesn't update status 
of the port
  10) port stays in BUILD state

  A simple fix would be to specify host at step 6 - in this case neutron
  server won't set port's status to BUILD because of host mismatch.

To manage notifications about this bug go to:
https://bugs.launchpad.net/mos/+bug/1702635/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp