Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic

2018-10-04 Thread Moshe Levi
Hi Julia,

Apologize we were not able to be there to better represent the use case.

PSB

From: Julia Kreger 
Sent: Monday, October 1, 2018 11:07 PM
To: OpenStack Development Mailing List (not for usage questions) 

Cc: isaku.yamah...@intel.com; Eyal Lavee 
Subject: Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic

Greetings, Comments in-line.

Thanks,

-Julia

On Sat, Sep 29, 2018 at 11:27 PM Moshe Levi 
mailto:mosh...@mellanox.com>> wrote:
Hi Julia,

I don't mind to update the ironic spec [1]. Unfortunately, I wasn't in the PTG 
but I had a sync meeting with Isuku.

As I see it there is 2 use-cases:
1.   Running the neutron ovs agent in the smartnic
2.   Running the neutron super ovs agent which manage the ovs running on 
the smartnic.

My takeaway from the meeting with neutron is that there would not be a neutron 
ovs agent running on the smartnic. That the configuration would need to be 
pushed at all times, which is ultimately better security wise if the tenant NIC 
is somehow compromised it reduces the control plane exposure.
[ML] - Can you elaborate on  the security concerns with running the neutron ovs 
agent on the smart NIC?
If you compare this to the standard virtualization use case, this is as secure 
if not more secure.
The tenant image runs in the bare metal host and receives only a network 
interface/port.
The host has no way to access the OS/services/agents running on the smart NIC 
CPUs, in the same way that a tenant image running in a VM has no way to access 
the services/agents running in the hypervisor.
It is in fact event more secure, as they are running in physically disjoint 
hardware and memory (thus not accessible even through side-channel 
vulnerabilities such as meltdown/spectre).

1.

It seem that most of the discussion was around the second use-case.

By the time Ironic and Neutron met together, it seemed like the first use case 
was no longer under consideration. I may be wrong, but very strong preference 
existed for the second scenario when we met the next day.
[ML] –
We are seeing great interest on smart NICs for bare metal use cases to allow to 
provide services (networking, storage and others) to bare metal servers that 
were previously only possible for VMs.
Conceptually the smart NIC can be thought of as an isolated hypervisor layer 
for the bare metal host.
The first service we are targeting in this spec is aligning the bare metal 
networking with the standard neutron ovs agent.
The target is to try to align (as possible) the bare metal implementation to 
the virtualization use case, up to the point of actually running (as possible) 
the same agents on the smart NIC (again acting as a hypervisor for the bare 
metal host use case).
This allows to reuse/align the implementation, and naturally scales with the 
number of bare metal servers, as opposed to running the agents on controller 
nodes, requiring potentially scaling the controllers to match the number of 
bare metal servers.
It also provides a path to providing more advanced services in the smart NIC in 
the next steps (not limiting the implementation to be OVSDB protocol specific).


This is my understanding on the ironic neutron PTG meeting:

  1.  Ironic cores don't want to change the deployment interface as proposed in 
[1].
  2.  We should  a new network_interface for use case 2. But what about the 
first use case? Should it be a new network_interface as well?
  3.  We should delay the port binding until the baremetal is powered on the 
ovs is running.

 *   For the first use case I was thinking to change the neutron server to 
just keep the port binding information in the neutron DB. Then when the neutron 
ovs agent is a live it will retrieve all the baremeal port , add them to the 
ovsdb and start the neutron ovs agent fullsync.
 *   For the second use case the agent is alive so the agent itself can 
monitor the ovsdb of the baremetal and configure it when it up

  1.  How to notify that neutron agent successfully/unsuccessfully bind the 
port ?

 *   In both use-cases we should use neutron-ironic notification to make 
sure the port binding was done successfully.

Is my understanding correct?

Not quite.

1) We as in Ironic recognize that there would need to be changes, it is the 
method as to how that we would prefer to be explicit and have chosen by the 
interface. The underlying behavior needs to be different, and the new 
network_interface should support both cases 1 and 2 because that interface 
contain needed logic for the conductor to determine the appropriate path 
forward. We should likely also put some guards in to prevent non-smart 
interfaces from being used in the same configuration due to the security issues 
that creates.
3) I believe this would be more of a matter of the network_interface knowing 
that the machine is powered up, and attempting to assert configuration through 
Neutron to push configuration to the smartnic.
3a) The 

Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic

2018-10-01 Thread Julia Kreger
Greetings, Comments in-line.

Thanks,

-Julia

On Sat, Sep 29, 2018 at 11:27 PM Moshe Levi  wrote:

> Hi Julia,
>
>
>
> I don't mind to update the ironic spec [1]. Unfortunately, I wasn't in the
> PTG but I had a sync meeting with Isuku.
>
>
>
> As I see it there is 2 use-cases:
>
>1. Running the neutron ovs agent in the smartnic
>2. Running the neutron super ovs agent which manage the ovs running on
>the smartnic.
>
>
My takeaway from the meeting with neutron is that there would not be a
neutron ovs agent running on the smartnic. That the configuration would
need to be pushed at all times, which is ultimately better security wise if
the tenant NIC is somehow compromised it reduces the control plane exposure.


>1.
>
>
>
> It seem that most of the discussion was around the second use-case.
>

By the time Ironic and Neutron met together, it seemed like the first use
case was no longer under consideration. I may be wrong, but very strong
preference existed for the second scenario when we met the next day.


>
> This is my understanding on the ironic neutron PTG meeting:
>
>1. Ironic cores don't want to change the deployment interface as
>proposed in [1].
>2. We should  a new network_interface for use case 2. But what about
>the first use case? Should it be a new network_interface as well?
>3. We should delay the port binding until the baremetal is powered on
>the ovs is running.
>   1. For the first use case I was thinking to change the neutron
>   server to just keep the port binding information in the neutron DB. Then
>   when the neutron ovs agent is a live it will retrieve all the baremeal 
> port
>   , add them to the ovsdb and start the neutron ovs agent fullsync.
>   2. For the second use case the agent is alive so the agent itself
>   can monitor the ovsdb of the baremetal and configure it when it up
>4. How to notify that neutron agent successfully/unsuccessfully bind
>the port ?
>   1. In both use-cases we should use neutron-ironic notification to
>   make sure the port binding was done successfully.
>
>
>
> Is my understanding correct?
>
>
> Not quite.

1) We as in Ironic recognize that there would need to be changes, it is the
method as to how that we would prefer to be explicit and have chosen by the
interface. The underlying behavior needs to be different, and the new
network_interface should support both cases 1 and 2 because that interface
contain needed logic for the conductor to determine the appropriate path
forward. We should likely also put some guards in to prevent non-smart
interfaces from being used in the same configuration due to the security
issues that creates.
3) I believe this would be more of a matter of the network_interface
knowing that the machine is powered up, and attempting to assert
configuration through Neutron to push configuration to the smartnic.
3a) The consensus is that the information to access the smartnic is
hardware configuration metadata and that ironic should be the source of
truth for information about that hardware. The discussion was push that as
needed into neutron to help enable the attachment. I proposed just
including it in the binding profile as a possibility, since it is transient
information.
3b) As I understood it, this would ultimately be the default operating
behavior.
4) Was not discussed, but something along the path is going to have to
check and retry as necessary. That item could be in the network_interface
code.
4a) This doesn't exist yet.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic

2018-10-01 Thread Isaku Yamahata
Hello. I'm willing to help it.

For detailed tech discussion, gerrit with updated spec would be better, though.
I'd like to supplement on Neutron port binding.

On Sun, Sep 30, 2018 at 06:25:58AM +,
Moshe Levi  wrote:

> Hi Julia,
> 
> I don't mind to update the ironic spec [1]. Unfortunately, I wasn't in the 
> PTG but I had a sync meeting with Isuku.
> 
> As I see it there is 2 use-cases:
> 
>   1.  Running the neutron ovs agent in the smartnic
>   2.  Running the neutron super ovs agent which manage the ovs running on the 
> smartnic.
> 
> It seem that most of the discussion was around the second use-case.
> 
> This is my understanding on the ironic neutron PTG meeting:
> 
>   1.  Ironic cores don't want to change the deployment interface as proposed 
> in [1].
>   2.  We should  a new network_interface for use case 2. But what about the 
> first use case? Should it be a new network_interface as well?

  * Which component, Ironic or Neutron, should take care that
SmartNIC is up/ready?
  * How is the up/readiness of smartnic defined?
Common way or device-dependent way. For example,
- able to connect via ovsdb/openflow
- agent responsible for smartNIC has reported to Neutron agentdb.
- able to ssh into smartnic
- device specific way with driver for each device.
- etc.


>   3.  We should delay the port binding until the baremetal is powered on the 
> ovs is running.
>  *   For the first use case I was thinking to change the neutron server 
> to just keep the port binding information in the neutron DB. Then when the 
> neutron ovs agent is a live it will retrieve all the baremeal port , add them 
> to the ovsdb and start the neutron ovs agent fullsync.
>  *   For the second use case the agent is alive so the agent itself can 
> monitor the ovsdb of the baremetal and configure it when it up

Can you please elaborate why port binding for smartnic should be delayed?
I'm failing to see the necessity. Probably I'm missing something.
Since we can skip the check of agent liveness with the assumption that
hostid given by Ironic is correct, we don't have to delay the port binding
for both case, 1 and 2 as above.


>   4.  How to notify that neutron agent successfully/unsuccessfully bind the 
> port ?
>  *   In both use-cases we should use neutron-ironic notification to make 
> sure the port binding was done successfully.

I agree that neutron-ironic notification is necessary.

There seems the confusion of port-binding and ovs-programing. The
success/failure of port-binding is the result of neutron port-update.
The current code synchronously checks the ovs-agent liveness on the host and
parameter validity. port-binding doesn't directly/synchronously program ovs.
It only triggers to start ovs programming eventually.

On the other hand, the success/failure of ovs programming is
asynchronous regarding to neutron REST API because ovs-agent does it
asynchronously on behalf of neutron server.
So here neutron-ironic notification is necessary.
When the ovs programming is done, the agent sets port::status = UP from DOWN.
(and nova is notified that port is ready to use.)
In the case of the failure, port::status is set to ERROR.

Thanks,

>
> Is my understanding correct?
> 
> 
> 
> [1] - https://review.openstack.org/#/c/582767/
> 
> From: Miguel Lavalle 
> Sent: Sunday, September 30, 2018 3:20 AM
> To: OpenStack Development Mailing List 
> Subject: Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic
> 
> Hi,
> 
> Yes, this also matches the recollection of the joint conversation in Denver. 
> Please look at the "Ironic x-project discussion - Smartnics" section in 
> http://lists.openstack.org/pipermail/openstack-dev/2018-September/135032.html<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openstack.org%2Fpipermail%2Fopenstack-dev%2F2018-September%2F135032.html&data=02%7C01%7Cmoshele%40mellanox.com%7Ce5607928a41c44bf065508d6266a9435%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636738636550233328&sdata=ZvAsZoTkk%2FFGlsig54J0d1EejcDiYdqsGlfRkuKBJTg%3D&reserved=0>
> 
> Regards
> 
> Miguel
> 
> 
> 
> On Thu, Sep 27, 2018 at 1:31 PM Julia Kreger 
> mailto:juliaashleykre...@gmail.com>> wrote:
> Greetings everyone,
> 
> Now that the PTG is over, I would like to go ahead and get the
> specification that was proposed to ironic-specs updated to represent
> the discussions that took place at the PTG.
> 
> A few highlights from my recollection:
> 
> * Ironic being the source of truth for the hardware configuration for
> the neutron agent to determine where to push configuration to. This
> would include the address and credential information (certificates
> right?).
> * The inform

Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic

2018-09-29 Thread Moshe Levi
Hi Julia,

I don't mind to update the ironic spec [1]. Unfortunately, I wasn't in the PTG 
but I had a sync meeting with Isuku.

As I see it there is 2 use-cases:

  1.  Running the neutron ovs agent in the smartnic
  2.  Running the neutron super ovs agent which manage the ovs running on the 
smartnic.

It seem that most of the discussion was around the second use-case.

This is my understanding on the ironic neutron PTG meeting:

  1.  Ironic cores don't want to change the deployment interface as proposed in 
[1].
  2.  We should  a new network_interface for use case 2. But what about the 
first use case? Should it be a new network_interface as well?
  3.  We should delay the port binding until the baremetal is powered on the 
ovs is running.
 *   For the first use case I was thinking to change the neutron server to 
just keep the port binding information in the neutron DB. Then when the neutron 
ovs agent is a live it will retrieve all the baremeal port , add them to the 
ovsdb and start the neutron ovs agent fullsync.
 *   For the second use case the agent is alive so the agent itself can 
monitor the ovsdb of the baremetal and configure it when it up
  4.  How to notify that neutron agent successfully/unsuccessfully bind the 
port ?
 *   In both use-cases we should use neutron-ironic notification to make 
sure the port binding was done successfully.

Is my understanding correct?



[1] - https://review.openstack.org/#/c/582767/

From: Miguel Lavalle 
Sent: Sunday, September 30, 2018 3:20 AM
To: OpenStack Development Mailing List 
Subject: Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic

Hi,

Yes, this also matches the recollection of the joint conversation in Denver. 
Please look at the "Ironic x-project discussion - Smartnics" section in 
http://lists.openstack.org/pipermail/openstack-dev/2018-September/135032.html<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openstack.org%2Fpipermail%2Fopenstack-dev%2F2018-September%2F135032.html&data=02%7C01%7Cmoshele%40mellanox.com%7Ce5607928a41c44bf065508d6266a9435%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636738636550233328&sdata=ZvAsZoTkk%2FFGlsig54J0d1EejcDiYdqsGlfRkuKBJTg%3D&reserved=0>

Regards

Miguel



On Thu, Sep 27, 2018 at 1:31 PM Julia Kreger 
mailto:juliaashleykre...@gmail.com>> wrote:
Greetings everyone,

Now that the PTG is over, I would like to go ahead and get the
specification that was proposed to ironic-specs updated to represent
the discussions that took place at the PTG.

A few highlights from my recollection:

* Ironic being the source of truth for the hardware configuration for
the neutron agent to determine where to push configuration to. This
would include the address and credential information (certificates
right?).
* The information required is somehow sent to neutron (possibly as
part of the binding profile, which we could send at each time port
actions are requested by Ironic.)
* The Neutron agent running on the control plane connects outbound to
the smartnic, using information supplied to perform the appropriate
network configuration.
* In Ironic, this would likely be a new network_interface driver
module, with some additional methods that help facilitate the
work-flow logic changes needed in each deploy_interface driver module.
* Ironic would then be informed or gain awareness that the
configuration has been completed and that the deployment can proceed.
(A different spec has been proposed regarding this.)

I have submitted a forum session based upon this and the agreed upon
goal at the PTG was to have the ironic spec written up to describe the
required changes.

I guess the next question is, who wants to update the specification?

-Julia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2FOpenStack-dev-request%40lists.openstack.org%3Fsubject%3Aunsubscribe&data=02%7C01%7Cmoshele%40mellanox.com%7Ce5607928a41c44bf065508d6266a9435%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636738636550233328&sdata=PexKWkCH7u4kRWjzs2kOIZsHFmzSL%2BCl6bqzY2B5KWA%3D&reserved=0>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.openstack.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fopenstack-dev&data=02%7C01%7Cmoshele%40mellanox.com%7Ce5607928a41c44bf065508d6266a9435%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636738636550243338&sdata=dHTuFDlyKrA8ouPU7JV4GKokCKNBg%2Fzw9KlpIrZW5x0%3D&reserved=0>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [ironic][neutron] SmartNics with Ironic

2018-09-29 Thread Miguel Lavalle
Hi,

Yes, this also matches the recollection of the joint conversation in
Denver. Please look at the "Ironic x-project discussion - Smartnics"
section in
http://lists.openstack.org/pipermail/openstack-dev/2018-September/135032.html

Regards

Miguel



On Thu, Sep 27, 2018 at 1:31 PM Julia Kreger 
wrote:

> Greetings everyone,
>
> Now that the PTG is over, I would like to go ahead and get the
> specification that was proposed to ironic-specs updated to represent
> the discussions that took place at the PTG.
>
> A few highlights from my recollection:
>
> * Ironic being the source of truth for the hardware configuration for
> the neutron agent to determine where to push configuration to. This
> would include the address and credential information (certificates
> right?).
> * The information required is somehow sent to neutron (possibly as
> part of the binding profile, which we could send at each time port
> actions are requested by Ironic.)
> * The Neutron agent running on the control plane connects outbound to
> the smartnic, using information supplied to perform the appropriate
> network configuration.
> * In Ironic, this would likely be a new network_interface driver
> module, with some additional methods that help facilitate the
> work-flow logic changes needed in each deploy_interface driver module.
> * Ironic would then be informed or gain awareness that the
> configuration has been completed and that the deployment can proceed.
> (A different spec has been proposed regarding this.)
>
> I have submitted a forum session based upon this and the agreed upon
> goal at the PTG was to have the ironic spec written up to describe the
> required changes.
>
> I guess the next question is, who wants to update the specification?
>
> -Julia
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [ironic][neutron] SmartNics with Ironic

2018-09-27 Thread Julia Kreger
Greetings everyone,

Now that the PTG is over, I would like to go ahead and get the
specification that was proposed to ironic-specs updated to represent
the discussions that took place at the PTG.

A few highlights from my recollection:

* Ironic being the source of truth for the hardware configuration for
the neutron agent to determine where to push configuration to. This
would include the address and credential information (certificates
right?).
* The information required is somehow sent to neutron (possibly as
part of the binding profile, which we could send at each time port
actions are requested by Ironic.)
* The Neutron agent running on the control plane connects outbound to
the smartnic, using information supplied to perform the appropriate
network configuration.
* In Ironic, this would likely be a new network_interface driver
module, with some additional methods that help facilitate the
work-flow logic changes needed in each deploy_interface driver module.
* Ironic would then be informed or gain awareness that the
configuration has been completed and that the deployment can proceed.
(A different spec has been proposed regarding this.)

I have submitted a forum session based upon this and the agreed upon
goal at the PTG was to have the ironic spec written up to describe the
required changes.

I guess the next question is, who wants to update the specification?

-Julia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev