Re: [ovs-dev] [PATCH 1/8] doc: Add an overview of the 'dpdk' port

[Stokes, Ian]

> On Mon, 2018-04-09 at 15:15 +, Stokes, Ian wrote:
> > > These ports are used to allow ingress/egress from the host and are
> > > therefore _reasonably_ important. However, there is no clear
> > > overview of what these ports actually are or why things are done the
> way they are.
> > > Start closing this gap by providing a standalone example of using
> > > these ports along with a little more detailed overview of the binding
> process.
> > >
> > > There is additional cleanup to be done for the DPDK howto, but that
> > > will be done separately.
> > >
> > > Signed-off-by: Stephen Finucane 
> > > Cc: Ciara Loftus 
> > > Cc: Kevin Traynor 
> > > ---
> > >  Documentation/topics/dpdk/index.rst |   1 +
> > >  Documentation/topics/dpdk/phy.rst   | 111
> > > 
> > >  2 files changed, 112 insertions(+)
> > >  create mode 100644 Documentation/topics/dpdk/phy.rst
> > >
> > > diff --git a/Documentation/topics/dpdk/index.rst
> > > b/Documentation/topics/dpdk/index.rst
> > > index da148b323..5f836a6e9 100644
> > > --- a/Documentation/topics/dpdk/index.rst
> > > +++ b/Documentation/topics/dpdk/index.rst
> > > @@ -28,5 +28,6 @@ The DPDK Datapath
> > >  .. toctree::
> > > :maxdepth: 2
> > >
> > > +   phy
> > > vhost-user
> > > ring
> > > diff --git a/Documentation/topics/dpdk/phy.rst
> > > b/Documentation/topics/dpdk/phy.rst
> > > new file mode 100644
> > > index 0..1c18e4e3d
> > > --- /dev/null
> > > +++ b/Documentation/topics/dpdk/phy.rst
> > > @@ -0,0 +1,111 @@
> > > +..
> > > +  Copyright 2018, Red Hat, Inc.
> > > +
> > > +  Licensed under the Apache License, Version 2.0 (the
> > > + "License"); you
> > > may
> > > +  not use this file except in compliance with the License. You
> > > + may
> > > obtain
> > > +  a copy of the License at
> > > +
> > > +  http://www.apache.org/licenses/LICENSE-2.0
> > > +
> > > +  Unless required by applicable law or agreed to in writing,
> software
> > > +  distributed under the License is distributed on an "AS IS"
> > > + BASIS,
> > > WITHOUT
> > > +  WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> implied.
> > > See the
> > > +  License for the specific language governing permissions and
> > > limitations
> > > +  under the License.
> > > +
> > > +  Convention for heading levels in Open vSwitch documentation:
> > > +
> > > +  ===  Heading 0 (reserved for the title in a document)
> > > +  ---  Heading 1
> > > +  ~~~  Heading 2
> > > +  +++  Heading 3
> > > +  '''  Heading 4
> > > +
> > > +  Avoid deeper levels because they do not render well.
> > > +
> > > +===
> > > +DPDK Physical Ports
> > > +===
> > > +
> > > +The DPDK datapath provides a way to attach DPDK-backed physical
> > > +interfaces to allow high-performance ingress/egress from the host.
> > > +
> > > +.. versionchanged:: 2.7.0
> > > +
> > > +   Before Open vSwitch 2.7.0, it was necessary to prefix port names
> > > + with
> > > a
> > > +   ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary.
> > > +
> > > +Quick Example
> > > +-
> > > +
> > > +This example demonstrates how to bind two ``dpdk`` ports, bound to
> > > +physical interfaces identified by hardware IDs ``:01:00.0`` and
> > > +``:01:00.1``, to an existing bridge called ``br0``::
> > > +
> > > +$ ovs-vsctl add-port br0 dpdk-p0 \
> > > +   -- set Interface dpdk-p0 type=dpdk options:dpdk-
> > > devargs=:01:00.0
> > > +$ ovs-vsctl add-port br0 dpdk-p1 \
> > > +   -- set Interface dpdk-p1 type=dpdk
> > > + options:dpdk-devargs=:01:00.1
> > > +
> > > +For the above example to work, the two physical interfaces must be
> > > +bound to the DPDK poll-mode drivers in userspace rather than the
> > > +traditional kernel drivers. See the `binding NIC drivers
> > > + > > nics>` section for details.
> >
> > I think an example should be added here for when multiple ports share
> > the same bus slot function.
> >
> > Support for this was added in as part of OVS 2.9.
> >
> > If not added here then we need to flag clearly that it's supported but
> > that users need to consult the dpdk-binding-nic section for specifics.
> 
> This is only a quick intro so I don't think it would belong here.
> However, we could definitely add this as another section. I'm not familiar

Sure, can be a to-do, once it's called out as an option along with a link to 
more detailed doc it should be ok. Some of these devices don’t use igb_uoi or 
vfio either. I think it's already documented in the phy doc so a note + a link 
would suffice.

Ian

> with this feature so would it be possible to submit this as a follow-up?
> I'd be happy to review it from a docs perspective.
> 
> Stephen
> 
> > Ian
> > > +
> > > +.. _dpdk-binding-nics:
> > > +
> > > +Binding NIC Drivers
> > > +---
> > > +
> 

Re: [ovs-dev] [PATCH 1/8] doc: Add an overview of the 'dpdk' port

[Stephen Finucane]

On Mon, 2018-04-09 at 15:15 +, Stokes, Ian wrote:
> > These ports are used to allow ingress/egress from the host and are
> > therefore _reasonably_ important. However, there is no clear overview of
> > what these ports actually are or why things are done the way they are.
> > Start closing this gap by providing a standalone example of using these
> > ports along with a little more detailed overview of the binding process.
> > 
> > There is additional cleanup to be done for the DPDK howto, but that will
> > be done separately.
> > 
> > Signed-off-by: Stephen Finucane 
> > Cc: Ciara Loftus 
> > Cc: Kevin Traynor 
> > ---
> >  Documentation/topics/dpdk/index.rst |   1 +
> >  Documentation/topics/dpdk/phy.rst   | 111
> > 
> >  2 files changed, 112 insertions(+)
> >  create mode 100644 Documentation/topics/dpdk/phy.rst
> > 
> > diff --git a/Documentation/topics/dpdk/index.rst
> > b/Documentation/topics/dpdk/index.rst
> > index da148b323..5f836a6e9 100644
> > --- a/Documentation/topics/dpdk/index.rst
> > +++ b/Documentation/topics/dpdk/index.rst
> > @@ -28,5 +28,6 @@ The DPDK Datapath
> >  .. toctree::
> > :maxdepth: 2
> > 
> > +   phy
> > vhost-user
> > ring
> > diff --git a/Documentation/topics/dpdk/phy.rst
> > b/Documentation/topics/dpdk/phy.rst
> > new file mode 100644
> > index 0..1c18e4e3d
> > --- /dev/null
> > +++ b/Documentation/topics/dpdk/phy.rst
> > @@ -0,0 +1,111 @@
> > +..
> > +  Copyright 2018, Red Hat, Inc.
> > +
> > +  Licensed under the Apache License, Version 2.0 (the "License"); you
> > may
> > +  not use this file except in compliance with the License. You may
> > obtain
> > +  a copy of the License at
> > +
> > +  http://www.apache.org/licenses/LICENSE-2.0
> > +
> > +  Unless required by applicable law or agreed to in writing, software
> > +  distributed under the License is distributed on an "AS IS" BASIS,
> > WITHOUT
> > +  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > See the
> > +  License for the specific language governing permissions and
> > limitations
> > +  under the License.
> > +
> > +  Convention for heading levels in Open vSwitch documentation:
> > +
> > +  ===  Heading 0 (reserved for the title in a document)
> > +  ---  Heading 1
> > +  ~~~  Heading 2
> > +  +++  Heading 3
> > +  '''  Heading 4
> > +
> > +  Avoid deeper levels because they do not render well.
> > +
> > +===
> > +DPDK Physical Ports
> > +===
> > +
> > +The DPDK datapath provides a way to attach DPDK-backed physical
> > +interfaces to allow high-performance ingress/egress from the host.
> > +
> > +.. versionchanged:: 2.7.0
> > +
> > +   Before Open vSwitch 2.7.0, it was necessary to prefix port names with
> > a
> > +   ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary.
> > +
> > +Quick Example
> > +-
> > +
> > +This example demonstrates how to bind two ``dpdk`` ports, bound to
> > +physical interfaces identified by hardware IDs ``:01:00.0`` and
> > +``:01:00.1``, to an existing bridge called ``br0``::
> > +
> > +$ ovs-vsctl add-port br0 dpdk-p0 \
> > +   -- set Interface dpdk-p0 type=dpdk options:dpdk-
> > devargs=:01:00.0
> > +$ ovs-vsctl add-port br0 dpdk-p1 \
> > +   -- set Interface dpdk-p1 type=dpdk
> > + options:dpdk-devargs=:01:00.1
> > +
> > +For the above example to work, the two physical interfaces must be
> > +bound to the DPDK poll-mode drivers in userspace rather than the
> > +traditional kernel drivers. See the `binding NIC drivers  > nics>` section for details.
> 
> I think an example should be added here for when multiple ports share
> the same bus slot function.
> 
> Support for this was added in as part of OVS 2.9.
> 
> If not added here then we need to flag clearly that it's supported
> but that users need to consult the dpdk-binding-nic section for
> specifics.

This is only a quick intro so I don't think it would belong here.
However, we could definitely add this as another section. I'm not
familiar with this feature so would it be possible to submit this as a
follow-up? I'd be happy to review it from a docs perspective.

Stephen 

> Ian
> > +
> > +.. _dpdk-binding-nics:
> > +
> > +Binding NIC Drivers
> > +---
> > +
> > +DPDK operates entirely in userspace and, as a result, requires use of
> > +its own poll-mode drivers in user space for physical interfaces and a
> > +passthrough-style driver for the devices in kernel space.
> > +
> > +There are two different tools for binding drivers: :command:`driverctl`
> > +which is a generic tool for persistently configuring alternative device
> > +drivers, and :command:`dpdk-devbind` which is a DPDK-specific tool and
> > +whose changes do not persist across reboots. In addition, there are two
> > +options 

Re: [ovs-dev] [PATCH 1/8] doc: Add an overview of the 'dpdk' port

[Stokes, Ian]

> These ports are used to allow ingress/egress from the host and are
> therefore _reasonably_ important. However, there is no clear overview of
> what these ports actually are or why things are done the way they are.
> Start closing this gap by providing a standalone example of using these
> ports along with a little more detailed overview of the binding process.
> 
> There is additional cleanup to be done for the DPDK howto, but that will
> be done separately.
> 
> Signed-off-by: Stephen Finucane 
> Cc: Ciara Loftus 
> Cc: Kevin Traynor 
> ---
>  Documentation/topics/dpdk/index.rst |   1 +
>  Documentation/topics/dpdk/phy.rst   | 111
> 
>  2 files changed, 112 insertions(+)
>  create mode 100644 Documentation/topics/dpdk/phy.rst
> 
> diff --git a/Documentation/topics/dpdk/index.rst
> b/Documentation/topics/dpdk/index.rst
> index da148b323..5f836a6e9 100644
> --- a/Documentation/topics/dpdk/index.rst
> +++ b/Documentation/topics/dpdk/index.rst
> @@ -28,5 +28,6 @@ The DPDK Datapath
>  .. toctree::
> :maxdepth: 2
> 
> +   phy
> vhost-user
> ring
> diff --git a/Documentation/topics/dpdk/phy.rst
> b/Documentation/topics/dpdk/phy.rst
> new file mode 100644
> index 0..1c18e4e3d
> --- /dev/null
> +++ b/Documentation/topics/dpdk/phy.rst
> @@ -0,0 +1,111 @@
> +..
> +  Copyright 2018, Red Hat, Inc.
> +
> +  Licensed under the Apache License, Version 2.0 (the "License"); you
> may
> +  not use this file except in compliance with the License. You may
> obtain
> +  a copy of the License at
> +
> +  http://www.apache.org/licenses/LICENSE-2.0
> +
> +  Unless required by applicable law or agreed to in writing, software
> +  distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT
> +  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the
> +  License for the specific language governing permissions and
> limitations
> +  under the License.
> +
> +  Convention for heading levels in Open vSwitch documentation:
> +
> +  ===  Heading 0 (reserved for the title in a document)
> +  ---  Heading 1
> +  ~~~  Heading 2
> +  +++  Heading 3
> +  '''  Heading 4
> +
> +  Avoid deeper levels because they do not render well.
> +
> +===
> +DPDK Physical Ports
> +===
> +
> +The DPDK datapath provides a way to attach DPDK-backed physical
> +interfaces to allow high-performance ingress/egress from the host.
> +
> +.. versionchanged:: 2.7.0
> +
> +   Before Open vSwitch 2.7.0, it was necessary to prefix port names with
> a
> +   ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary.
> +
> +Quick Example
> +-
> +
> +This example demonstrates how to bind two ``dpdk`` ports, bound to
> +physical interfaces identified by hardware IDs ``:01:00.0`` and
> +``:01:00.1``, to an existing bridge called ``br0``::
> +
> +$ ovs-vsctl add-port br0 dpdk-p0 \
> +   -- set Interface dpdk-p0 type=dpdk options:dpdk-
> devargs=:01:00.0
> +$ ovs-vsctl add-port br0 dpdk-p1 \
> +   -- set Interface dpdk-p1 type=dpdk
> + options:dpdk-devargs=:01:00.1
> +
> +For the above example to work, the two physical interfaces must be
> +bound to the DPDK poll-mode drivers in userspace rather than the
> +traditional kernel drivers. See the `binding NIC drivers  nics>` section for details.

I think an example should be added here for when multiple ports share the same 
bus slot function.

Support for this was added in as part of OVS 2.9.

If not added here then we need to flag clearly that it's supported but that 
users need to consult the dpdk-binding-nic section for specifics.
 
Ian
> +
> +.. _dpdk-binding-nics:
> +
> +Binding NIC Drivers
> +---
> +
> +DPDK operates entirely in userspace and, as a result, requires use of
> +its own poll-mode drivers in user space for physical interfaces and a
> +passthrough-style driver for the devices in kernel space.
> +
> +There are two different tools for binding drivers: :command:`driverctl`
> +which is a generic tool for persistently configuring alternative device
> +drivers, and :command:`dpdk-devbind` which is a DPDK-specific tool and
> +whose changes do not persist across reboots. In addition, there are two
> +options available for this kernel space driver - VFIO (Virtual Function
> +I/O) and UIO (Userspace I/O) - along with a number of drivers for each
> +option. We will demonstrate examples of both tools and will use the
> +``vfio-pci`` driver, which is the more secure, robust driver of those
> +available. More information can be found in the `DPDK documentation
> `__.
> +
> +To list devices using :command:`driverctl`, run::
> +
> +$ driverctl -v list-devices | grep -i net
> +:07:00.0 igb (I350 Gigabit Network Connection (Ethernet Server
> Adapter I350-T2))
> +

Re: [ovs-dev] [PATCH 1/8] doc: Add an overview of the 'dpdk' port

[Aaron Conole]

Hi Stephen,

Stephen Finucane  writes:

> These ports are used to allow ingress/egress from the host and are
> therefore _reasonably_ important. However, there is no clear overview of
> what these ports actually are or why things are done the way they are.
> Start closing this gap by providing a standalone example of using these
> ports along with a little more detailed overview of the binding process.
>
> There is additional cleanup to be done for the DPDK howto, but that will
> be done separately.
>
> Signed-off-by: Stephen Finucane 
> Cc: Ciara Loftus 
> Cc: Kevin Traynor 
> ---
>  Documentation/topics/dpdk/index.rst |   1 +
>  Documentation/topics/dpdk/phy.rst   | 111 
> 
>  2 files changed, 112 insertions(+)
>  create mode 100644 Documentation/topics/dpdk/phy.rst
>
> diff --git a/Documentation/topics/dpdk/index.rst 
> b/Documentation/topics/dpdk/index.rst
> index da148b323..5f836a6e9 100644
> --- a/Documentation/topics/dpdk/index.rst
> +++ b/Documentation/topics/dpdk/index.rst
> @@ -28,5 +28,6 @@ The DPDK Datapath
>  .. toctree::
> :maxdepth: 2
>  
> +   phy
> vhost-user
> ring
> diff --git a/Documentation/topics/dpdk/phy.rst 
> b/Documentation/topics/dpdk/phy.rst
> new file mode 100644
> index 0..1c18e4e3d
> --- /dev/null
> +++ b/Documentation/topics/dpdk/phy.rst
> @@ -0,0 +1,111 @@
> +..
> +  Copyright 2018, Red Hat, Inc.
> +
> +  Licensed under the Apache License, Version 2.0 (the "License"); you may
> +  not use this file except in compliance with the License. You may obtain
> +  a copy of the License at
> +
> +  http://www.apache.org/licenses/LICENSE-2.0
> +
> +  Unless required by applicable law or agreed to in writing, software
> +  distributed under the License is distributed on an "AS IS" BASIS, 
> WITHOUT
> +  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See 
> the
> +  License for the specific language governing permissions and limitations
> +  under the License.
> +
> +  Convention for heading levels in Open vSwitch documentation:
> +
> +  ===  Heading 0 (reserved for the title in a document)
> +  ---  Heading 1
> +  ~~~  Heading 2
> +  +++  Heading 3
> +  '''  Heading 4
> +
> +  Avoid deeper levels because they do not render well.
> +
> +===
> +DPDK Physical Ports
> +===
> +
> +The DPDK datapath provides a way to attach DPDK-backed physical
> interfaces to

I think it may make more sense:

  The netdev datapath allows attaching DPDK-backed physical interfaces...

> +allow high-performance ingress/egress from the host.
> +
> +.. versionchanged:: 2.7.0
> +
> +   Before Open vSwitch 2.7.0, it was necessary to prefix port names with a
> +   ``dpdk`` prefix. Starting with 2.7.0, this is no longer necessary.
> +
> +Quick Example
> +-
> +
> +This example demonstrates how to bind two ``dpdk`` ports, bound to physical
> +interfaces identified by hardware IDs ``:01:00.0`` and ``:01:00.1``, 
> to
> +an existing bridge called ``br0``::
> +
> +$ ovs-vsctl add-port br0 dpdk-p0 \
> +   -- set Interface dpdk-p0 type=dpdk options:dpdk-devargs=:01:00.0
> +$ ovs-vsctl add-port br0 dpdk-p1 \
> +   -- set Interface dpdk-p1 type=dpdk options:dpdk-devargs=:01:00.1
> +
> +For the above example to work, the two physical interfaces must be bound to
> +the DPDK poll-mode drivers in userspace rather than the traditional kernel
> +drivers. See the `binding NIC drivers ` section for 
> details.
> +
> +.. _dpdk-binding-nics:
> +
> +Binding NIC Drivers
> +---
> +
> +DPDK operates entirely in userspace and, as a result, requires use of its own
> +poll-mode drivers in user space for physical interfaces and a 
> passthrough-style
> +driver for the devices in kernel space.
> +
> +There are two different tools for binding drivers: :command:`driverctl` which
> +is a generic tool for persistently configuring alternative device drivers, 
> and
> +:command:`dpdk-devbind` which is a DPDK-specific tool and whose changes do 
> not
> +persist across reboots. In addition, there are two options available for this
> +kernel space driver - VFIO (Virtual Function I/O) and UIO (Userspace I/O) -
> +along with a number of drivers for each option. We will demonstrate examples 
> of
> +both tools and will use the ``vfio-pci`` driver, which is the more secure,
> +robust driver of those available. More information can be found in the `DPDK
> +documentation `__.

Just commentary - really happy to see driverctl getting some love.

> +
> +To list devices using :command:`driverctl`, run::
> +
> +$ driverctl -v list-devices | grep -i net
> +:07:00.0 igb (I350 Gigabit Network Connection (Ethernet Server 
> Adapter I350-T2))
> +:07:00.1 igb (I350 Gigabit Network Connection (Ethernet Server