Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[David Miller]

From: Doug Ledford 
Date: Wed, 14 Jun 2017 15:30:13 -0400

> On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
>> From: Saeed Mahameed 
>> Date: Tue, 23 May 2017 14:43:58 +0300
>> 
>> > Hi Dave and Doug,
>> > 
>> > This series introduces some small updates and FPGA support to the
>> mlx5
>> > core/ethernet and IB drivers.
>> > 
>> > For more details please see below.
>> > 
>> > Please pull and let me know if there's any problem.
>> 
>> Ok, I've pulled this into net-next.
>> 
>> Doug let me know if there are any merge hassles we need to coordinate
>> on.
> 
> Turns out that you had pulled this prior to your net-next tree making
> it up to v4.12-rc3, while I had my -rc branch based on v4.12-rc3, so
> when I pulled just up to your merge commit for this pull request, it
> meant a later merge of my -rc branch would pull in a bunch of stuff
> between here and -rc3.  So I ended up pulling your latest net-next as
> of today as the easiest way to resolve that issue.  Now I can merge my
> -rc branch in and it won't cause any extra noise in the merge.  It
> seems to be a fairly regular pattern that I'm going to have to hold my
> pull request to Linus until after your tree is pulled, so I might just
> start planning on that from now on ;-)

Hehe, ok :)

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[Doug Ledford]

On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
> From: Saeed Mahameed 
> Date: Tue, 23 May 2017 14:43:58 +0300
> 
> > Hi Dave and Doug,
> > 
> > This series introduces some small updates and FPGA support to the
> mlx5
> > core/ethernet and IB drivers.
> > 
> > For more details please see below.
> > 
> > Please pull and let me know if there's any problem.
> 
> Ok, I've pulled this into net-next.
> 
> Doug let me know if there are any merge hassles we need to coordinate
> on.

Turns out that you had pulled this prior to your net-next tree making
it up to v4.12-rc3, while I had my -rc branch based on v4.12-rc3, so
when I pulled just up to your merge commit for this pull request, it
meant a later merge of my -rc branch would pull in a bunch of stuff
between here and -rc3.  So I ended up pulling your latest net-next as
of today as the easiest way to resolve that issue.  Now I can merge my
-rc branch in and it won't cause any extra noise in the merge.  It
seems to be a fairly regular pattern that I'm going to have to hold my
pull request to Linus until after your tree is pulled, so I might just
start planning on that from now on ;-)

-- 
Doug Ledford 
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[Saeed Mahameed]

On Fri, Jun 2, 2017 at 7:57 PM, Alexei Starovoitov
 wrote:
> On Fri, Jun 02, 2017 at 12:08:39PM -0400, David Miller wrote:
>> From: Alexei Starovoitov 
>> Date: Fri, 2 Jun 2017 09:06:43 -0700
>>
>> > On Fri, Jun 02, 2017 at 06:39:40PM +0300, Leon Romanovsky wrote:
>> >> On Thu, Jun 01, 2017 at 06:57:59PM -0400, Doug Ledford wrote:
>> >> > On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
>> >> > > From: Saeed Mahameed 
>> >> > > Date: Tue, 23 May 2017 14:43:58 +0300
>> >> > >
>> >> > > > Hi Dave and Doug,
>> >> > > >
>> >> > > > This series introduces some small updates and FPGA support to the
>> >> > > mlx5
>> >> > > > core/ethernet and IB drivers.
>> >> > > >
>> >> > > > For more details please see below.
>> >> > > >
>> >> > > > Please pull and let me know if there's any problem.
>> >> > >
>> >> > > Ok, I've pulled this into net-next.
>> >> > >
>> >> > > Doug let me know if there are any merge hassles we need to coordinate
>> >> > > on.
>> >> > >
>> >> > > Thanks.
>> >> >
>> >> > Will do.  But is the question of a possible revert settled?  Because
>> >> > once I pull, a revert means I have to pull all the way up to the revert
>> >> > as well...
>> >>
>> >> Revert is unlikely to happen, Saeed is working with Alexey to resolve
>> >> his raised concerns.
>> >
>> > I still think the direction is absolutely wrong.
>> > Reinventing fpga interfaces via nic driver is no-go.
>> > mlx needs to send a patch to revert it to show that they are willing
>> > to listen to the community.
>>
>> "Two people from Facebook" is not the commnity Alexei. :-)
>
> I've read the thread that fpga folks are not excited about
> fpga behind the nic either... but yeah i'm mainly expressing
> my opinions here.
>
>> And ironically, you guys will want to be one of the main users of
>> these facilities for the KTLS offload bits, so the desire to separate
>> it out and make "backports easier" really confuses me.
>
> We will not be using ktls offload via hidden fpga which
> is not properly exposed to the kernel.
> We've learned our lesson with eswitch.
>

KTLS/IPSec or any mlx5 FPGA/Innova applications is yet to be submitted,
and as Ilan said the kernel API is WIP, and we will not push anything
until the kernel API is submitted and accepted.

The patch in question that you are asking to revert, is adding no new
fancy functionality or FPGA capabilities to the mlx5 driver yet.
it is only basic mlx5 core stuff to add support of new device
"Mellanox Innova™ Flex 4 Lx EN Adapter Card" [1], so such device which
is already in production
can work with the mlx5_core.ko as is (ConnectX4-Lx) with no special
offloads or special Innova applications.
so this patch is mandatory for those who have this device to have
basic and standard ConnectX4-Lx networking interfaces.

It also adds the needed code separation between mlx5 basic core and
Innova stuff for future Innova applications and API development
(IPsec,KTLS, etc ..), so it will be easy for you to remove Innova
support at compilation time. And when the time comes, each new part
will be introduced with the suitable kernel API in the right place.

> ktls patches are independent of hw offload. It's great that
> mlx fpga can accelerate ktls tx which is extra confirmation
> that ktls api is on the right track. There is still 'ktls rx'
> to finish and 'ktls rx offload' is even trickier, since it's
> more intrusive into the core networking. Once crypto scatter gather
> is improved, we expect to see 8% perf gain out of sw ktls alone.
> So ktls hw offload is complementary and not mandatory.
>
> Back to eswitch analogy. All I hear from mlx is "at this stage
> it's not approriate ...". Well, I've been asking to do
> 'the right thing' for eswitch for months now and eswitch was
> in the driver for years. What are the chances that this fpga
> will ever be exposed? Once the code is merged there is no
> incentive for the vendor to do it differently.
>

"mlx5 eswitch == ethernet SRIOV" and today we already implement most
-if not all- of SRIOV VF ndos, and all are accessible from
"(ip link show) and (ip set vf xyz) " this is for the Legacy mode.
for the new switchdev mode, eswitch is managed by TC tool and
switchdev APIs only.

So we already have max kernel/user visibility in both modes.

can you clarify what bothers you and what is "the right thing"?
because i think we already agreed on the MLX5_ESWITCH/SRIOV kconfig
directive to eliminate eswitch for those who don't need it.
What exactly is missing ?

[1] 
http://www.mellanox.com/page/products_dyn?product_family=228=programmable_adapter_cards_

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[Or Gerlitz]

On Fri, Jun 2, 2017 at 7:57 PM, Alexei Starovoitov
 wrote:
> Back to eswitch analogy. All I hear from mlx is "at this stage
> it's not approriate ...". Well, I've been asking to do
> 'the right thing' for eswitch for months now and eswitch was
> in the driver for years. What are the chances that this fpga
> will ever be exposed? Once the code is merged there is no
> incentive for the vendor to do it differently.

Alexei, can you please clarify the claims re eswitch? are you
referring to the interfaces to program SRIOV eswitch? if not, what?

Or.

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[Alexei Starovoitov]

On Fri, Jun 02, 2017 at 12:08:39PM -0400, David Miller wrote:
> From: Alexei Starovoitov 
> Date: Fri, 2 Jun 2017 09:06:43 -0700
> 
> > On Fri, Jun 02, 2017 at 06:39:40PM +0300, Leon Romanovsky wrote:
> >> On Thu, Jun 01, 2017 at 06:57:59PM -0400, Doug Ledford wrote:
> >> > On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
> >> > > From: Saeed Mahameed 
> >> > > Date: Tue, 23 May 2017 14:43:58 +0300
> >> > >
> >> > > > Hi Dave and Doug,
> >> > > > 
> >> > > > This series introduces some small updates and FPGA support to the
> >> > > mlx5
> >> > > > core/ethernet and IB drivers.
> >> > > > 
> >> > > > For more details please see below.
> >> > > > 
> >> > > > Please pull and let me know if there's any problem.
> >> > >
> >> > > Ok, I've pulled this into net-next.
> >> > >
> >> > > Doug let me know if there are any merge hassles we need to coordinate
> >> > > on.
> >> > >
> >> > > Thanks.
> >> >
> >> > Will do.  But is the question of a possible revert settled?  Because
> >> > once I pull, a revert means I have to pull all the way up to the revert
> >> > as well...
> >> 
> >> Revert is unlikely to happen, Saeed is working with Alexey to resolve
> >> his raised concerns.
> > 
> > I still think the direction is absolutely wrong.
> > Reinventing fpga interfaces via nic driver is no-go.
> > mlx needs to send a patch to revert it to show that they are willing
> > to listen to the community.
> 
> "Two people from Facebook" is not the commnity Alexei. :-)

I've read the thread that fpga folks are not excited about
fpga behind the nic either... but yeah i'm mainly expressing
my opinions here.

> And ironically, you guys will want to be one of the main users of
> these facilities for the KTLS offload bits, so the desire to separate
> it out and make "backports easier" really confuses me.

We will not be using ktls offload via hidden fpga which
is not properly exposed to the kernel.
We've learned our lesson with eswitch.

ktls patches are independent of hw offload. It's great that
mlx fpga can accelerate ktls tx which is extra confirmation
that ktls api is on the right track. There is still 'ktls rx'
to finish and 'ktls rx offload' is even trickier, since it's
more intrusive into the core networking. Once crypto scatter gather
is improved, we expect to see 8% perf gain out of sw ktls alone.
So ktls hw offload is complementary and not mandatory.

Back to eswitch analogy. All I hear from mlx is "at this stage
it's not approriate ...". Well, I've been asking to do
'the right thing' for eswitch for months now and eswitch was
in the driver for years. What are the chances that this fpga
will ever be exposed? Once the code is merged there is no
incentive for the vendor to do it differently.

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[David Miller]

From: Alexei Starovoitov 
Date: Fri, 2 Jun 2017 09:06:43 -0700

> On Fri, Jun 02, 2017 at 06:39:40PM +0300, Leon Romanovsky wrote:
>> On Thu, Jun 01, 2017 at 06:57:59PM -0400, Doug Ledford wrote:
>> > On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
>> > > From: Saeed Mahameed 
>> > > Date: Tue, 23 May 2017 14:43:58 +0300
>> > >
>> > > > Hi Dave and Doug,
>> > > > 
>> > > > This series introduces some small updates and FPGA support to the
>> > > mlx5
>> > > > core/ethernet and IB drivers.
>> > > > 
>> > > > For more details please see below.
>> > > > 
>> > > > Please pull and let me know if there's any problem.
>> > >
>> > > Ok, I've pulled this into net-next.
>> > >
>> > > Doug let me know if there are any merge hassles we need to coordinate
>> > > on.
>> > >
>> > > Thanks.
>> >
>> > Will do.  But is the question of a possible revert settled?  Because
>> > once I pull, a revert means I have to pull all the way up to the revert
>> > as well...
>> 
>> Revert is unlikely to happen, Saeed is working with Alexey to resolve
>> his raised concerns.
> 
> I still think the direction is absolutely wrong.
> Reinventing fpga interfaces via nic driver is no-go.
> mlx needs to send a patch to revert it to show that they are willing
> to listen to the community.

"Two people from Facebook" is not the commnity Alexei. :-)

And ironically, you guys will want to be one of the main users of
these facilities for the KTLS offload bits, so the desire to separate
it out and make "backports easier" really confuses me.

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[Alexei Starovoitov]

On Fri, Jun 02, 2017 at 06:39:40PM +0300, Leon Romanovsky wrote:
> On Thu, Jun 01, 2017 at 06:57:59PM -0400, Doug Ledford wrote:
> > On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
> > > From: Saeed Mahameed 
> > > Date: Tue, 23 May 2017 14:43:58 +0300
> > >
> > > > Hi Dave and Doug,
> > > > 
> > > > This series introduces some small updates and FPGA support to the
> > > mlx5
> > > > core/ethernet and IB drivers.
> > > > 
> > > > For more details please see below.
> > > > 
> > > > Please pull and let me know if there's any problem.
> > >
> > > Ok, I've pulled this into net-next.
> > >
> > > Doug let me know if there are any merge hassles we need to coordinate
> > > on.
> > >
> > > Thanks.
> >
> > Will do.  But is the question of a possible revert settled?  Because
> > once I pull, a revert means I have to pull all the way up to the revert
> > as well...
> 
> Revert is unlikely to happen, Saeed is working with Alexey to resolve
> his raised concerns.

I still think the direction is absolutely wrong.
Reinventing fpga interfaces via nic driver is no-go.
mlx needs to send a patch to revert it to show that they are willing
to listen to the community.

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[Leon Romanovsky]

On Thu, Jun 01, 2017 at 06:57:59PM -0400, Doug Ledford wrote:
> On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
> > From: Saeed Mahameed 
> > Date: Tue, 23 May 2017 14:43:58 +0300
> >
> > > Hi Dave and Doug,
> > > 
> > > This series introduces some small updates and FPGA support to the
> > mlx5
> > > core/ethernet and IB drivers.
> > > 
> > > For more details please see below.
> > > 
> > > Please pull and let me know if there's any problem.
> >
> > Ok, I've pulled this into net-next.
> >
> > Doug let me know if there are any merge hassles we need to coordinate
> > on.
> >
> > Thanks.
>
> Will do.  But is the question of a possible revert settled?  Because
> once I pull, a revert means I have to pull all the way up to the revert
> as well...

Revert is unlikely to happen, Saeed is working with Alexey to resolve
his raised concerns.

Thanks

>
> --
> Doug Ledford 
>     GPG KeyID: B826A3330E572FDD
>    
> Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


signature.asc
Description: PGP signature

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[Doug Ledford]

On Thu, 2017-05-25 at 12:02 -0400, David Miller wrote:
> From: Saeed Mahameed 
> Date: Tue, 23 May 2017 14:43:58 +0300
> 
> > Hi Dave and Doug,
> > 
> > This series introduces some small updates and FPGA support to the
> mlx5
> > core/ethernet and IB drivers.
> > 
> > For more details please see below.
> > 
> > Please pull and let me know if there's any problem.
> 
> Ok, I've pulled this into net-next.
> 
> Doug let me know if there are any merge hassles we need to coordinate
> on.
> 
> Thanks.

Will do.  But is the question of a possible revert settled?  Because
once I pull, a revert means I have to pull all the way up to the revert
as well...

-- 
Doug Ledford 
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

Re: [pull request][for-next 0/6] Mellanox mlx5 updates 2017-05-23

[David Miller]

From: Saeed Mahameed 
Date: Tue, 23 May 2017 14:43:58 +0300

> Hi Dave and Doug,
> 
> This series introduces some small updates and FPGA support to the mlx5
> core/ethernet and IB drivers.
> 
> For more details please see below.
> 
> Please pull and let me know if there's any problem.

Ok, I've pulled this into net-next.

Doug let me know if there are any merge hassles we need to coordinate on.

Thanks.