Re: [HEADS UP] - OFED/RDMA stack update
iw_cxgbe still doesn't build in the branch because of some socket changes in head that are not in 11. I need to add the 11 equivalent of the dequeue_socket code to iw_cxgbe to get it to compile, and then try it to see if it actually works. Regards, Navdeep On 03/22/2018 04:36, Meny Yossefi wrote: > Hi Navdeep, > > What's the current iWARP status on the project branch? > If it's ready we'd like to merge it to FreeBSD11-STABLE immediately. > > -Meny > > > -Original Message- > From: Navdeep Parhar [mailto:npar...@gmail.com] On Behalf Of Navdeep Parhar > Sent: Tuesday, March 20, 2018 10:08 PM > To: Hans Petter Selasky ; Konstantin Belousov > ; 'freebsd-infinib...@freebsd.org' > ; freebsd-drivers > ; Meny Yossefi ; > 'FreeBSD-stable@FreeBSD.org' ; freebsd-arch > > Subject: Re: [HEADS UP] - OFED/RDMA stack update > > On 03/17/2018 13:03, Hans Petter Selasky wrote: >> On 03/17/18 20:52, Navdeep Parhar wrote: >>> Hold your horses. Do you have confirmation from the affected party >>> that the shims are adequate for them? I have been waiting for that >>> before looking at this branch. >> >> Hi Navdeep, >> >> Mellanox has received an API list from at least one party, and has >> taken the action to support all the required APIs. >> >>> Is the iw_cxgbe breakage a simple merge conflict as previously >>> discussed or do the shims require driver changes? >> >> It is a merge conflict. The code already compiles in 12-current. > > I tried backing out r329391 and r329017 in a local copy and then was able to > merge r320418, r323082, and r326169 in that order without any conflicts. But > iw_cxgbe/cm.c still doesn't compile in the projects branch because it has > some socket code that relies on some of glebius's changes available only in > head (I checked with him and they aren't MFC'able). I'm trying to figure out > what to do about those. > > And what about the cxgb breakage? Is there any simple way to make an old > style driver work with the new stack? T3 iw_cxgb in head was retired before > the ofed overhaul. > > Regards, > Navdeep > ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
RE: [HEADS UP] - OFED/RDMA stack update
Hi Navdeep, What's the current iWARP status on the project branch? If it's ready we'd like to merge it to FreeBSD11-STABLE immediately. -Meny -Original Message- From: Navdeep Parhar [mailto:npar...@gmail.com] On Behalf Of Navdeep Parhar Sent: Tuesday, March 20, 2018 10:08 PM To: Hans Petter Selasky ; Konstantin Belousov ; 'freebsd-infinib...@freebsd.org' ; freebsd-drivers ; Meny Yossefi ; 'FreeBSD-stable@FreeBSD.org' ; freebsd-arch Subject: Re: [HEADS UP] - OFED/RDMA stack update On 03/17/2018 13:03, Hans Petter Selasky wrote: > On 03/17/18 20:52, Navdeep Parhar wrote: >> Hold your horses. Do you have confirmation from the affected party >> that the shims are adequate for them? I have been waiting for that >> before looking at this branch. > > Hi Navdeep, > > Mellanox has received an API list from at least one party, and has > taken the action to support all the required APIs. > >> Is the iw_cxgbe breakage a simple merge conflict as previously >> discussed or do the shims require driver changes? > > It is a merge conflict. The code already compiles in 12-current. I tried backing out r329391 and r329017 in a local copy and then was able to merge r320418, r323082, and r326169 in that order without any conflicts. But iw_cxgbe/cm.c still doesn't compile in the projects branch because it has some socket code that relies on some of glebius's changes available only in head (I checked with him and they aren't MFC'able). I'm trying to figure out what to do about those. And what about the cxgb breakage? Is there any simple way to make an old style driver work with the new stack? T3 iw_cxgb in head was retired before the ofed overhaul. Regards, Navdeep ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
On 03/20/18 21:07, Navdeep Parhar wrote: And what about the cxgb breakage? Is there any simple way to make an old style driver work with the new stack? T3 iw_cxgb in head was retired before the ofed overhaul. Hi, The most simple way is to pull this driver from Linux 4.9, including libcxgb for user-space, if possible, though I'm not sure about the FreeBSD specific areas of iWarp and socket handling. I think you know that part better than I do. After pulling from Linux 4.9 you need to extend this change to cover the iw_cxgb: https://svnweb.freebsd.org/changeset/base/330508 Or can you retire iw_cxgb? --HPS ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
On 03/17/2018 13:03, Hans Petter Selasky wrote: > On 03/17/18 20:52, Navdeep Parhar wrote: >> Hold your horses. Do you have confirmation from the affected party that >> the shims are adequate for them? I have been waiting for that before >> looking at this branch. > > Hi Navdeep, > > Mellanox has received an API list from at least one party, and has taken > the action to support all the required APIs. > >> Is the iw_cxgbe breakage a simple merge conflict as previously discussed >> or do the shims require driver changes? > > It is a merge conflict. The code already compiles in 12-current. I tried backing out r329391 and r329017 in a local copy and then was able to merge r320418, r323082, and r326169 in that order without any conflicts. But iw_cxgbe/cm.c still doesn't compile in the projects branch because it has some socket code that relies on some of glebius's changes available only in head (I checked with him and they aren't MFC'able). I'm trying to figure out what to do about those. And what about the cxgb breakage? Is there any simple way to make an old style driver work with the new stack? T3 iw_cxgb in head was retired before the ofed overhaul. Regards, Navdeep ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
On Sat, Mar 17, 2018 at 09:03:40PM +0100, Hans Petter Selasky wrote: > On 03/17/18 20:52, Navdeep Parhar wrote: > >Hold your horses. Do you have confirmation from the affected party that > >the shims are adequate for them? I have been waiting for that before > >looking at this branch. > > Hi Navdeep, > > Mellanox has received an API list from at least one party, and has taken the > action to support all the required APIs. > > >Is the iw_cxgbe breakage a simple merge conflict as previously discussed > >or do the shims require driver changes? > > It is a merge conflict. The code already compiles in 12-current. Ok that doesn't sound that bad. I'll take a look early this week. Regards, Navdeep > > >If they don't then it should be > >possible to drop the iw_cxgbe from head into this branch and it should > >just work, is that correct? > > Yes, it shouldn't be too hard to do and I would appreciate if Chelsio could > throw some resources at this ASAP. I believe the new code will benefit > everyone using Infiniband/RoCE and iWarp with FreeBSD. > > --HPS > ___ > freebsd-a...@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
On 03/17/18 20:52, Navdeep Parhar wrote: Hold your horses. Do you have confirmation from the affected party that the shims are adequate for them? I have been waiting for that before looking at this branch. Hi Navdeep, Mellanox has received an API list from at least one party, and has taken the action to support all the required APIs. Is the iw_cxgbe breakage a simple merge conflict as previously discussed or do the shims require driver changes? It is a merge conflict. The code already compiles in 12-current. If they don't then it should be possible to drop the iw_cxgbe from head into this branch and it should just work, is that correct? Yes, it shouldn't be too hard to do and I would appreciate if Chelsio could throw some resources at this ASAP. I believe the new code will benefit everyone using Infiniband/RoCE and iWarp with FreeBSD. --HPS ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
Hold your horses. Do you have confirmation from the affected party that
the shims are adequate for them? I have been waiting for that before
looking at this branch.
Is the iw_cxgbe breakage a simple merge conflict as previously discussed
or do the shims require driver changes? If they don't then it should be
possible to drop the iw_cxgbe from head into this branch and it should
just work, is that correct?
Regards,
Navdeep
On Fri, Mar 16, 2018 at 05:44:41PM +0100, Hans Petter Selasky wrote:
> Hi,
>
> The bsd_rdma_4_9_stable_11 projects branch is close to being merged into
> FreeBSD 11-stable. Mellanox plans to merge no later than 12:00 CEST TUE 20th
> of March 2018, unless objections are received.
>
> A compatibility header file has been created, ib_verbs_compat.h, which
> offers full source compatibility to existing OFED kernel applications, as a
> response to the raised conserns. User-space compatibility is maintained
> through library symbol versioning.
>
> https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/ofed/include/rdma/ib_verbs_compat.h
>
> An example client for this header file can be found here:
>
> https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/contrib/rdma/krping_compat/
>
> Currently the cxgb and cxgbe i-Warp drivers are not building, and will be
> stubbed from the kernel build before the branch is merged, unless Chelsio
> can add patches for these.
>
> Here is a quick and dirty patch to make the bsd_rdma_4_9_stable_11 branch
> build:
>
> >diff --git a/sys/modules/Makefile b/sys/modules/Makefile
> >index 6b005c854d7..b918a208f21 100644
> >--- a/sys/modules/Makefile
> >+++ b/sys/modules/Makefile
> >@@ -530,7 +530,7 @@ _txp= txp
> > .if ${MK_SOURCELESS_UCODE} != "no" && ${MACHINE_CPUARCH} != "arm" && \
> >${MACHINE_ARCH:C/mips(el)?/mips/} != "mips" && \
> >${MACHINE_ARCH} != "powerpc" && ${MACHINE_CPUARCH} != "riscv"
> >-_cxgbe=cxgbe
> >+#_cxgbe= cxgbe
> > .endif
> > .if ${MK_TESTS} != "no" || defined(ALL_MODULES)
> >@@ -554,7 +554,7 @@ _vpo= vpo
> > _sym= sym
> > # intr_disable() is a macro, causes problems
> > .if ${MK_SOURCELESS_UCODE} != "no"
> >-_cxgb= cxgb
> >+#_cxgb=cxgb
> > .endif
> > .endif
>
> --HPS
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
Hi,
The bsd_rdma_4_9_stable_11 projects branch is close to being merged into
FreeBSD 11-stable. Mellanox plans to merge no later than 12:00 CEST TUE
20th of March 2018, unless objections are received.
A compatibility header file has been created, ib_verbs_compat.h, which
offers full source compatibility to existing OFED kernel applications,
as a response to the raised conserns. User-space compatibility is
maintained through library symbol versioning.
https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/ofed/include/rdma/ib_verbs_compat.h
An example client for this header file can be found here:
https://svnweb.freebsd.org/base/projects/bsd_rdma_4_9_stable_11/sys/contrib/rdma/krping_compat/
Currently the cxgb and cxgbe i-Warp drivers are not building, and will
be stubbed from the kernel build before the branch is merged, unless
Chelsio can add patches for these.
Here is a quick and dirty patch to make the bsd_rdma_4_9_stable_11
branch build:
diff --git a/sys/modules/Makefile b/sys/modules/Makefile
index 6b005c854d7..b918a208f21 100644
--- a/sys/modules/Makefile
+++ b/sys/modules/Makefile
@@ -530,7 +530,7 @@ _txp= txp
.if ${MK_SOURCELESS_UCODE} != "no" && ${MACHINE_CPUARCH} != "arm" && \
${MACHINE_ARCH:C/mips(el)?/mips/} != "mips" && \
${MACHINE_ARCH} != "powerpc" && ${MACHINE_CPUARCH} != "riscv"
-_cxgbe=cxgbe
+#_cxgbe= cxgbe
.endif
.if ${MK_TESTS} != "no" || defined(ALL_MODULES)
@@ -554,7 +554,7 @@ _vpo= vpo
_sym= sym
# intr_disable() is a macro, causes problems
.if ${MK_SOURCELESS_UCODE} != "no"
-_cxgb= cxgb
+#_cxgb=cxgb
.endif
.endif
--HPS
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
On Mon, Feb 26, 2018 at 02:21:39PM -0800, Navdeep Parhar wrote: > +freebsd-arch@ > > Hi Meny, > > Can you please post the KPI/KBI analysis that you generated to some > public location and provide a link here? A straight MFC would be a > major break of KPI/KBI in -STABLE and the options we're looking at are: I put the report at https://kib.kiev.ua/kib/ibcore_11_to_11_merged_compat_report.html > > a) Ignore the breakage and let downstream consumers deal with the > fallout. This obviously isn't ideal in a -STABLE branch. > > b) Provide compat shims to at least preserve the KPI. One challenge is > that the changes include functions with the same name but different > signature/behavior. See, for example, ib_create_cq in Meny's list once > he publishes it. Project did handled similar issues already. One of the approaches is to renname the ib_create_cq with the new signature to ib_create_cq_n12 and check for (e.g.) _WANT_NEW_OFED symbol and to select one or another: #ifdef _WANT_NEW_OFED #define ib_create_cq(new args there) ib_create_cq_n21(new args there) #else #define ib_create_cq (ib_create_cq) #endif Then ULP that wants new KPI defines _WANT_NEW_OFED. > > c) Have two versions of the OFED interfaces in 11-STABLE and not break > existing downstream consumers at all. It is possible to make them loadable simultaneously as modules, but it is quite confusing to users, because Mellanox clearly wants mlx5_ib and mlx4_ib to work only with new OFED, while cxgbe would use old OFED ? Also, either we would need to mess with the ibcore.ko module name, or with version. I am not sure that our module handling is robust enough to make the version trick possible. > > I've reached out to users that I know of and know will be affected. > If you use OFED and FreeBSD 11 this would be a good time to weigh > in with your thoughts, ideas, concerns etc.. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: [HEADS UP] - OFED/RDMA stack update
+freebsd-arch@ Hi Meny, Can you please post the KPI/KBI analysis that you generated to some public location and provide a link here? A straight MFC would be a major break of KPI/KBI in -STABLE and the options we're looking at are: a) Ignore the breakage and let downstream consumers deal with the fallout. This obviously isn't ideal in a -STABLE branch. b) Provide compat shims to at least preserve the KPI. One challenge is that the changes include functions with the same name but different signature/behavior. See, for example, ib_create_cq in Meny's list once he publishes it. c) Have two versions of the OFED interfaces in 11-STABLE and not break existing downstream consumers at all. I've reached out to users that I know of and know will be affected. If you use OFED and FreeBSD 11 this would be a good time to weigh in with your thoughts, ideas, concerns etc.. Regards, Navdeep On Mon, 2018-02-26 at 14:49 +, Meny Yossefi wrote: > Hi, > > We are currently working on MFC'ing the OFED/RDMA stack update > mentioned below to FreeBSD11-STABLE. > > We already have working version in 'projects/bsd_rdma_4_9_stable_11' > (pending iWARP updates) and currently working on ULP integration. > > Again, as always, for any concern/comments you might have, please > don't hesitate contacting us. > > freebsd-driv...@mellanox.com > > > Regards, > > Meny Yossefi, > Mellanox technologies > > > -Original Message- > From: Meny Yossefi > Sent: Monday, November 13, 2017 11:09 AM > To: 'freebsd-infinib...@freebsd.org' ; > 'freebsd-curr...@freebsd.org' > Cc: freebsd-drivers > Subject: [HEADS UP] - OFED/RDMA stack update > > Hi, > > This is to inform you that by end of this week we plan to merge the > OFED/RDMA stack update from the project - 'bsd_rdma_4_9' into 12- > CURRENT. > The update aligns the OFED common code and RDMA vendor drivers with > Linux v4.9. > > We are still working on final modifications and build testing it prior > to submission. > > > For any concern/comments you might have, please don't hesitate > contacting us. > > freebsd-driv...@mellanox.com > > > Regards, > > Meny Yossefi, > Mellanox technologies > ___ > freebsd-infinib...@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freeb > sd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
RE: [HEADS UP] - OFED/RDMA stack update
Hi, We are currently working on MFC'ing the OFED/RDMA stack update mentioned below to FreeBSD11-STABLE. We already have working version in 'projects/bsd_rdma_4_9_stable_11' (pending iWARP updates) and currently working on ULP integration. Again, as always, for any concern/comments you might have, please don't hesitate contacting us. freebsd-driv...@mellanox.com Regards, Meny Yossefi, Mellanox technologies -Original Message- From: Meny Yossefi Sent: Monday, November 13, 2017 11:09 AM To: 'freebsd-infinib...@freebsd.org' ; 'freebsd-curr...@freebsd.org' Cc: freebsd-drivers Subject: [HEADS UP] - OFED/RDMA stack update Hi, This is to inform you that by end of this week we plan to merge the OFED/RDMA stack update from the project - 'bsd_rdma_4_9' into 12-CURRENT. The update aligns the OFED common code and RDMA vendor drivers with Linux v4.9. We are still working on final modifications and build testing it prior to submission. For any concern/comments you might have, please don't hesitate contacting us. freebsd-driv...@mellanox.com Regards, Meny Yossefi, Mellanox technologies ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
