-----Original Message-----
From: Bruce Richardson [mailto:bruce.richard...@intel.com]
Sent: Wednesday, November 14, 2018 3:48 AM
To: Burdick, Cliff
Cc: Thomas Monjalon; Burakov, Anatoly; dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH 1/1] eal: Don't fail secondary if primary is
missing tailqs
On Tue, Nov 13, 2018 at 11:42:51PM +0000, Burdick, Cliff wrote:
> >
> > ________________________________________
> > From: Thomas Monjalon [tho...@monjalon.net]
> > Sent: Tuesday, November 13, 2018 2:18 PM
> > To: Burdick, Cliff
> > Cc: Burakov, Anatoly; dev@dpdk.org; bruce.richard...@intel.com
> > Subject: Re: [dpdk-dev] [PATCH 1/1] eal: Don't fail secondary if
> > primary is missing tailqs
> >
> > 13/11/2018 23:08, Burdick, Cliff:
> > > From: Thomas Monjalon [mailto:tho...@monjalon.net]
> > > > 13/11/2018 17:38, Burdick, Cliff:
> > > > > From: Thomas Monjalon [mailto:tho...@monjalon.net]
> > > > > 13/11/2018 16:45, Burdick, Cliff:
> > > > > > From: Burakov, Anatoly [mailto:anatoly.bura...@intel.com]
> > > > > > > On 13-Nov-18 9:21 AM, Thomas Monjalon wrote:
> > > > > > > > 13/11/2018 00:33, Burdick, Cliff:
> > > > > > > >> This patch was submitted by Jean Tourrilhes over two
> > > > > > > >> years ago, but didn't receive any responses. I hit the
> > > > > > > >> same issue recently when trying to use cgo (Golang) as a
> > > > > > > >> primary process linked to libdpdk.a against a C++
> > > > > > > >> application linked against the same library.> > >
> > > > > > > >
> > > > > > > > The question is to know why you don't have the same
> > > > > > > > constructors in primary and seconday?
> > > > > > >
> > > > > > > I've hit similar things in the past. I believe it was caused
> > > > > > > by our build system stripping out unused libraries (such as
> > > > > > > rte_hash) from the binary and thus not calling the
> > > > > > > constructor in the primary, but doing so in the secondary
> > > > > > > (or something to that effect). In any case, this is caused
> > > > > > > by linking different number of libraries to primary and
> > > > > > > secondary, and should probably be fixed in the build system,
> > > > > > > not in the tailqs code (unless we specifically support
> > > > > > > having different linked libraries to primary and
> > > > > > > secondary?).> > > >
> > > > > > Right, I think the original author of the patch stated the
> > > > > > reasons in the link I provided. The build system seems like
> > > > > > the most appropriate place to fix it, but the patch got me
> > > > > > going quickly. I think the question is whether you want DPDK
> > > > > > to support these other ways of linking. I'm certainly not the
> > > > > > first to use cgo, since there's a virtual switch project doing the
> > > > > > same:
> > > > > >
> > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.co
> > > > > > m_lago
> > > > > > pu
> > > > > > s_vsw&d=DwICAg&c=jcv3orpCsv7C4ly8-ubDoUfrxF5xIGWmptxGWP5vi5w&r
> > > > > > =m1RLQ
> > > > > > OG
> > > > > > Okz9MauvVLZmiGtyWc5mA7DejbPFNE1IDj-4&m=hQqVCNwW7eoEzB_hLFK97i8
> > > > > > idS8FI qX
> > > > > > oPeclwsIZq7Y&s=BMoBlwkqljwWIBY3SE3pPMCfVnOUlDuZLrno4-SojKM&e=
> > > > > >
> > > > > > They don't use primary/secondary processes, though, so the
> > > > > >issue is never hit. I'm in a situation where using cgo seemed
> > > > > >like the easiest path to accomplish what I'm doing since I
> > > > > >needed specialized libraries for it that were not available in
> > > > > >C/C++. At some point I bet someone would use Cython to start
> > > > > >linking against DPDK as well, and we'd likely run into the
> > > > > >same issue.> > > > For sure, we want to support using DPDK with cgo
> > > > > >or cython.
> > > > > >But it is not clear what is the relation with not having the
> > > > > >same compilation for primary and secondary. Please could you
> > > > > >elaborate?> > >
> > > > > Hi Thomas, I think Jean explained it well here:
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__dev.dpdk.narkive.
> > > > > com_ZM3a7QD1_dpdk-2Ddev-2Dbug-2Dstatic-2Dconstructors-2Dconsider
> > > > > ed-2De
> > > > > vil&d=DwICAg&c=jcv3orpCsv7C4ly8-ubDoUfrxF5xIGWmptxGWP5vi5w&r=m1R
> > > > > LQOGOk
> > > > > z9MauvVLZmiGtyWc5mA7DejbPFNE1IDj-4&m=C69wDgrjDX8_oXp1M_3bnmWOOZd
> > > > > GqwBBG
> > > > > 9vzkyGDWGQ&s=YYn2N7WrzJpH1ptNrLZad0nPAQdrUyqBckk2uFuWYPQ&e=
> > > > >
> > > > > "The build system of the application does not have all the
> > > > > subtelties of the DPDK build system, and ends up including *all*
> > > > > the constructors, wether they are used or not in the code.
> > > > > Moreover, they are included in a different order. Actually, by
> > > > > default the builds include no constructors at all (which is a
> > > > > big fail), so the library needs to be included with
> > > > > --whole-archive (see Snort DPDK instructions)."
> > > > >
> > > > > I will get to the bottom of my exact case to understand what's
> > > > > happening, but my primary application is a cgo application that
> > > > > I'm linking to by using almost exactly the same flags that are
> > > > > used in the DPDK build system to build examples. The DPDK
> > > > > libraries I'm linking against is a single location for both
> > > > > primary and secondary; in other words, I don't build DPDK twice.
> > > > >
> > > > > OK I understand, thanks.
> > > > >
> > > > > You had alluded to a pkg-config for DPDK in the 2015 thread,
> > > > > which cgo can use, but I don't know if that ever was
> > > > > implemented. Cgo can use pkg-config if it's available, otherwise
> > > > > the only tools are specifying LDFLAGS and CFLAGS into their build
> > > > > system.
> > > >Yes, the right answer is still pkg-config :) The good news is that it is
> > > >now implemented thanks to the meson build system:
> > > >
> > > >https://urldefense.proofpoint.com/v2/url?u=http-3A__git.dpdk.org_dp
> > > >dk_tree_doc_build-2Dsdk-2Dmeson.txt-23n182&d=DwICAg&c=jcv3orpCsv7C4
> > > >ly8-ubDoUfrxF5xIGWmptxGWP5vi5w&r=m1RLQOGOkz9MauvVLZmiGtyWc5mA7DejbP
> > > >FNE1IDj->4&m=C69wDgrjDX8_oXp1M_3bnmWOOZdGqwBBG9vzkyGDWGQ&s=oC86KD_R
> > > >J1T6rfzi3x5zFT1Ri13ELpKmsyFqpgDbgFg&e=
> > >
> > > Hi Thomas, are there instructions on how to use pkg-config to build the
> > > mlx4/5 PMD? I noticed a patch was submitted in September to add support
> > > for it, but that link you provided on using meson doesn't say how to
> > > build specific drivers. It appears to be disabled by default.
> >
> > > If the dependency is found, meson will enable the PMD when building DPDK.
> >
> > Do you know where exactly that is? I've been using mlx5 for a while on this
> > system, and I can see that 18.11-rc2 meson build+ninja built the pmd, but
> > it's not on the --libs listing for pkg-config. Does it tell me what I was
> > missing?
> >
> For dynamic linking of applications, the drivers are not normally linked in.
> Instead, they should be loaded from the drivers directory as .so files
> - normally by default in EAL as the driver .so's should be copied to the
> EAL_PMD_PATH where EAL finds them automatically. [This applies to both meson
> and make builds, though only meson generates the .pc file for pkg-config]
>
> If you are doing a static build, then you need to explicitly link in the
> drivers. You can get a list from pkg-config using the "--static" flag in this
> case. A good example of how to use pkg-config in this way can be found in the
> Makefiles for most examples, e.g. examples/helloworld/Makefile, reproduced
> below.
>
> Regards,
> /Bruce
>
> all: shared
> .PHONY: shared static
> shared: build/$(APP)-shared
> ln -sf $(APP)-shared build/$(APP)
> static: build/$(APP)-static
> ln -sf $(APP)-static build/$(APP)
>
> PC_FILE := $(shell pkg-config --path libdpdk) CFLAGS += -O3 $(shell
> pkg-config --cflags libdpdk) LDFLAGS_SHARED = $(shell pkg-config --libs
> libdpdk) LDFLAGS_STATIC = -Wl,-Bstatic $(shell pkg-config --static --libs
> libdpdk)
>
> build/$(APP)-shared: $(SRCS-y) Makefile $(PC_FILE) | build
> $(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_SHARED)
>
> build/$(APP)-static: $(SRCS-y) Makefile $(PC_FILE) | build
> $(CC) $(CFLAGS) $(SRCS-y) -o $@ $(LDFLAGS) $(LDFLAGS_STATIC)
>
> build:
> @mkdir -p $@
Thanks Bruce. I tried getting this to compile using cgo, and it's causing more
pain than it's worth. I used the --static --libs options, and there were still
numerous linker errors, some specific to mlx, and some not. Specifically
libmlx5 and libmnl are both needed, but they're not part of the linker line
from pkg-config. At this point I'll just break the Go application up into a
separate C application that handles all the DPDK parts, and send messages
between them.
