Re: [OE-core] Issues with meson in SDK with cross-file
On 02/06/2018 06:28 PM, Joshua Watt wrote: On Tue, 2018-02-06 at 15:39 -0800, Martin Kelly wrote: (ping) Paul, what do you about the options we have here for making meson work properly in the SDK? To recap, here are the available options. I'm wondering if you could give your opinion on the best fit for how OE SDKs work: - Generate meson.cross toolchain file at SDK extraction time. - Wrap meson with a shell script that dynamically generates a toolchain FWIW, I *just* pushed up a patch to add support for this to so that I could have Icecream distributed compiling support in the SDK. I called them post-relocation scripts, see http://lists.openembedded.org/piperma il/openembedded-core/2018-February/147282.html This patch show how you install a post-relocation script from a recipe: http://lists.openembedded.org/pipermail/openembedded-core/2018-February /147283.html (look at icecc-setup.sh). It's pretty easy and modeled after the environment setup scripts. The issue that meson is having sounds similar to what I encountered with Icecream: something needed in the SDK can only really be generated by host system when the SDK is installed. There isn't enough information or the ability to generate it beforehand when the SDK is constructed by Yocto. Granted, this patch has not yet gone through the rigors of code review so it will likely change, but I think that it is safe to say that such a feature would be useful for more that just meson. I think this is exactly what we need, thanks so much! When your patches get merged, I'll try this out with meson. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On Tue, 2018-02-06 at 15:39 -0800, Martin Kelly wrote: > (ping) > > Paul, what do you about the options we have here for making meson > work > properly in the SDK? > > To recap, here are the available options. I'm wondering if you could > give your opinion on the best fit for how OE SDKs work: > > - Generate meson.cross toolchain file at SDK extraction time. > - Wrap meson with a shell script that dynamically generates a > toolchain FWIW, I *just* pushed up a patch to add support for this to so that I could have Icecream distributed compiling support in the SDK. I called them post-relocation scripts, see http://lists.openembedded.org/piperma il/openembedded-core/2018-February/147282.html This patch show how you install a post-relocation script from a recipe: http://lists.openembedded.org/pipermail/openembedded-core/2018-February /147283.html (look at icecc-setup.sh). It's pretty easy and modeled after the environment setup scripts. The issue that meson is having sounds similar to what I encountered with Icecream: something needed in the SDK can only really be generated by host system when the SDK is installed. There isn't enough information or the ability to generate it beforehand when the SDK is constructed by Yocto. Granted, this patch has not yet gone through the rigors of code review so it will likely change, but I think that it is safe to say that such a feature would be useful for more that just meson. > file and then runs meson pointing to it. > - Change meson to support pulling in env vars in meson.cross, and use > a > fixed meson.cross file that references the env vars. > > On 01/17/2018 09:47 AM, Martin Kelly wrote: > > Paul, any opinion? > > > > On 01/12/2018 04:35 AM, Alexander Kanavin wrote: > > > On 01/11/2018 09:22 PM, Martin Kelly wrote: > > > > Khem and Alexander, could you comment on which solution is > > > > preferable > > > > from an SDK standpoint? Otherwise, could you nominate someone > > > > else to > > > > do so in your place? :) > > > > > > I'm not sure who is the resident SDK expert, perhaps Paul > > > Eggleton? > > > > > > Alex -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
(ping) Paul, what do you about the options we have here for making meson work properly in the SDK? To recap, here are the available options. I'm wondering if you could give your opinion on the best fit for how OE SDKs work: - Generate meson.cross toolchain file at SDK extraction time. - Wrap meson with a shell script that dynamically generates a toolchain file and then runs meson pointing to it. - Change meson to support pulling in env vars in meson.cross, and use a fixed meson.cross file that references the env vars. On 01/17/2018 09:47 AM, Martin Kelly wrote: Paul, any opinion? On 01/12/2018 04:35 AM, Alexander Kanavin wrote: On 01/11/2018 09:22 PM, Martin Kelly wrote: Khem and Alexander, could you comment on which solution is preferable from an SDK standpoint? Otherwise, could you nominate someone else to do so in your place? :) I'm not sure who is the resident SDK expert, perhaps Paul Eggleton? Alex -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
Paul, any opinion? On 01/12/2018 04:35 AM, Alexander Kanavin wrote: On 01/11/2018 09:22 PM, Martin Kelly wrote: Khem and Alexander, could you comment on which solution is preferable from an SDK standpoint? Otherwise, could you nominate someone else to do so in your place? :) I'm not sure who is the resident SDK expert, perhaps Paul Eggleton? Alex -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On 01/11/2018 09:22 PM, Martin Kelly wrote: Khem and Alexander, could you comment on which solution is preferable from an SDK standpoint? Otherwise, could you nominate someone else to do so in your place? :) I'm not sure who is the resident SDK expert, perhaps Paul Eggleton? Alex -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On 01/11/2018 11:33 AM, Martin Kelly wrote: On 01/11/2018 11:26 AM, Khem Raj wrote: On Thu, Jan 11, 2018 at 11:22 AM, Martin Kellywrote: Khem and Alexander, could you comment on which solution is preferable from an SDK standpoint? Otherwise, could you nominate someone else to do so in your place? :) Here are the possible solutions proposed: - Generate meson.cross toolchain file at SDK extraction time. - Wrap meson with a shell script that dynamically generates a toolchain file and then runs meson pointing to it. - Change meson to support pulling in env vars in meson.cross, and use a fixed meson.cross file that references the env vars. We already have environment file, could it be used for meson as well and needed bits be generated during build time for SDK. Yes, it is certainly technically possible. I'm wondering whether or not that involves ugly special-casing, as I'm not familiar with the SDK extraction code. If not, it's probably the best solution. Specifically, looking at script meta/files/toolchain-shar-extract.sh, I don't see a clean way to do something special for a single meson package. There is no per-package hook or similar logic. On 01/09/2018 12:33 PM, Martin Kelly wrote: On 01/09/2018 10:40 AM, Jussi Pakkanen wrote: On Tue, Jan 9, 2018 at 8:20 PM, Martin Kelly wrote: Note the "native C compiler" line, which directly uses $CC. I'm not sure if this is correct, but an easy way to fix the issue is to ignore $CC for internal sanity checking when a cross file is specified. In that way, meson would probe the system and use the normal gcc for sanity checking while still using the cross file for the actual build. That breaks the whole reason the sanity check is there in the first place. Its point is to test "is the native compiler the user has specified working and capable of creating executables". If we change it then that becomes "is the system default compiler (which we might or might not use) working". We need to be able to support the case of users defining both the cross compiler and the native compiler. So something like this: CC=/some/native/cc meson --cross-file=mycross.txt Yeah, that makes sense. The issue here is that the OE SDK sets CC, CXX etc to point to the cross compiler, not to the native compiler, so when meson assumes it's native, things will break. I think we need a way to specify both cross and host compilers separately from the env vars. For example, if the binaries section were split in two: "host-binaries" and "target-binaries", then in the cross-file case, meson could use "host-binaries" instead of looking at CC and other vars. Right now, the SDK contains fixed contents, and there is some top-level logic for rewriting a few paths to make everything relocatable. I don't think OE wants to inject a special-case one-time generation of the toolchain file at SDK extraction time, as it circumvents the normal build process, Would it not be possible to generate the cross file when creating the SDK contents originally? The only change it would need is the same kind of path fixing. The setup does not really change. I think it would be possible, but I'm guessing it would require some special-casing that we may not want to do. Khem probably has a better informed opinion on this than I. Another approach would be to generate the toolchain file truly "on-the-fly", such that the meson command points to a script that first generates a toolchain file based on the contents of CC, CXX, etc. and then runs meson. I think this is a bad idea because it is complex (will definitely surprise people) and slow. It also breaks people in surprising ways when they accidentally use a meson from outside the SDK due to the PATH setup. This is, roughly, what Debian does currently. That's too bad :). -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On 01/11/2018 11:26 AM, Khem Raj wrote: On Thu, Jan 11, 2018 at 11:22 AM, Martin Kellywrote: Khem and Alexander, could you comment on which solution is preferable from an SDK standpoint? Otherwise, could you nominate someone else to do so in your place? :) Here are the possible solutions proposed: - Generate meson.cross toolchain file at SDK extraction time. - Wrap meson with a shell script that dynamically generates a toolchain file and then runs meson pointing to it. - Change meson to support pulling in env vars in meson.cross, and use a fixed meson.cross file that references the env vars. We already have environment file, could it be used for meson as well and needed bits be generated during build time for SDK. Yes, it is certainly technically possible. I'm wondering whether or not that involves ugly special-casing, as I'm not familiar with the SDK extraction code. If not, it's probably the best solution. On 01/09/2018 12:33 PM, Martin Kelly wrote: On 01/09/2018 10:40 AM, Jussi Pakkanen wrote: On Tue, Jan 9, 2018 at 8:20 PM, Martin Kelly wrote: Note the "native C compiler" line, which directly uses $CC. I'm not sure if this is correct, but an easy way to fix the issue is to ignore $CC for internal sanity checking when a cross file is specified. In that way, meson would probe the system and use the normal gcc for sanity checking while still using the cross file for the actual build. That breaks the whole reason the sanity check is there in the first place. Its point is to test "is the native compiler the user has specified working and capable of creating executables". If we change it then that becomes "is the system default compiler (which we might or might not use) working". We need to be able to support the case of users defining both the cross compiler and the native compiler. So something like this: CC=/some/native/cc meson --cross-file=mycross.txt Yeah, that makes sense. The issue here is that the OE SDK sets CC, CXX etc to point to the cross compiler, not to the native compiler, so when meson assumes it's native, things will break. I think we need a way to specify both cross and host compilers separately from the env vars. For example, if the binaries section were split in two: "host-binaries" and "target-binaries", then in the cross-file case, meson could use "host-binaries" instead of looking at CC and other vars. Right now, the SDK contains fixed contents, and there is some top-level logic for rewriting a few paths to make everything relocatable. I don't think OE wants to inject a special-case one-time generation of the toolchain file at SDK extraction time, as it circumvents the normal build process, Would it not be possible to generate the cross file when creating the SDK contents originally? The only change it would need is the same kind of path fixing. The setup does not really change. I think it would be possible, but I'm guessing it would require some special-casing that we may not want to do. Khem probably has a better informed opinion on this than I. Another approach would be to generate the toolchain file truly "on-the-fly", such that the meson command points to a script that first generates a toolchain file based on the contents of CC, CXX, etc. and then runs meson. I think this is a bad idea because it is complex (will definitely surprise people) and slow. It also breaks people in surprising ways when they accidentally use a meson from outside the SDK due to the PATH setup. This is, roughly, what Debian does currently. That's too bad :). -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On Thu, Jan 11, 2018 at 11:22 AM, Martin Kellywrote: > Khem and Alexander, could you comment on which solution is preferable from > an SDK standpoint? Otherwise, could you nominate someone else to do so in > your place? :) > > Here are the possible solutions proposed: > > - Generate meson.cross toolchain file at SDK extraction time. > - Wrap meson with a shell script that dynamically generates a toolchain file > and then runs meson pointing to it. > - Change meson to support pulling in env vars in meson.cross, and use a > fixed meson.cross file that references the env vars. > We already have environment file, could it be used for meson as well and needed bits be generated during build time for SDK. > > On 01/09/2018 12:33 PM, Martin Kelly wrote: >> >> On 01/09/2018 10:40 AM, Jussi Pakkanen wrote: >>> >>> On Tue, Jan 9, 2018 at 8:20 PM, Martin Kelly wrote: >>> Note the "native C compiler" line, which directly uses $CC. I'm not sure if this is correct, but an easy way to fix the issue is to ignore $CC for internal sanity checking when a cross file is specified. In that way, meson would probe the system and use the normal gcc for sanity checking while still using the cross file for the actual build. >>> >>> >>> That breaks the whole reason the sanity check is there in the first >>> place. Its point is to test "is the native compiler the user has >>> specified working and capable of creating executables". If we change >>> it then that becomes "is the system default compiler (which we might >>> or might not use) working". We need to be able to support the case of >>> users defining both the cross compiler and the native compiler. So >>> something like this: >>> >>> CC=/some/native/cc meson --cross-file=mycross.txt >>> >> >> Yeah, that makes sense. The issue here is that the OE SDK sets CC, CXX etc >> to point to the cross compiler, not to the native compiler, so when meson >> assumes it's native, things will break. I think we need a way to specify >> both cross and host compilers separately from the env vars. For example, if >> the binaries section were split in two: "host-binaries" and >> "target-binaries", then in the cross-file case, meson could use >> "host-binaries" instead of looking at CC and other vars. >> Right now, the SDK contains fixed contents, and there is some top-level logic for rewriting a few paths to make everything relocatable. I don't think OE wants to inject a special-case one-time generation of the toolchain file at SDK extraction time, as it circumvents the normal build process, >>> >>> >>> Would it not be possible to generate the cross file when creating the >>> SDK contents originally? The only change it would need is the same >>> kind of path fixing. The setup does not really change. >>> >> >> I think it would be possible, but I'm guessing it would require some >> special-casing that we may not want to do. Khem probably has a better >> informed opinion on this than I. >> Another approach would be to generate the toolchain file truly "on-the-fly", such that the meson command points to a script that first generates a toolchain file based on the contents of CC, CXX, etc. and then runs meson. I think this is a bad idea because it is complex (will definitely surprise people) and slow. It also breaks people in surprising ways when they accidentally use a meson from outside the SDK due to the PATH setup. >>> >>> >>> This is, roughly, what Debian does currently. >>> >> >> That's too bad :). -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
Khem and Alexander, could you comment on which solution is preferable from an SDK standpoint? Otherwise, could you nominate someone else to do so in your place? :) Here are the possible solutions proposed: - Generate meson.cross toolchain file at SDK extraction time. - Wrap meson with a shell script that dynamically generates a toolchain file and then runs meson pointing to it. - Change meson to support pulling in env vars in meson.cross, and use a fixed meson.cross file that references the env vars. On 01/09/2018 12:33 PM, Martin Kelly wrote: On 01/09/2018 10:40 AM, Jussi Pakkanen wrote: On Tue, Jan 9, 2018 at 8:20 PM, Martin Kellywrote: Note the "native C compiler" line, which directly uses $CC. I'm not sure if this is correct, but an easy way to fix the issue is to ignore $CC for internal sanity checking when a cross file is specified. In that way, meson would probe the system and use the normal gcc for sanity checking while still using the cross file for the actual build. That breaks the whole reason the sanity check is there in the first place. Its point is to test "is the native compiler the user has specified working and capable of creating executables". If we change it then that becomes "is the system default compiler (which we might or might not use) working". We need to be able to support the case of users defining both the cross compiler and the native compiler. So something like this: CC=/some/native/cc meson --cross-file=mycross.txt Yeah, that makes sense. The issue here is that the OE SDK sets CC, CXX etc to point to the cross compiler, not to the native compiler, so when meson assumes it's native, things will break. I think we need a way to specify both cross and host compilers separately from the env vars. For example, if the binaries section were split in two: "host-binaries" and "target-binaries", then in the cross-file case, meson could use "host-binaries" instead of looking at CC and other vars. Right now, the SDK contains fixed contents, and there is some top-level logic for rewriting a few paths to make everything relocatable. I don't think OE wants to inject a special-case one-time generation of the toolchain file at SDK extraction time, as it circumvents the normal build process, Would it not be possible to generate the cross file when creating the SDK contents originally? The only change it would need is the same kind of path fixing. The setup does not really change. I think it would be possible, but I'm guessing it would require some special-casing that we may not want to do. Khem probably has a better informed opinion on this than I. Another approach would be to generate the toolchain file truly "on-the-fly", such that the meson command points to a script that first generates a toolchain file based on the contents of CC, CXX, etc. and then runs meson. I think this is a bad idea because it is complex (will definitely surprise people) and slow. It also breaks people in surprising ways when they accidentally use a meson from outside the SDK due to the PATH setup. This is, roughly, what Debian does currently. That's too bad :). -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On 01/09/2018 11:50 AM, Nirbheek Chauhan wrote: Recently, we got meson added to OE-core as part of Yocto (thanks Alexander Kanavin!). Great to hear that! Thanks for your work, Alexander :) I've built on the meta-oe meson recipe created by other people (tried to credit them in the commit). My contribution was mostly polish and testing, so that the recipe is of sufficient quality to be included in oe-core - particularly that it doesn't break anything on the autobuilder across architectures and configurations, and switching recipes from autotools to meson does not come with feature regressions (e.g. gobject-introspection, which is notoriously tricky for us). - When --cross-file is specified, make meson not assume that env vars like CC and CXX and similar point to native tools. Instead, just ignore them and look for native tools in the normal way that meson does when these vars are unspecified. We still may wan to inject CFLAGS, LDFLAGS, etc into the build because people may mess with them interactively during development. Would it be possible to tell Yocto to not set those vars when building a recipe that uses meson? FWIW, I fully agree. Environment variables are an obsolete technique from old Unices, and a notorious source of frustration - because it's very easy for software to quietly modify them behind the human's back, as opposed to setting things via config files and command line switches, which are both a lot more controlled and disciplined. Ideally, I want to reduce or eliminate the use of env vars altogether in Yocto (long way to go there, I know). Alex -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On 01/09/2018 10:40 AM, Jussi Pakkanen wrote: On Tue, Jan 9, 2018 at 8:20 PM, Martin Kellywrote: Note the "native C compiler" line, which directly uses $CC. I'm not sure if this is correct, but an easy way to fix the issue is to ignore $CC for internal sanity checking when a cross file is specified. In that way, meson would probe the system and use the normal gcc for sanity checking while still using the cross file for the actual build. That breaks the whole reason the sanity check is there in the first place. Its point is to test "is the native compiler the user has specified working and capable of creating executables". If we change it then that becomes "is the system default compiler (which we might or might not use) working". We need to be able to support the case of users defining both the cross compiler and the native compiler. So something like this: CC=/some/native/cc meson --cross-file=mycross.txt Yeah, that makes sense. The issue here is that the OE SDK sets CC, CXX etc to point to the cross compiler, not to the native compiler, so when meson assumes it's native, things will break. I think we need a way to specify both cross and host compilers separately from the env vars. For example, if the binaries section were split in two: "host-binaries" and "target-binaries", then in the cross-file case, meson could use "host-binaries" instead of looking at CC and other vars. Right now, the SDK contains fixed contents, and there is some top-level logic for rewriting a few paths to make everything relocatable. I don't think OE wants to inject a special-case one-time generation of the toolchain file at SDK extraction time, as it circumvents the normal build process, Would it not be possible to generate the cross file when creating the SDK contents originally? The only change it would need is the same kind of path fixing. The setup does not really change. I think it would be possible, but I'm guessing it would require some special-casing that we may not want to do. Khem probably has a better informed opinion on this than I. Another approach would be to generate the toolchain file truly "on-the-fly", such that the meson command points to a script that first generates a toolchain file based on the contents of CC, CXX, etc. and then runs meson. I think this is a bad idea because it is complex (will definitely surprise people) and slow. It also breaks people in surprising ways when they accidentally use a meson from outside the SDK due to the PATH setup. This is, roughly, what Debian does currently. That's too bad :). -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On 01/09/2018 02:06 AM, Nirbheek Chauhan wrote: On Tue, Jan 9, 2018 at 3:20 PM, Nirbheek Chauhanwrote: If we want to setup a cross-file to use these arguments, we would have to generate the cross-file on-the-fly (not good). Out of interest, why is that not good? That is precisely what Gentoo does and it has to solve the same problems here that Yocto is facing. To expand on this, the reason why meson requires a cross file for cross-compilation is because we allow build files to use both native and cross compilers within the same build. This is quite useful when you want to build a tool (or generator) as part of the project that you then use to build/generate other things that are used in the cross build. To be clear, I think the cross-file is extremely important for correct cross-compilation, and I'm fully sold on its benefits. I believe OE sets CC, CXX, LD not because it's the best way to do cross compilation but because it needs to support all possible tools, including straight Makefiles with no wrappers. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
+CC Khem Raj, who has done a lot of work on the SDK and may have an opinion about handling relocatable cross files. On 01/09/2018 01:50 AM, Nirbheek Chauhan wrote: On Tue, Jan 9, 2018 at 5:21 AM, Martin Kellywrote: [Jussi Pakkanen, Nirbheek Chauhan, I know you may not be on the list; I added you to get Yocto/OE and meson upstream all on the same thread to discuss integrating the two] Hi all, Recently, we got meson added to OE-core as part of Yocto (thanks Alexander Kanavin!). Great to hear that! Thanks for your work, Alexander :) [snip] - The cross-file does not support relocatable cross-tools. This is an issue because OE SDKs are self-extracting shell scripts that can be installed anywhere on a machine. Once installed, they populate the env with --sysroot= and other flags pointing into the installed SDK location. For example, if I install an ARM SDK into /tmp/sdk, I would get something like this: It does support that as of 0.43: http://mesonbuild.com/Release-notes-for-0-43-0.html#can-override-executables-in-the-cross-file I believe this should solve your problems with cross-tools at least. Note that this also works for pkg-config, g-ir-compiler, g-ir-scanner, etc. This sounds like the right idea, but in its current form I'm not sure if it will solve the problem. The issue I hit when building actually were not caused by anything in the meson build file itself; they were caused by the meson sanity checker. meson appears to assume that anything found in CC, CXX, etc. are all native tools: $ . /tmp/sdk/environment-setup-cortexa7hf-neon-vfpv4-poky-linux-gnueabi $ echo $CC arm-poky-linux-gnueabi-gcc -march=armv7ve -marm -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot=/tmp/sdk/sysroots/cortexa7hf-neon-vfpv4-poky-linux-gnueabi $ meson --cross-file /tmp/meson.cross build . The Meson build system Version: 0.43.0 Source dir: /home/martin/xlib Build dir: /home/martin/xlib/build Build type: cross build Project name: xlib Native C compiler: arm-poky-linux-gnueabi-gcc -march=armv7ve -marm -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot=/tmp/sdk/sysroots/cortexa7hf-neon-vfpv4-poky-linux-gnueabi (gcc 7.2.0) Appending CFLAGS from environment: ' -O2 -pipe -g -feliminate-unused-debug-types ' Appending LDFLAGS from environment: '-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed' Cross C compiler: arm-poky-linux-gnueabi-gcc (gcc 7.2.0) /home/martin/xlib/build/meson-private/sanitycheckcpp.exe: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory Meson encountered an error in file meson.build, line 1, column 0: Executables created by cpp compiler arm-poky-linux-gnueabi-g++ -march=armv7ve -marm -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot=/tmp/sdk/sysroots/cortexa7hf-neon-vfpv4-poky-linux-gnueabi are not runnable. Note the "native C compiler" line, which directly uses $CC. I'm not sure if this is correct, but an easy way to fix the issue is to ignore $CC for internal sanity checking when a cross file is specified. In that way, meson would probe the system and use the normal gcc for sanity checking while still using the cross file for the actual build. $ echo $CC arm-poky-linux-gnueabi-gcc -march=armv7ve -marm -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot=/tmp/sdk/sysroots/cortexa7hf-neon-vfpv4-poky-linux-gnueabi If we want to setup a cross-file to use these arguments, we would have to generate the cross-file on-the-fly (not good). Out of interest, why is that not good? That is precisely what Gentoo does and it has to solve the same problems here that Yocto is facing. Right now, the SDK contains fixed contents, and there is some top-level logic for rewriting a few paths to make everything relocatable. I don't think OE wants to inject a special-case one-time generation of the toolchain file at SDK extraction time, as it circumvents the normal build process, which normally sets up the right environment for generating the right toolchain file. For example, cmake generates its toolchain as part of the build process and the SDK just extracts it without special knowledge of cmake or its contents. To my knowledge, nothing else in the OE special-cases its extraction logic, so it would be a special-case that would likely cause us problems in the future. At the very least, we would need to get more OE maintainers to voice an opinion on this. Another approach would be to generate the toolchain file truly "on-the-fly", such that the meson command points to a script that first generates a toolchain file based on the contents of CC, CXX, etc. and then runs meson. I think this is a bad idea because it is complex (will definitely surprise people) and slow. It also breaks people in surprising ways when they accidentally use a meson from outside the SDK due to the PATH setup. -- ___ Openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
On Tue, Jan 9, 2018 at 3:20 PM, Nirbheek Chauhanwrote: >> If we want to setup a cross-file to use these arguments, we would have to >> generate the cross-file on-the-fly (not good). > > Out of interest, why is that not good? That is precisely what Gentoo > does and it has to solve the same problems here that Yocto is facing. > To expand on this, the reason why meson requires a cross file for cross-compilation is because we allow build files to use both native and cross compilers within the same build. This is quite useful when you want to build a tool (or generator) as part of the project that you then use to build/generate other things that are used in the cross build. That workflow is really difficult or impossible in Autotools and CMake, but it's a surprisingly common use-case. Generating cross files is quite easy since they're INI files[1]. If you use the same toolchain for the entire build, you only need to generate it once. 1. https://docs.python.org/3/library/configparser.html Cheers, Nirbheek -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Issues with meson in SDK with cross-file
(re-sending because the previous one was automatically rejected because I wasn't subscribed) On Tue, Jan 9, 2018 at 5:21 AM, Martin Kellywrote: > [Jussi Pakkanen, Nirbheek Chauhan, I know you may not be on the list; I > added you to get Yocto/OE and meson upstream all on the same thread to > discuss integrating the two] > > Hi all, > > Recently, we got meson added to OE-core as part of Yocto (thanks Alexander > Kanavin!). Great to hear that! Thanks for your work, Alexander :) [snip] > - meson assumes that CC, CXX, CFLAGS, and other vars point to native tools. > However, in an OE SDK, they point to cross-tools. Thus meson fails under the > SDK when because it tries to use the cross-tools for native tasks (I get the > error "nm does not work" because meson tried to use the cross-nm for symbol > extraction). > > - The cross-file does not support relocatable cross-tools. This is an issue > because OE SDKs are self-extracting shell scripts that can be installed > anywhere on a machine. Once installed, they populate the env with --sysroot= > and other flags pointing into the installed SDK location. For example, if I > install an ARM SDK into /tmp/sdk, I would get something like this: > It does support that as of 0.43: http://mesonbuild.com/Release-notes-for-0-43-0.html#can-override-executables-in-the-cross-file I believe this should solve your problems with cross-tools at least. Note that this also works for pkg-config, g-ir-compiler, g-ir-scanner, etc. > $ echo $CC > arm-poky-linux-gnueabi-gcc -march=armv7ve -marm -mfpu=neon-vfpv4 > -mfloat-abi=hard -mcpu=cortex-a7 > --sysroot=/tmp/sdk/sysroots/cortexa7hf-neon-vfpv4-poky-linux-gnueabi > > If we want to setup a cross-file to use these arguments, we would have to > generate the cross-file on-the-fly (not good). Out of interest, why is that not good? That is precisely what Gentoo does and it has to solve the same problems here that Yocto is facing. > The way the cmake toolchain > file solves this is by encoding env vars into the toolchain file, which then > picks up the dynamically set CC, CXX, CFLAGS that point to the cross-tools: > > https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/cmake/cmake/OEToolchainConfig.cmake > > I'm not a meson expert, so I don't know if these are the right solutions, > but here are my proposed solutions just so I can get the discussion started: > Meson tries to reduce reliance on env vars as much as possible; the cross file is meant to be a standalone object. I would be happier if we could find a way to make things work for Yocto within that framework to keep complexity down. If that's not possible, we can look at changing things around. > - When --cross-file is specified, make meson not assume that env vars like > CC and CXX and similar point to native tools. Instead, just ignore them and > look for native tools in the normal way that meson does when these vars are > unspecified. We still may wan to inject CFLAGS, LDFLAGS, etc into the build > because people may mess with them interactively during development. > Would it be possible to tell Yocto to not set those vars when building a recipe that uses meson? > - Add the ability to reference env vars in cross files to support > relocatable toolchains. > That's one idea, but it adds a certain amount of complexity. Let's keep that on the table for now. Cheers, Nirbheek -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] Issues with meson in SDK with cross-file
[Jussi Pakkanen, Nirbheek Chauhan, I know you may not be on the list; I added you to get Yocto/OE and meson upstream all on the same thread to discuss integrating the two] Hi all, Recently, we got meson added to OE-core as part of Yocto (thanks Alexander Kanavin!). That said, it doesn't yet work with the Yocto SDK. I tried adding SDK support to the existing meson recipe (BBCLASSEXTEND = "nativesdk") and hit some issues that require some thought. I wanted to get OE and meson upstream on one thread to discuss the issues and come up with clean solutions for solving them. The issues I'm seeing arise because meson and OE appear to have differing views of how a cross-compile should look. Specifically: - meson assumes that CC, CXX, CFLAGS, and other vars point to native tools. However, in an OE SDK, they point to cross-tools. Thus meson fails under the SDK when because it tries to use the cross-tools for native tasks (I get the error "nm does not work" because meson tried to use the cross-nm for symbol extraction). - The cross-file does not support relocatable cross-tools. This is an issue because OE SDKs are self-extracting shell scripts that can be installed anywhere on a machine. Once installed, they populate the env with --sysroot= and other flags pointing into the installed SDK location. For example, if I install an ARM SDK into /tmp/sdk, I would get something like this: $ echo $CC arm-poky-linux-gnueabi-gcc -march=armv7ve -marm -mfpu=neon-vfpv4 -mfloat-abi=hard -mcpu=cortex-a7 --sysroot=/tmp/sdk/sysroots/cortexa7hf-neon-vfpv4-poky-linux-gnueabi If we want to setup a cross-file to use these arguments, we would have to generate the cross-file on-the-fly (not good). The way the cmake toolchain file solves this is by encoding env vars into the toolchain file, which then picks up the dynamically set CC, CXX, CFLAGS that point to the cross-tools: https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-devtools/cmake/cmake/OEToolchainConfig.cmake I'm not a meson expert, so I don't know if these are the right solutions, but here are my proposed solutions just so I can get the discussion started: - When --cross-file is specified, make meson not assume that env vars like CC and CXX and similar point to native tools. Instead, just ignore them and look for native tools in the normal way that meson does when these vars are unspecified. We still may wan to inject CFLAGS, LDFLAGS, etc into the build because people may mess with them interactively during development. - Add the ability to reference env vars in cross files to support relocatable toolchains. This would basically make meson behave the way cmake does today. Of course, others may have better ideas; I welcome thoughts. Thanks, Martin -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core