Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Tue, 28 Feb 2023 at 22:44, Helmut Grohne wrote: > Would you agree to either include the PR in your upload or pass > --debarch=$DEB_HOST_ARCH instead of --debarch=auto in debcrossgen? Because of #1032168 the current priority is to get 1.0.1 to testing. Thus I'm not going to do any changes to the packaging (apart from fixing that one issue if needed) until that has happened. I'll re-evaluate this once 1.0.1 is in testing. Thanks,
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Hi Jussi, On Mon, Feb 27, 2023 at 12:53:16AM +0200, Jussi Pakkanen wrote: > I filed an MR upstream with all changes that should go in the next > release (which is to say they won't make it to bookworm): > > https://github.com/mesonbuild/meson/pull/11464 This all looks good to me. > I also uploaded a new version to experimental that removes the test > patch and adds the ppc64 patch to debcrossgen. If people want to do > test it, go ahead, but after a few days I'll reupload that to > unstable. That will be the final version in bookworm unless major bugs > are found. The -3 upload looks suitable for bookworm to me. Unfortunately, it misses the PR, so if I were to locally patch debcrossgen to use the env2mfile implementation, it would pass --debarch=auto, but env2mfile would consider auto as an architecture name (given that your PR is missing) and fail. Would you agree to either include the PR in your upload or pass --debarch=$DEB_HOST_ARCH instead of --debarch=auto in debcrossgen? Doing so would allow me to switch my local autobuilder to env2mfile with a one line patch: sbuild ... '--chroot-setup-commands='sed -i -e "/revert_to_internal =/s/True/False" /usr/share/meson/debcrossgen || :' I still think that revert_to_internal should keep being True for bookworm and given what I've seen, please flip it to False in the trixie cycle. Helmut
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Sun, 26 Feb 2023 at 21:39, Helmut Grohne wrote: > Given that, I think you can move forward with the change post bookworm. > Thanks for bearing with me and sorry for the inadequate remarks. I filed an MR upstream with all changes that should go in the next release (which is to say they won't make it to bookworm): https://github.com/mesonbuild/meson/pull/11464 I also uploaded a new version to experimental that removes the test patch and adds the ppc64 patch to debcrossgen. If people want to do test it, go ahead, but after a few days I'll reupload that to unstable. That will be the final version in bookworm unless major bugs are found.
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Hi Jussi, On Sun, Feb 26, 2023 at 05:40:06PM +0200, Jussi Pakkanen wrote: > I find this a bit confusing. As far as I can tell those are in the file: > > https://github.com/mesonbuild/meson/blob/master/mesonbuild/scripts/env2mfile.py#L134 Mea culpa. My look into it was too brief. you're entirely right here. > The "deb" part of this script is basically the debcrossgen script > copypasted and slightly tweaked. I can see that there is no special > handling for "ppc64el", but that does not exist in debcrossgen either > and seems like a bug. That bug has a number, it is #1019413 and has a patch. > I also tried generating an armhf cross file and the output file had > cpu family set to "arm" and cpu to "arm7hlf" which is the same thing > that debcrossgen does. Even environment variables set via e.g. > CPPFLAGS get written in the cross file. My source review was not up to the task. I did not attempt to run it this time as my patches got stuck in the bts for quite long. > If this is not happening to you then it is very puzzling. Could you > give a few more details, like the exact command you are running and > how the generated cross file differs from what you expect? Given that, I think you can move forward with the change post bookworm. Thanks for bearing with me and sorry for the inadequate remarks. Helmut
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Sun, 26 Feb 2023 at 09:22, Helmut Grohne wrote: > I also spent a brief look at the env2mfile implementation and am > bewildered that it essentially discards the experience gained with > debcrossgen over the years. The mapping replacements for armhf, i386 and > mips64el are missing. Compiler flags are missing. So it looks more like > ppc64el happened to work by sheer luck. I find this a bit confusing. As far as I can tell those are in the file: https://github.com/mesonbuild/meson/blob/master/mesonbuild/scripts/env2mfile.py#L134 The "deb" part of this script is basically the debcrossgen script copypasted and slightly tweaked. I can see that there is no special handling for "ppc64el", but that does not exist in debcrossgen either and seems like a bug. I also tried generating an armhf cross file and the output file had cpu family set to "arm" and cpu to "arm7hlf" which is the same thing that debcrossgen does. Even environment variables set via e.g. CPPFLAGS get written in the cross file. If this is not happening to you then it is very puzzling. Could you give a few more details, like the exact command you are running and how the generated cross file differs from what you expect? Thanks,
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Hi Jussi, On Sun, Feb 26, 2023 at 03:16:08AM +0200, Jussi Pakkanen wrote: > I uploaded a new version to experimental that does exactly what I > described earlier. Basically everything should "just work" without any > changes needed. Feel free to try it out and report any failures to me. I'd appreciate if you could test your upload yourself rather than deferring to me. In any case, this can be done with sbuild -d unstable --build-dep-resolver=aptitude --host=ppc64el bolt --add-depends="meson (>= 1.0.1-2)" --extra-repository="deb http://deb.debian.org/debian experimental main" and I confirm that this succeeds now. So the experimental version will not break the world. It might still break some things. It quite certainly is inappropriate to bookworm due to the freeze timing. I also spent a brief look at the env2mfile implementation and am bewildered that it essentially discards the experience gained with debcrossgen over the years. The mapping replacements for armhf, i386 and mips64el are missing. Compiler flags are missing. So it looks more like ppc64el happened to work by sheer luck. A relatively simple method to address this would be adding a test case that compares a generated cross file to a native file and for every architecture and have it fail on differences. So no, this is not appropriate. I fear we'll have to live with debcrossgen for a longer while as env2mfile doesn't seem to catch up nor fix the patches reported. I admit being less than impressed by the progress on this matter. Helmut
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Sat, 25 Feb 2023 at 18:20, Helmut Grohne wrote: > While I appreciate the idea, I think the result is not useful. > debcrossgen is something that we want to delete before too long. It also > is platform logic by definition as it is specific to Debian. Carrying > platform logic (like touching DEB_HOST_* variables) there is something > that is quite simple and architecture-independent. As such, I think it > is the better compromise (for a limited amount of time). I uploaded a new version to experimental that does exactly what I described earlier. Basically everything should "just work" without any changes needed. Feel free to try it out and report any failures to me.
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Hi Jussi, On Sat, Feb 25, 2023 at 02:04:32PM +0200, Jussi Pakkanen wrote: > What I wanted was to isolate _all_ code that has to do with platform > detection and selection in exactly one place. The only thing > debcrossgen should do is to call a different command. It should have > _zero_ "platform logic" code, meaning it should not touch any > DEB_HOST_* environment variables. While I appreciate the idea, I think the result is not useful. debcrossgen is something that we want to delete before too long. It also is platform logic by definition as it is specific to Debian. Carrying platform logic (like touching DEB_HOST_* variables) there is something that is quite simple and architecture-independent. As such, I think it is the better compromise (for a limited amount of time). Anyway, your call. Both variants will work in practice. Helmut
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Sat, 25 Feb 2023 at 12:08, Helmut Grohne wrote:
> I think I understand this part. At no point did Simon or me ask for
> changing the interface of env2mfile. What we asked for is to retain the
> interface of debcrossgen, which is Debian-specific. These requirements
> seem quite easy to reconcile to me.
Sure, but in Meson (and distro packaging specifically) we want to move
_all_ functionality in core rather than keep them around as patches.
Having both an autodetection and an explicit platform selection is
useful on its own so it should be in upstream code. This approach does
that while still retaining the public interface for debcrossgen.
> Instead, we can make debcrossgen forward its --arch flag as --debarch if
> present. This part already happens if you use the external
> implementation. If no --arch is given, debcrossgen can add a
> --debarch=${DEB_HOST_ARCH} flag. This way the way debhelper calls
> debcrossgen continues to work and debcrossgen becomes a compatibility
> layer.
What I wanted was to isolate _all_ code that has to do with platform
detection and selection in exactly one place. The only thing
debcrossgen should do is to call a different command. It should have
_zero_ "platform logic" code, meaning it should not touch any
DEB_HOST_* environment variables.
> > - when everything has settled, debhelper can be changed to call
> > env2mfile --debarch=auto
>
> Instead, debhelper can simply call env2mfile --debarch=${DEB_HOST_ARCH}.
With this approach both of these work.
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Hi Jussi,
On Sat, Feb 25, 2023 at 03:18:57AM +0200, Jussi Pakkanen wrote:
> The reason for this is that env2file is meant to be general. It needs
> to support custom toolchains, distro package building and the like.
> Debcrossgen only does Debian packaging (and is quite a bit older,
> roo).
I think I understand this part. At no point did Simon or me ask for
changing the interface of env2mfile. What we asked for is to retain the
interface of debcrossgen, which is Debian-specific. These requirements
seem quite easy to reconcile to me.
> How about this then:
>
> - make env2mfile accept --debarch=auto that does the same thing as
> debcrossgen currently
I think this is entirely unnecessary. We don't need env2mfile changes.
> - make debcrossgen simply call 'meson env2mfile --debarch=auto'
Instead, we can make debcrossgen forward its --arch flag as --debarch if
present. This part already happens if you use the external
implementation. If no --arch is given, debcrossgen can add a
--debarch=${DEB_HOST_ARCH} flag. This way the way debhelper calls
debcrossgen continues to work and debcrossgen becomes a compatibility
layer.
> - when everything has settled, debhelper can be changed to call
> env2mfile --debarch=auto
Instead, debhelper can simply call env2mfile --debarch=${DEB_HOST_ARCH}.
> - eventually remove debcrossgen
Yes. Please keep it until one Debian stable release contains a debhelper
that calls env2mfile. That would mean including debcrossgen in trixie at
least.
Do you agree that my variation simplifies matters on the meson side? Do
you see any downsides?
Helmut
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Fri, 24 Feb 2023 at 23:29, Simon McVittie wrote: > The option seems to be debcrossgen --arch rather than --debarch, although > it becomes meson env2mfile --debarch. The reason for this is that env2file is meant to be general. It needs to support custom toolchains, distro package building and the like. Debcrossgen only does Debian packaging (and is quite a bit older, roo). > The way debhelper currently invokes debcrossgen (without --arch) should > continue to behave as if --arch=$DEB_HOST_ARCH had been used, unless > someone changes debhelper to always give it the --arch argument (or use > meson env2mfile more directly) during a post-bookworm release cycle. > If a version of meson makes an incompatible change to the debcrossgen > interface, then that version will need a Breaks: debhelper (<< x.y~) > where x.y is the first version of debhelper that used the new interface > (whatever that new interface is). How about this then: - make env2mfile accept --debarch=auto that does the same thing as debcrossgen currently - make debcrossgen simply call 'meson env2mfile --debarch=auto' - when everything has settled, debhelper can be changed to call env2mfile --debarch=auto - eventually remove debcrossgen
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Fri, 24 Feb 2023 at 22:42:09 +0200, Jussi Pakkanen wrote:
> Using the log info I managed to find out why it does not work. The
> tests that I have specifically pass --debarch=xxx to debcrossgen which
> then passes it on to env2mfile and things work. However the package
> builds do not do that, instead they call `debcrossgetn -oXXX` which
> then fails.
The common case is that debhelper invokes debcrossgen, via
/usr/share/perl5/Debian/Debhelper/Buildsystem/meson.pm, and you'll
see that in the current version of debhelper it never specifies the --arch
option.
The option seems to be debcrossgen --arch rather than --debarch, although
it becomes meson env2mfile --debarch.
The current interface when no --arch is specified appears to be that
debcrossgen runs dpkg-architecture with no arguments, and parses its
output, which is documented to vary depending on the value of the
DEB_HOST_ARCH environment variable (which should always be set to some
suitable value while cross-compiling a Debian package, and debcrossgen
only makes sense during cross-compilation, so I think it would be OK
for debcrossgen to fail or even crash if it's unset).
> The proper fix for this would require knowing how Debian intends to
> use things. If it is ok to require package builders to pass
> --debarch=armhf or somesuch or does it need to work by detecting magic
> envvars (which I personally do not like as envvars are an endless
> source of pain, but they are fairly entrenched so sometimes you have
> to use them).
The way debhelper currently invokes debcrossgen (without --arch) should
continue to behave as if --arch=$DEB_HOST_ARCH had been used, unless
someone changes debhelper to always give it the --arch argument (or use
meson env2mfile more directly) during a post-bookworm release cycle.
If a version of meson makes an incompatible change to the debcrossgen
interface, then that version will need a Breaks: debhelper (<< x.y~)
where x.y is the first version of debhelper that used the new interface
(whatever that new interface is).
As someone who backports meson into derivatives of (much) older Debian
versions during my day job, without usually backporting debhelper,
I'd personally prefer it if the current debcrossgen interface continues
to work indefinitely, even if it's considered deprecated. If a change
is necessary then so be it, but it seems to me as though logic in
debcrossgen like
parser.add_argument('--arch', default=os.environ['DEB_HOST_ARCH'])
would be an easy way to make the old interface work forever, while also
providing an "explicit is better than implicit" interface for specifying
the architecture more directly.
smcv
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Fri, 24 Feb 2023 at 19:38, Helmut Grohne wrote: > > Could it be that you tested env2mfile, but not debcrossgen? It would > always exit non-zero no matter what. No error message. This really > should have been caught by the "crossbuild" autopkgtest, but wasn't. I > don't quite understand why. It happened on every single build using > meson via dh_auto_configure. Using the log info I managed to find out why it does not work. The tests that I have specifically pass --debarch=xxx to debcrossgen which then passes it on to env2mfile and things work. However the package builds do not do that, instead they call `debcrossgetn -oXXX` which then fails. The output is swallowed somewhere, but I don't really know where. The proper fix for this would require knowing how Debian intends to use things. If it is ok to require package builders to pass --debarch=armhf or somesuch or does it need to work by detecting magic envvars (which I personally do not like as envvars are an endless source of pain, but they are fairly entrenched so sometimes you have to use them).
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Hi Jussi, On Fri, Feb 24, 2023 at 02:27:20PM +0200, Jussi Pakkanen wrote: > I do have a test that runs the new cross file generator. I ran it > again by hand and it does work. The files it produces are not exactly > the same, but the information within seems to be the same. The only > major difference is the new cmake section, which should be ignored if > CMake is not used in the build (and not even generated if the CMake > binary is not present. Could it be that you tested env2mfile, but not debcrossgen? It would always exit non-zero no matter what. No error message. This really should have been caught by the "crossbuild" autopkgtest, but wasn't. I don't quite understand why. It happened on every single build using meson via dh_auto_configure. > Thus I'd need logs of the actual build failure that occurred, but I > could not easily find any. The NMU was done without a corresponding > bug, so there is no trace there. I can upload a version with the new > code path enabled to experimental if that helps. One of the obvious examples is bolt. You can find logs at https://crossqa.debian.net/src/bolt Yeah, let's try experimental. Can you try building bolt using your meson before uploading? An easy way is using sbuild together with --extra-package. Please ping me after the upload. Helmut
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Fri, 24 Feb 2023 at 08:26, Helmut Grohne wrote: > I think you know, but let's include the answer here for clarity: You > change debcrossgen to become a wrapper of env2mfile. However doing so > requires that env2mfile actually works and last time you attempted this, > env2mfile was so horribly broken that I had to upload an emergency > reversion. So please retry after having verified that env2mfile actually I do have a test that runs the new cross file generator. I ran it again by hand and it does work. The files it produces are not exactly the same, but the information within seems to be the same. The only major difference is the new cmake section, which should be ignored if CMake is not used in the build (and not even generated if the CMake binary is not present. Thus I'd need logs of the actual build failure that occurred, but I could not easily find any. The NMU was done without a corresponding bug, so there is no trace there. I can upload a version with the new code path enabled to experimental if that helps.
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Hi Jussi, On Sun, Aug 28, 2022 at 10:57:00PM +0300, Jussi Pakkanen wrote: > The first step would be to change the default packaging rules to stop > using debcrossgen. This is now possible since there is a new enough > Meson version in unstable. What's the correct way of getting that > default changed? I think you know, but let's include the answer here for clarity: You change debcrossgen to become a wrapper of env2mfile. However doing so requires that env2mfile actually works and last time you attempted this, env2mfile was so horribly broken that I had to upload an emergency reversion. So please retry after having verified that env2mfile actually works well enough to support some use cases and preferrably also fixes some of the bugs we reported such as this one, #895636, #1019413 and #1023744. And yeah, I know it's always the wrong time. When you read this mail Debian is in freeze or meson is in freeze or both. I'm not asking to do this now, but maybe letting such bugs rot for four years is not the best answer either. The current process of pinging you until there is a convenient time does not work well. Helmut
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
On Tue, 16 Aug 2022 at 12:51, Simon McVittie wrote: > debcrossgen puts c_args, cpp_args, c_link_args and cpp_link_args in the > [properties] section of the cross-file that it generates, but recent > versions of Meson want this to be in the [built-in options] section > instead. We have been moving the functionality of debcrossgen into Meson proper. Eventually we'd like for Debian's packaging to not use debcrossgen but instead `meson env2mfile --debarch`. In this way all needed changes can be put into master through proper code review, as they should, rather than living as distro patches. The first step would be to change the default packaging rules to stop using debcrossgen. This is now possible since there is a new enough Meson version in unstable. What's the correct way of getting that default changed?
Bug#1017436: meson: deprecation warning when cross-compiling: foo_args in the [properties] section of the machine file is deprecated
Package: meson Version: 0.63.1-1 Severity: minor debcrossgen puts c_args, cpp_args, c_link_args and cpp_link_args in the [properties] section of the cross-file that it generates, but recent versions of Meson want this to be in the [built-in options] section instead.
