Re: Cross compilation status
Am Montag, dem 18.09.2023 um 17:17 +0300 schrieb Efraim Flashner: > On Sun, Sep 10, 2023 at 04:59:35PM +0200, Liliana Marie Prikler > wrote: > > Hi Mathieu, > > > > Am Sonntag, dem 10.09.2023 um 11:14 +0200 schrieb Mathieu Othacehe: > > > I have CC'ed members of the python, java, ruby, rust, r, haskell > > > and > > > emacs teams. Any plans adding cross-compilation support to your > > > build-system, barriers to overcome? > > For Emacs, there are layers of problems. First things first, we > > are only doing byte compilation atm, which may or may not be > > arch-dependant based on stuff like endianness. For native > > compilation, there is the thing that even a slightly differently > > configured emacs won't load the packages you've just compiled > > without compiling it natively on its own. > > Granted, if we want to do cross-compilation, having just the > > bytecode might already be "good enough", but I wanted to throw this > > out there. > > > > I think we'd need some support from Emacs proper before we can > > start implementing cross builds. > > I will note that the only big-endian architecture which we "support" > is 32-bit powerpc, so we don't have to worry that much about wrong > endianness. Fine, let's add word size to point out the obvious :)
Re: Cross compilation status
On Sun, Sep 10, 2023 at 04:59:35PM +0200, Liliana Marie Prikler wrote: > Hi Mathieu, > > Am Sonntag, dem 10.09.2023 um 11:14 +0200 schrieb Mathieu Othacehe: > > I have CC'ed members of the python, java, ruby, rust, r, haskell and > > emacs teams. Any plans adding cross-compilation support to your > > build-system, barriers to overcome? > For Emacs, there are layers of problems. First things first, we are > only doing byte compilation atm, which may or may not be arch-dependant > based on stuff like endianness. For native compilation, there is the > thing that even a slightly differently configured emacs won't load the > packages you've just compiled without compiling it natively on its own. > Granted, if we want to do cross-compilation, having just the bytecode > might already be "good enough", but I wanted to throw this out there. > > I think we'd need some support from Emacs proper before we can start > implementing cross builds. I will note that the only big-endian architecture which we "support" is 32-bit powerpc, so we don't have to worry that much about wrong endianness. -- Efraim Flashner רנשלפ םירפא GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted signature.asc Description: PGP signature
Re: Cross compilation status
Mathieu Othacehe skribis: > In order for Guix to become an alternative to tools such as Yocto and > Buildroot, having most or all our packages cross-compiling is a > prerequisite. > > Here is a status of cross-compilation in Guix. For cross-compilation to > work, the build-system needs to support cross-compilation. > > The following build-systems explicitly refuse cross-compiling packages: > > haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, cargo, > ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, maven, asdf, r, > ocaml, node > > while the rest of the build-systems do accept cross-compiling packages: > > clojure, qt, copy, minetest-mod, tree-sitter, raw, linux-module, glib-or-gtk, > asdf/source, go, cmake, minify, perl, trivial, guile, elm, font, gnu, > asdf/ecl, > asdf/sbcl, meson, mozilla, texlive Hi. I'm surprised to see asdf/* in the list of build systems accepting to cross-compile packages. How did you test them? Because I get (on a x86-64 machine): --8<---cut here---start->8--- $ guix build --target=aarch64-linux-gnu sbcl-alexandria guix build: error: gnu/packages/lisp-xyz.scm:168:2: sbcl-alexandria@1.4: build system `asdf/sbcl' does not support cross builds $ guix build --target=aarch64-linux-gnu ecl-alexandria guix build: error: gnu/packages/lisp-xyz.scm:168:2: ecl-alexandria@1.4: build system `asdf/ecl' does not support cross builds --8<---cut here---end--->8--- I'm not even sure if sbcl and ecl have the ability to compile Common Lisp code for an architecture different from the one they are running on. signature.asc Description: PGP signature
Re: Cross compilation status
Am Montag, dem 11.09.2023 um 18:19 +0200 schrieb Simon Tournier: > Hi Mathieu, > > On Sun, 10 Sep 2023 at 11:14, Mathieu Othacehe > wrote: > > > Here is a status of cross-compilation in Guix. For cross- > > compilation to > > work, the build-system needs to support cross-compilation. > > > > The following build-systems explicitly refuse cross-compiling > > packages: > > > > haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, > > cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, > > maven, asdf, r, ocaml, node > > > > [...] > > > --8<---cut here---start->8--- > > Total packages: 28076 > > > > clojure: 13 > > qt: 317 > > copy: 134 > > minetest-mod: 22 > > tree-sitter: 35 > > raw: 2 > > linux-module: 17 > > glib-or-gtk: 139 > > asdf/source: 794 > > go: 632 > > cmake: 1278 > > minify: 12 > > perl: 839 > > trivial: 250 > > guile: 49 > > elm: 30 > > font: 101 > > gnu: 4178 > > asdf/ecl: 753 > > asdf/sbcl: 814 > > meson: 504 > > mozilla: 2 > > texlive: 4150 > > Cross-compilation OK: 15065 > > > > ocaml: 61 > > haskell: 812 > > maven: 2 > > chicken: 12 > > node: 57 > > emacs: 1370 > > dune: 289 > > android-ndk: 12 > > waf: 21 > > julia: 300 > > pyproject: 433 > > r: 2441 > > cargo: 3535 > > ruby: 597 > > rebar: 22 > > scons: 15 > > rakudo: 22 > > agda: 6 > > ant: 559 > > python: 2488 > > Cross-compilation KO: 13054 > > --8<---cut here---end--->8--- > > The build-systems renpy, dub is listed in “refuse“ but not then in > Cross-compilation KO. Is it expected? On the renpy side, there currently are no packages to cross-compile. "The Question", which ships with renpy, is built as an output rather than with renpy-build-system and was only used as a proof of concept back then. If I do encounter a renpy-based game that I actually can get the sources for, I'll gladly package it. HTH
Re: Cross compilation status
Hi Mathieu, On Sun, 10 Sep 2023 at 11:14, Mathieu Othacehe wrote: > Here is a status of cross-compilation in Guix. For cross-compilation to > work, the build-system needs to support cross-compilation. > > The following build-systems explicitly refuse cross-compiling packages: > > haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, cargo, > ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, maven, asdf, r, > ocaml, node > [...] > --8<---cut here---start->8--- > Total packages: 28076 > > clojure: 13 > qt: 317 > copy: 134 > minetest-mod: 22 > tree-sitter: 35 > raw: 2 > linux-module: 17 > glib-or-gtk: 139 > asdf/source: 794 > go: 632 > cmake: 1278 > minify: 12 > perl: 839 > trivial: 250 > guile: 49 > elm: 30 > font: 101 > gnu: 4178 > asdf/ecl: 753 > asdf/sbcl: 814 > meson: 504 > mozilla: 2 > texlive: 4150 > Cross-compilation OK: 15065 > > ocaml: 61 > haskell: 812 > maven: 2 > chicken: 12 > node: 57 > emacs: 1370 > dune: 289 > android-ndk: 12 > waf: 21 > julia: 300 > pyproject: 433 > r: 2441 > cargo: 3535 > ruby: 597 > rebar: 22 > scons: 15 > rakudo: 22 > agda: 6 > ant: 559 > python: 2488 > Cross-compilation KO: 13054 > --8<---cut here---end--->8--- The build-systems renpy, dub is listed in “refuse“ but not then in Cross-compilation KO. Is it expected? The build-system asdf is listed as refuse but appears in the list Cross-compilation OK. Is it expected? > Over the 28076 packages in Guix, 15065 are part of build-systems with > cross-compilation support and 13054 are part of build-systems without > cross-compilation support. > > Overall 46.5% of our packages will refuse to cross-compile with errors > such as: > > --8<---cut here---start->8--- > mathieu@meije ~$ guix build --target=aarch64-linux-gnu librsvg > guix build: error: gnu/packages/gnome.scm:3500:2: librsvg@2.54.5: build > system `cargo' does not support cross builds > --8<---cut here---end--->8--- > > I'd like to help converge towards the situation where all build-systems > support cross-compilation. Considering Julia, the “julia compiler” support of upstream [1] reads: + ARMv8 (64-bit): Tier 2 + ARMv7 (32-bit): Tier 3 + PowerPC (64-bit): Tier 3 + then stuff about CUDA, ROCM, oneAPI where: + Tier 2: Julia is guaranteed to build from source using the default build options, but may or may not pass all tests. Official binaries are available on a case-by-case basis. + Tier 3: Julia may or may not build. If it does, it is unlikely to pass tests. Binaries may be available in some cases. When they are, they should be considered experimental. Ongoing support is dependent on community efforts. But I guess, these are about native compilation. To my tiny knowledge on that topic, Julia supports some cross-compilation. However, Julia depends on heavy packages as ’suitesparse’ or ’openblas’ and I do not know if we already cross-compile them. Since Julia is used for scientific computations, I do not know if the effort is worth. 1: https://julialang.org/downloads/ > I have CC'ed members of the python, java, ruby, rust, r, haskell and > emacs teams. Any plans adding cross-compilation support to your > build-system, barriers to overcome? About Haskell, from experience, we are already not able to maintain an usable ecosystem for i686, but that’s another story. :-) Well, I do not know if cross-compilation is well-supported by the Haskell compiler GHC. Maybe some resources seems there: https://github.com/input-output-hk/haskell.nix About R, similarly as Julia, since they are mainly used for scientific computations, I do not know if supporting cross-compilation is worth our effort. Cheers, simon
Re: Cross compilation status
Hi Mathieu, Am Sonntag, dem 10.09.2023 um 11:14 +0200 schrieb Mathieu Othacehe: > I have CC'ed members of the python, java, ruby, rust, r, haskell and > emacs teams. Any plans adding cross-compilation support to your > build-system, barriers to overcome? For Emacs, there are layers of problems. First things first, we are only doing byte compilation atm, which may or may not be arch-dependant based on stuff like endianness. For native compilation, there is the thing that even a slightly differently configured emacs won't load the packages you've just compiled without compiling it natively on its own. Granted, if we want to do cross-compilation, having just the bytecode might already be "good enough", but I wanted to throw this out there. I think we'd need some support from Emacs proper before we can start implementing cross builds. Cheers
Re: Cross compilation status
Hi Mathieu! Mathieu Othacehe writes: > Hello, > > In order for Guix to become an alternative to tools such as Yocto and > Buildroot, having most or all our packages cross-compiling is a > prerequisite. > > Here is a status of cross-compilation in Guix. For cross-compilation to > work, the build-system needs to support cross-compilation. > > The following build-systems explicitly refuse cross-compiling packages: > > haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, > cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, > maven, asdf, r, ocaml, node For Python, it's already reported at: https://issues.guix.gnu.org/60733. And even implemented here: https://issues.guix.gnu.org/60849. Why it hasn't been merged yet is because it's coupled with a more daring change (unified build system for Python) and there were comments that this should be an all or nothing change, so I'd need to workout all the build systems and see what breakage happens at the global scale, which is a rather large undertaking. Maybe it could be decoupled from that change and tested in isolation, pushed first; we'd loose the benefit of the later change which is the ability to avoid wrapping native-inputs in our wrappers. -- Thanks, Maxim
Re: Cross compilation status
On Sun, Sep 10, 2023 at 11:14:29AM +0200, Mathieu Othacehe wrote: > > Hello, > > In order for Guix to become an alternative to tools such as Yocto and > Buildroot, having most or all our packages cross-compiling is a > prerequisite. > > Here is a status of cross-compilation in Guix. For cross-compilation to > work, the build-system needs to support cross-compilation. > > The following build-systems explicitly refuse cross-compiling packages: > > > ocaml: 61 > haskell: 812 > maven: 2 > chicken: 12 > node: 57 > emacs: 1370 > dune: 289 > android-ndk: 12 > waf: 21 > julia: 300 > pyproject: 433 > r: 2441 > cargo: 3535 > ruby: 597 > rebar: 22 > scons: 15 > rakudo: 22 > agda: 6 > ant: 559 > python: 2488 > Cross-compilation KO: 13054 > --8<---cut here---end--->8--- > > Over the 28076 packages in Guix, 15065 are part of build-systems with > cross-compilation support and 13054 are part of build-systems without > cross-compilation support. > > Overall 46.5% of our packages will refuse to cross-compile with errors > such as: > > --8<---cut here---start->8--- > mathieu@meije ~$ guix build --target=aarch64-linux-gnu librsvg > guix build: error: gnu/packages/gnome.scm:3500:2: librsvg@2.54.5: build > system `cargo' does not support cross builds > --8<---cut here---end--->8--- > > I'd like to help converge towards the situation where all build-systems > support cross-compilation. > > I have CC'ed members of the python, java, ruby, rust, r, haskell and > emacs teams. Any plans adding cross-compilation support to your > build-system, barriers to overcome? I suspect librsvg is especially a problem since it is almost always pulled in with any sort of image building. I haven't looked at it closely yet, but I believe that in order to support cross-compilation support in rust we'll need to enable more targets, which will increase the build time for the final rust and the final output size. Considering that nothing built with rust links to it, I'm not convinced this is a problem. We will likely see people confused that we have a rust with cross-compiling abilities but with windows- and mac-specific (pre-compiled) libraries removed, so it wouldn't be able to cross-compile to those (or some other) platforms without changes. -- Efraim Flashner רנשלפ םירפא GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted signature.asc Description: PGP signature
Re: Cross compilation status
On Sun, Sep 10, 2023 at 11:14:29AM +0200, Mathieu Othacehe wrote: > > I have CC'ed members of the python, java, ruby, rust, r, haskell and > emacs teams. Any plans adding cross-compilation support to your > build-system, barriers to overcome? Speaking as the one who added the rakudo build system and the perl6 packages, IMO all the perl6 packages should be removed and also the rakudo-build-system. I don't believe most of the packages have built for more than a year and I don't believe they have ever had any adoption. If it turns out that Raku (no longer perl6 for several years now) would make use of the current rakudo-build-system then we can always revert removing it. -- Efraim Flashner רנשלפ םירפא GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted signature.asc Description: PGP signature
Cross compilation status
Hello, In order for Guix to become an alternative to tools such as Yocto and Buildroot, having most or all our packages cross-compiling is a prerequisite. Here is a status of cross-compilation in Guix. For cross-compilation to work, the build-system needs to support cross-compilation. The following build-systems explicitly refuse cross-compiling packages: haskell, agda, waf, chicken, rakudo, julia, python, emacs, rebar, cargo, ruby, renpy, dub, android-ndk, scons, dune, ant, pyproject, maven, asdf, r, ocaml, node while the rest of the build-systems do accept cross-compiling packages: clojure, qt, copy, minetest-mod, tree-sitter, raw, linux-module, glib-or-gtk, asdf/source, go, cmake, minify, perl, trivial, guile, elm, font, gnu, asdf/ecl, asdf/sbcl, meson, mozilla, texlive It does not mean that all the packages relying on those build-systems will cross-compile OK. It just means that we will at least try to build those. In term of numbers, the attached script produces the following output: --8<---cut here---start->8--- Total packages: 28076 clojure: 13 qt: 317 copy: 134 minetest-mod: 22 tree-sitter: 35 raw: 2 linux-module: 17 glib-or-gtk: 139 asdf/source: 794 go: 632 cmake: 1278 minify: 12 perl: 839 trivial: 250 guile: 49 elm: 30 font: 101 gnu: 4178 asdf/ecl: 753 asdf/sbcl: 814 meson: 504 mozilla: 2 texlive: 4150 Cross-compilation OK: 15065 ocaml: 61 haskell: 812 maven: 2 chicken: 12 node: 57 emacs: 1370 dune: 289 android-ndk: 12 waf: 21 julia: 300 pyproject: 433 r: 2441 cargo: 3535 ruby: 597 rebar: 22 scons: 15 rakudo: 22 agda: 6 ant: 559 python: 2488 Cross-compilation KO: 13054 --8<---cut here---end--->8--- Over the 28076 packages in Guix, 15065 are part of build-systems with cross-compilation support and 13054 are part of build-systems without cross-compilation support. Overall 46.5% of our packages will refuse to cross-compile with errors such as: --8<---cut here---start->8--- mathieu@meije ~$ guix build --target=aarch64-linux-gnu librsvg guix build: error: gnu/packages/gnome.scm:3500:2: librsvg@2.54.5: build system `cargo' does not support cross builds --8<---cut here---end--->8--- I'd like to help converge towards the situation where all build-systems support cross-compilation. I have CC'ed members of the python, java, ruby, rust, r, haskell and emacs teams. Any plans adding cross-compilation support to your build-system, barriers to overcome? Thanks, Mathieu cross.scm Description: Binary data
