Re: full-source bootstrap and Python
Janneke Nieuwenhuizen wrote: > For the current situtation (that's less than great and are > working on to resolve), making essential GNU packages less > bootstrappable is of no consequence. OK. That's what I conjectured. Thanks for confirming. > Cleaning-up the full-source > bootstrap and making it more or less future-proof, might be challenged > by such a new dependency. I see. So, I wondered whether Python can be compiled by the last C-only GCC, gcc-4.7.3. And yes, Python-3.7.17 (the minimum supported version for gnulib-tool) compiles with gcc-4.7.3. Whereas a newer version, Python-3.12.2, doesn't. > As an example, we have been working on the > RISC-V bootstrap for about a year with three people. One of the > problems here is that RISC-V was only added to a non-bootstrappable > version of GCC: 7.5.0, while the GCC team failed to maintain their last > bootstrappable version: 4.7.4. In other words, the RISC-V backend > needed to backported and someone else now needs to maintain a > bootstrappable version of GCC. Oh, you don't allow yourself to use a cross-compiler from, say, x86? I mean, running a complete x86 bootstrap, followed by the creation of a cross-compiled RISC-V compiler, in a virtualized environment on RISC-V could work in theory (although it would be very slow). Bruno
Re: full-source bootstrap and Python
Janneke Nieuwenhuizen writes: >> Also, from the diagrams in [1][2][3] it looks like the full-source bootstrap >> uses tarballs frozen in time (make-3.80, gcc-2.95.3, gcc 4.7.3, etc.). So, >> even if newer versions of 'make' or 'gcc' will use a Python-based >> gnulib-tool, >> there won't be a problem, because the bootstrap of these old tarballs will >> be unaffected. > > indeed. For the current situtation (that's less than great and are > working on to resolve), making essential GNU packages less > bootstrappable is of no consequence. Cleaning-up the full-source > bootstrap and making it more or less future-proof, might be challenged > by such a new dependency. Rather than finding out what dependencies are problematic through tedious manual work, is there a recommendation we can articulate that would help the bootstrappable effort? For example, in Libtasn1 (which I guess is fairly low in the bootstrapping graph) I made the CI/CD pipeline [1] build the tarball on Debian 4 etch (2010, first amd64 release), and using 'pcc' and 'tcc' as alternative C compilers. I'm hoping this has some value, but I have no good way to tell. What actual testable environments would it make sense to test a project in, to help the bootstrappable effort? Right now these targets build fine, but if at some point 'pcc' stops building, I may be inclinced to simply drop this target rather than to fix the bugs since I have no idea if supporting building with 'pcc' helps anyone. I'm thinking suggestions like 'Build and test project on i386 Debian 3', or 'Cross-build project from amd64 to mipsel on Debian 4'. I can't seem to find docker images for CentOS 3-6, maybe old CentOS is a good long-term target too. If there were concrete fact-based suggestions like that, I would make an effort to CI/CD build libidn, libidn2, inetutils, and some other projects to make sure they continue to work on old platforms. /Simon [1] https://gitlab.com/gnutls/libtasn1/-/pipelines/ signature.asc Description: PGP signature
Re: full-source bootstrap and Python
Simon Josefsson wrote: > If we want to minimize the work for full-source bootstrap people we > increase the cost of people maintaining modern software, and vice versa, +1 > Consider the > extreme situation where gnulib-tool version A would require coreutils > verison B, and coreutils version B+1 would require gnulib-tool version > A+1, and gnulib-tool version A+2 would require coreutils version B+1 and > so on for really short release version increments. Then a full-source > bootstrap will need to package and keep maintain all those coreutils and > gnulib-tool versions -- or start to patch things to avoid the > dependencies. Right. This way of working was pretty common in the Lisp and Haskell worlds, many years ago. For gnulib-tool, we're relying on Python 3.7, which is already 6 years old; so, we are clearly not doing the extreme thing. Bruno
Re: full-source bootstrap and Python
Bruno Haible writes: Hi! > Janneke Nieuwenhuizen wrote: >> Are are we creating a problem for >> bootstrapping (or even a dependency cycle) when introducing this new >> dependency into a certain package. > > I think you answered this question with "no", when writing in [1]: > > "Even more recently (2018), the GNU C Library glibc-2.28 adds Python >as a build requirement" How so? In 2013, the GCC folks decided that their gcc-4.8 would no longer be a directly bootstrappable compiler by introducing a (casual?) dependency on c++. That's pretty bad for bootstrapping, and it would be amazing if someone could revet that mistake. With glibc-2.28, a similar bootstrappable mistake was made. Making an essential GNU package such as GCC or Glibc non-bootstrappable has severe consequences. As an example, we have been working on the RISC-V bootstrap for about a year with three people. One of the problems here is that RISC-V was only added to a non-bootstrappable version of GCC: 7.5.0, while the GCC team failed to maintain their last bootstrappable version: 4.7.4. In other words, the RISC-V backend needed to backported and someone else now needs to maintain a bootstrappable version of GCC. When something else like this changes in the future, that isn't as "easily" backported, what do we do? Terrible! > So, how do you avoid Python when building glibc? Do you use musl libc as > a first stage, and only build glibc once a python built with musl exists? Currently we aren't directly hit by this because we build glibc-2.2.5 and glibc-2.16 first. We added these earlier Glibc's because we allowed ourcelves to cut some corners and there earlier versions were more easy to bootstrap. On the roadmap is to remove as many ancient versions as we can, as they are potential time-bombs. In fact, glibc-2.2.5 is problematic for the ARM and the RISC-V bootstrap, so we'll get rid of that. We won't be able to get rid of advancing beyond glibc-2.27 without adding a yet another non-GNU package such as musl libc. Possibly a nice hack for now, but what to do when we want to port the bootstrap to the Hurd? Again, terrible! > Also, from the diagrams in [1][2][3] it looks like the full-source bootstrap > uses tarballs frozen in time (make-3.80, gcc-2.95.3, gcc 4.7.3, etc.). So, > even if newer versions of 'make' or 'gcc' will use a Python-based gnulib-tool, > there won't be a problem, because the bootstrap of these old tarballs will > be unaffected. indeed. For the current situtation (that's less than great and are working on to resolve), making essential GNU packages less bootstrappable is of no consequence. Cleaning-up the full-source bootstrap and making it more or less future-proof, might be challenged by such a new dependency. Greetings, Janneke -- Janneke Nieuwenhuizen | GNU LilyPond https://LilyPond.org Freelance IT https://www.JoyOfSource.com | Avatar® https://AvatarAcademy.com
Re: full-source bootstrap and Python
Bruno Haible writes: > Janneke Nieuwenhuizen wrote: >> Are are we creating a problem for >> bootstrapping (or even a dependency cycle) when introducing this new >> dependency into a certain package. > > I think you answered this question with "no", when writing in [1]: > > "Even more recently (2018), the GNU C Library glibc-2.28 adds Python >as a build requirement" > > So, how do you avoid Python when building glibc? Do you use musl libc as > a first stage, and only build glibc once a python built with musl exists? > > Also, from the diagrams in [1][2][3] it looks like the full-source bootstrap > uses tarballs frozen in time (make-3.80, gcc-2.95.3, gcc 4.7.3, etc.). So, > even if newer versions of 'make' or 'gcc' will use a Python-based gnulib-tool, > there won't be a problem, because the bootstrap of these old tarballs will > be unaffected. While I agree, I think there is one nuance that could be added here: it is true that full-source bootstraps usually needs to use earlier releases of software tarballs to build more recent projects because of cyclic dependencies. However this cause extra work for people involved. It also means we will have to keep maintaining and patching all the software that is involved in the full-source bootstrap to keep it working in the future. So there is a cost involved here. The takeaway from that situation should NOT be "don't use python" or "don't use modern tools", that would be absurd. The takeaway should be that one should carefully evaluate the implications of using Python and modern tools, and look at the costs. If we want to minimize the work for full-source bootstrap people we increase the cost of people maintaining modern software, and vice versa, I don't see how we can get away from this conflict. If everyone wrote everything in machine code, there would be nothing for the bootstrap people to do. However what we can do is to reduce the total amount of work involved by not introducing too many bootstrap dependency cycles. Consider the extreme situation where gnulib-tool version A would require coreutils verison B, and coreutils version B+1 would require gnulib-tool version A+1, and gnulib-tool version A+2 would require coreutils version B+1 and so on for really short release version increments. Then a full-source bootstrap will need to package and keep maintain all those coreutils and gnulib-tool versions -- or start to patch things to avoid the dependencies. (I'm ignoring the fact that normally gnulib-tool is not involved at all when building projects.) I think gnulib is already quite careful in dependency tracking, more so than most projects I'm familiar with, but it doesn't mean this can be improved. Janneke: is there any recommendation from you as a bootstrapping person on what dependency we should be careful with, and which dependencies that are fine? I suppose/hope that if gnulib-tool required python version 2, you would not have a serious problem? I'm certain that python version 2 is possible to build using really old toolchains. At what version of python would it lead require added another bootstrapping step to the graph? Python 3.0, 3.7, 3.12? Maybe it is not easy to answer this without generating the graph. But I also think you would have a better feeling of what the answer would be than most of us. /Simon > Bruno > > [1] > https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ > [2] > https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed-Bootstrap.html > [3] > https://guix.gnu.org/manual/devel/en/html_node/Full_002dSource-Bootstrap.html > > > > > signature.asc Description: PGP signature
Re: full-source bootstrap and Python
Janneke Nieuwenhuizen wrote: > Are are we creating a problem for > bootstrapping (or even a dependency cycle) when introducing this new > dependency into a certain package. I think you answered this question with "no", when writing in [1]: "Even more recently (2018), the GNU C Library glibc-2.28 adds Python as a build requirement" So, how do you avoid Python when building glibc? Do you use musl libc as a first stage, and only build glibc once a python built with musl exists? Also, from the diagrams in [1][2][3] it looks like the full-source bootstrap uses tarballs frozen in time (make-3.80, gcc-2.95.3, gcc 4.7.3, etc.). So, even if newer versions of 'make' or 'gcc' will use a Python-based gnulib-tool, there won't be a problem, because the bootstrap of these old tarballs will be unaffected. Bruno [1] https://guix.gnu.org/en/blog/2023/the-full-source-bootstrap-building-from-source-all-the-way-down/ [2] https://guix.gnu.org/manual/en/html_node/Reduced-Binary-Seed-Bootstrap.html [3] https://guix.gnu.org/manual/devel/en/html_node/Full_002dSource-Bootstrap.html
