Jean Louis writes:
> * Jan Nieuwenhuizen <jann...@gnu.org> [2021-04-08 16:43]:
>> Martin writes:
>>
>> > Maybe freedom in "free software" shouldn't require from the code to be
>> > open neither. Let's just blindly trust some saint developers who
>> > cannot even control their own binaries. Actually today we are closer
>> > and closer to that sad scenario like never before in the history,
>> > because in fact most of the open-source and GNU "free software"
>> > nowadays base on blackboxed binary seeds that cannot be verified by
>> > the users not even by the core developers.
>>
>> The bootstrappable project, GNU Mes and GNU Guix are working to fix that
>>
>>
>> https://guix.gnu.org/en/blog/2020/guix-further-reduces-bootstrap-seed-to-25/
>> https://fosdem.org/2021/schedule/event/gnumes/
>
> Jan, in regards to this, how to start with first stage, how are you
> entering it?
> Will you use for example Emacs and hexl-mode to enter bytes?
I am not, I'm downloading the initial 357-byte binary from
'bootstrap-seeds'
https://raw.githubusercontent.com/oriansj/bootstrap-seeds/master/POSIX/x86/hex0-seed
and then run this script
https://github.com/oriansj/stage0-posix/blob/master/x86/mescc-tools-seed-kaem.kaem
See below for an overview and the next steps.
Note that we are only in the 3rd iteration of removing binary seeds from
the bootstrap; so in a way this project is just starting. A year ago,
Ludovic Courtès already prototyped a step to build the first packages in
the initial ramdisk, such as to remove dependencies on the Guix build
daemon.
Also note that major forces are going into "unbootstrappable" projects,
that may even undo the small steps that we have just achieved. It has
become much harder to bootstrap GCC from source then 10 years ago: we
need unmaintained earlier versions of GCC to do so. Dependencies on
Python, Rust are moving up the package graph.
I choose not to believe that "nobody cares" about bootstrappability,
i.e., binary-blob-free computing, I just think we need to raise
awareness about it if we don't want to lose it.
Greetings,
Janneke
The bootstrappable effort to bootstrap a full GNU/Linux system from
source consists mainly of three projects: Stage0 by Jeremiah Orians
and others, GNU Mes by Janneke and others, and the actual bootstrap
packaging and deployment in GNU Guix:
https://guix.gnu.org/en/manual/en/html_node/Bootstrapping.html#Bootstrapping
Efforts are ongoing to implement such a bootstrap in other
distributions.
The reduced binary seed bootstrap was developed in GNU Guix in and is
used in production
https://guix.gnu.org/blog/2019/guix-reduces-bootstrap-seed-by-50/
https://guix.gnu.org/blog/2020/guix-further-reduces-bootstrap-seed-to-25
There have been several efforts to avoid using a package manager such
as GNU Guix and most hove been abandoned. Possibly because it's fun
to start such a thing, and pretty easy even to do the first steps
without isolated builds, (cross-)build, or packaging support.
The first working development version of 'wip-full-source-bootstrap'
in Guix triggered a yet another effort like that, the live-bootstrap
https://github.com/fosslinux/live-bootstrap
The full source bootstrap is not in production; a development preview
lives here
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/commencement.scm?h=wip-full-source-bootstrap
The package graph for this bootstrap; the bottom of the guix package
graph as generated by 'guix graph' looks like this
https://git.savannah.gnu.org/cgit/mes.git/plain/doc/talks/fosdem21/gcc-core-mesboot0-fsb-no-guile-unversioned.png?h=wip-talks
which can be split into several stages
https://git.savannah.gnu.org/cgit/mes.git/plain/doc/talks/fosdem21/stage-0.png?h=wip-talks
https://git.savannah.gnu.org/cgit/mes.git/plain/doc/talks/fosdem21/stage-1.png?h=wip-talks
https://git.savannah.gnu.org/cgit/mes.git/plain/doc/talks/fosdem21/stage-2.png?h=wip-talks
https://git.savannah.gnu.org/cgit/mes.git/plain/doc/talks/fosdem21/stage-mes.png?h=wip-talks
it may be easier to read that from the slides of my FOSDEM'21 talk
https://fosdem.org/2021/schedule/event/gnumes/attachments/slides/4531/export/events/attachments/gnumes/slides/4531/gnu_mes_fosdem21.pdf
You can see it all starts with 'bootstrap-seeds'. You can "build" that
package (the seeds are just a download, of course) by (checking out
the guix wip-full-source-bootstrap branch, building it) and, doing
$ ./pre-inst-env guix build -e '(@@ (gnu packages commencement)
bootstrap-seeds)'
/gnu/store/sg0fxjd0pyfn5sm0m33227i3rz32l6z6-bootstrap-seeds-1.0.0
--8<---------------cut here---------------start------------->8---
(define m2-planet-boot
;; The initial bootstrap package: no binary inputs except for a 357 byte
;; binary seed: `x86/hex0-seed'.
[..]
(let* ((bootstrap-seeds (assoc-ref %build-inputs "bootstrap-seeds"))
(mescc-tools-seed (assoc-ref %build-inputs "mescc-tools-seed"))
(mescc-tools (assoc-ref %build-inputs "mescc-tools"))
(source (assoc-ref %build-inputs "source"))
(tar (assoc-ref %build-inputs "bootar"))
(out (assoc-ref %outputs "out"))
(bindir (string-append out "/bin")))
(invoke "tar" "xvf" mescc-tools-seed)
(chdir
(string-append "mescc-tools-seed-" ,mescc-tools-seed-version))
(copy-recursively bootstrap-seeds "bootstrap-seeds")
(invoke "tar" "xvf" mescc-tools)
(rmdir "mescc-tools")
(symlink (string-append "mescc-tools-" ,mescc-tools-version)
"mescc-tools")
(invoke "tar" "xvf" source)
(rmdir "M2-Planet")
(symlink (string-append "m2-planet-" ,version) "M2-Planet")
(mkdir-p bindir)
;; XXX TODO: Update mescc-tools-seed
(copy-file "M2-Planet/test/common_x86/x86_defs.M1" "x86/x86_defs.M1")
(with-directory-excursion "x86"
(invoke "../bootstrap-seeds/POSIX/x86/kaem-optional-seed"
"mescc-tools-seed-kaem.kaem")
(invoke "../bootstrap-seeds/POSIX/x86/kaem-optional-seed"
"mescc-tools-mini-kaem.kaem")
(invoke "../bootstrap-seeds/POSIX/x86/kaem-optional-seed"
"mescc-tools-full-kaem.kaem"))
(with-directory-excursion "bin"
(install-file "hex2" bindir)
(install-file "M1" bindir)
(install-file "blood-elf" bindir)
(install-file "kaem" bindir)
(install-file "get_machine" bindir)
(install-file "M2-Planet" bindir))
#t)))))))
--8<---------------cut here---------------end--------------->8---
and you'll see the 357-byte hex0-seed in there. This is used to build
the 'm2-planet-boot' package which uses kaem scripts (any shell will
do) to build hex0, hex1, M0, M1, hex2 and m2-planet.
As you can see, the very first step is running this script
https://raw.githubusercontent.com/oriansj/mescc-tools-seed/master/x86/mescc-tools-seed-kaem.kaem
which uses this 357-byte seed
https://raw.githubusercontent.com/oriansj/bootstrap-seeds/master/POSIX/x86/hex0-seed
built using hex0 itself from
https://raw.githubusercontent.com/oriansj/bootstrap-seeds/master/POSIX/x86/hex0_x86.hex0
hex0 can be approximated with:
sed 's/[;#].*$//g' $input_file | xxd -r -p > $output_file
kaem-optional-seed can be replaced by any shell of your choice as it
is just a minimal shell written in hex0.
Then onto
https://raw.githubusercontent.com/oriansj/mescc-tools-seed/master/x86/mescc-tools-mini-kaem.kaem
and
https://raw.githubusercontent.com/oriansj/mescc-tools-seed/master/x86/mescc-tools-full-kaem.kaem
and so on; or simply:
$ ./pre-inst-env guix build -e '(@@ (gnu packages commencement)
m2-planet-boot)'
/gnu/store/qxsd6zg910zjgwsm13c39hys2g931ij8-m2-planet-boot-1.7.0-31-g358b6cf
then onto mes-boot
$ ./pre-inst-env guix build -e '(@@ (gnu packages commencement) mes-boot)'
/gnu/store/dfwkand25k6nsmrk85p8hsa1yq74y314-mes-boot-0.22-305-g2ab4c5c67
and so on. In fact, each time you build or install a package using
Guix, these (bootstrap) dependencies are resolved and -if necessary- all
built automagically.
--
Jan Nieuwenhuizen <jann...@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com
