Fixing xen build

[Development of GNU Guix and the GNU System distribution.]

Xen currently does not build on x86_64-linux 
(http://ci.guix.gnu.org/search?query=xen+system:x86_64-linux). It will build if 
all instances of .note.gnu.build-id are quoted in xen/arch/x86/xen.lds.S. This 
is because of the hyphen in the symbol in the argument to ADDR. I tested this 
by placing the following into a file named xen.scm and running guix build -f 
xen.scm:

```(use-modules (gnu)             (guix build utils)             (guix 
packages)             (srfi srfi-1))(use-package-modules virtualization)

(define (transform-keyword-value search-key transformer input output)  (let 
((next-key (first input))        (next-value (second input))        (remaining 
(drop input 2)))    (if (eq? search-key next-key)        (append (list 
search-key (transformer next-value)) remaining output)        
(transform-keyword-value search-key transformer remaining (cons* next-key 
next-value output)

(define (add-quotation-phase original-phases)  `(modify-phases ,original-phases 
   (add-before 'patch 'quotation-phase      (lambda (arg . args)        (format 
#t "PWD: ~A~%" (getcwd))        (substitute* "xen/arch/x86/xen.lds.S"          
((".note.gnu.build-id")           "\".note.gnu.build-id\""))

(package  (inherit xen)  (arguments (transform-keyword-value #:phases 
add-quotation-phase (package-arguments xen) '(```

This produces the output path 
gnu/store/jzv2s5d01g7cz136xjcc9d27sdkjpdjd-xen-4.14.1 on this system:

```$ guix describeGeneration 6 Feb 25 2022 08:10:38 (current)  guix ed8d236    
repository URL: https://git.savannah.gnu.org/git/guix.git    branch: master    
commit: ed8d236eb938bc123a9ca8a079fea95edf7df6cd```

I'm not sure if this should be reported elsewhere. It might be that it's 
intended that symbols with hyphens need to be quoted when used in numeric 
expressions (which would make sense, it could be confused with subtraction) in 
which case Xen's script needs to be updated. It might be that ld is supposed to 
be able to understand that it's part of the symbol, in which case there's a bug 
in ld. But neither makes sense to me. Xen's package definition (including the 
source form) has not changed since the successful builds in December, so it 
worked before. Quoting the symbol seems reasonable based on how other languages 
behave, but .note.gnu.build-id seems like a standard practice based on some 
searches, and it's mentioned in ld's man page, so I would expect it to work 
easily. If the hyphen was going to cause issues, would the standard practice 
not use an underscore? I don't understand enough to know where else this should 
be reported, but the above code fixes the guix build which is what I care about 
most.

signature.asc
Description: OpenPGP digital signature

Re: Guix as a system vs as an end-user dev tool (re: Building a software toolchain that works)

[Aurora]

Zhu Zihao  writes:

> Make Guix available on Windows platform will be a painful job.
>
> 1. Windows doesn't allow user to create symbolic link without admin
> permission, which guix use intensely.
>
> 2. There's no RUNPATH for Windows DLL, so all dynamic library
> dependencies should in the same directory to allow Windows find it.
>

I think it's worth noting that the use of VM-based mechanisms in a
similar way to Docker for Desktop, suggested in the OP, would limit
these issues somewhat.

Native Guix would be exactly as troublesome as you suggest.

It might be possible to just rely on WSL.

Re: [bug#54394] [PATCH core-updates 00/12] Remove old GNU utilities from early bootstrap

[Jan Nieuwenhuizen]

Timothy Sample writes:

Hello!

> This is culmination of a lot of work, so I’m excited to be submitting
> it!  The main thrust of this series is to update Gash and Gash-Utils,
> and then remove most of the old GNU utilities we use during early
> bootstrap.  To elaborate, the current situation is that we climb a
> ladder through time: first we build Bash 2, then we build Bash 4, and
> finally we build Bash 5.  This is true for most GNU utilities: Sed,
> Gawk, Coreutils, etc.  The reason?  Until now, our Scheme
> implementations of those utilities (Gash and Gash-Utils) were very
> limited.  Bash 2 used to be a lot more useful then Gash, for example.
> Now, with recent releases of Gash and Gash-Utils, the Scheme utilities
> are, in general, capable of powering the builds of modern GNU software.

What an amazing piece of work.  Not only adding support for this in Gash
and Gash-Utils, also adding the "follow-up" for the Guix bootstrap.
Wow!

Not meaning to create more unwanted work for you, but I believe this
could do with a blog post.  I very much like your "climb a ladder
through time" description.

> To be clear, we still climb the time-ladder in a few cases.  The main
> ones are GCC, Glibc, and Binutils.  For example, we have to use GCC 2 to
> transition from TCC To GCC 4 (which is the last non-C++ GCC [1]).
> Fixing this would require quite a bit of TCC hacking

Yes, that needs work on MesCC, especially the Mes C Library, and TCC.
Possibly some work on mpz/gmp.  We definately want to jump from TCC to
GCC 4.6 (the last modular distribution) in the (near) future.

> We still use old versions of Gzip, Make, patch, and
> Gawk.  The fact that Gawk is still there disappoints me quite a bit, but
> ‘glibc-mesboot’ fails in a way that I just can’t figure out when
> building with Gash-Utils.

Yeah, I can imagine, especially after your (2nd?) rewrite of AWK.

> Gzip is not strictly necessary, but also pretty easy to replace.

and replacing possibly has quite a speed penalty.

> I’m not sure, but I think patch is only there
> to avoid using patches in ‘origin’ records during bootstrap.  We now
> have a way to do that, so it may no longer be necessary.

Yes, I believe patch is needed for the manual "apply-boot-patch" stages
to avoid adding a dependency on xz, created by adding patches to
"origin" records.

> Make will likely need to be rewritten in Scheme, but we have a head
> start: Potato Make [2].
>
> [2] https://github.com/spk121/potato-make

Interesting!

POSIX Makefile Parser

 Recipes can contain the following parser function

 (parse ...) reads a standard Makefile and creates
 rules based on its contents.

Building glibc without GNU Make may be pretty difficult, though.

> Here’s some detailed info for reviewers.

> Ludovic Courtès writes:
> > The patches all LGTM!  I think you can push them to ‘core-updates’ if
> > there are no objections.

> Pushed!  (Maybe I didn’t leave enough time for objections – apologies if
> it ends up requiring a revert!  I’ll be around to work through any
> issues this might cause.)

Patches LGTM, and thanks for pushing.

> The rest hopefully speak for themselves.  Sadly, the last one is
> something of a jumble.  Originally I wanted to remove the utilities one
> at a time, but it turns out there are interference effects.  I can’t
> remember the exact details now, but to get an idea, removing both old
> Bash and old Grep might work, but removing one and then the other (in
> either order) might not.  Hence, many utilities get updated in one
> commit.

That's quite understandable, and certainly acceptable.  Much of this
time-ladder was created by me, it was an enormous effort to find old
(and some ancient) versions of the tools that worked well together.
There are undocumented build dependencies all around.  Some newer
packages cannot be used to build older versions of other packages, etc.

So, while this kludge was a necessary evil, I'm extremely happy we can
now do without it.  We should work towards a bootstrap that depends
mainly on current packages, and strive to keep current packages
bootstrappable.

To summarize what you removed:

bzip2 (not an old version)
sed-1.18
sed-4.0.6
binutils-2.14 (really great!)
bash-2.05b
bash-4.4
gawk-3.0.0
tar-1.22
grep-2.0
coreutils-5.0
xz-5.0.0 (wow, that was difficult to build)

that's 11 old and ancient packages, if have counted correctly.

Greetings,
Janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Guix as a system vs as an end-user dev tool (re: Building a software toolchain that works)

[Josselin Poiret]

Hello everyone,

Just a small comment stemming from my time fiddling with Win32 API and
friends.

Zhu Zihao  writes:
> 2. There's no RUNPATH for Windows DLL, so all dynamic library
> dependencies should in the same directory to allow Windows find it.

There actually is an analogue for RUNPATH, because Windows applications
often vendor DLLs, and need a way to manipulate where they are loaded
from.  You can get a similar result using assembly manifests, see [1].

[1] https://docs.microsoft.com/en-us/windows/win32/sbscs/assembly-manifests

Best,
-- 
Josselin Poiret

Packaging go-github-com-hashicorp-vault-sdk -- ‘no Go files found’

[Maxime Devos]

(I sent this one previously to help-guix two days ago, and again one day
ago, but it seems that it did not end up in

so I'm resending to guix-devel this time)

Hi,

I'm trying to package go-github-com-hashicorp-vault-sdk.
I ran "guix import go github.com/hashicorp/vault/sdk", corrected
the commit and (for now) remove the inputs (many aren't packaged yet).

When I now try to build it, I end up with the following error
message (from "guix build"):

  phase `patch-source-shebangs' succeeded after 0.6 seconds
  starting phase `build'
  package github.com/hashicorp/vault/sdk: no Go files in 
/tmp/guix-build-go-github-com-hashicorp-vault-sdk-0.4.1.drv-0/src/github.com/hashicorp/vault/sdk
  Building 'github.com/hashicorp/vault/sdk' failed.

However, there is go stuff there!

$ ls 
/tmp/guix-build-go-github-com-hashicorp-vault-sdk-0.4.1.drv-4/src/github.com/hashicorp/vault/sdk
> database  framework  go.mod  go.sum  helper  logical  physical  plugin  queue 
>  README.md  version

There's a go.mod, go.sum and the subdirectories have .go files!

Does anyone know how to resolve this kind of problem?

Greetings,
Maxime.




(use-modules (guix packages)
	 (guix build-system go)
	 (guix git-download)
	 ((guix licenses) #:prefix license:))

(package
 (name "go-github-com-hashicorp-vault-sdk")
 (version "0.4.1")
 (source
  (origin
   (method git-fetch)
   (uri (git-reference
 (url "https://github.com/hashicorp/vault;)
 (commit (string-append "sdk/v" version
   (file-name (git-file-name name version))
   (sha256
(base32 "0vndpb0f490clvm18m1vx12cl3zqrn12q3zz0bd5i55czfdinsl5"
 (build-system go-build-system)
 (arguments
  '(#:import-path
"github.com/hashicorp/vault/sdk" #;"TODO no go files found"
#:unpack-path
"github.com/hashicorp/vault"))
 ;; TODO: add inputs
 (home-page "https://github.com/hashicorp/vault;)
 (synopsis "Vault SDK libs")
 (description
  "This package provides the @code{sdk} package which contains code useful for
developing Vault plugins.")
 (license license:mpl2.0))

;; phase `patch-source-shebangs' succeeded after 0.6 seconds
;; starting phase `build'
;; package github.com/hashicorp/vault/sdk: no Go files in /tmp/guix-build-go-github-com-hashicorp-vault-sdk-0.4.1.drv-0/src/github.com/hashicorp/vault/sdk
;; Building 'github.com/hashicorp/vault/sdk' failed.

;; However, there is go stuff there
;; ls /tmp/guix-build-go-github-com-hashicorp-vault-sdk-0.4.1.drv-4/src/github.com/hashicorp/vault/sdk
;; database  framework  go.mod  go.sum  helper  logical  physical  plugin  queue  README.md  version


signature.asc
Description: This is a digitally signed message part

Re: Guix as a system vs as an end-user dev tool (re: Building a software toolchain that works)

[Zhu Zihao]

Make Guix available on Windows platform will be a painful job.

1. Windows doesn't allow user to create symbolic link without admin
permission, which guix use intensely.

2. There's no RUNPATH for Windows DLL, so all dynamic library
dependencies should in the same directory to allow Windows find it.

3. Many developers on Guix, but less developer on Guile. I post some thread on 
guile-devel,
but there's few people will reply me. And the Windows support of Guile 3
is still missing.


Ryan Prior  writes:

> One side-thread in "Building a software toolchain that works" notes that Guix 
> faces challenges for adoption because it's
> not readily available to users of proprietary operating systems like macOS 
> and Windows.
>
> I've witnessed over the past decade that GNU/Linux development on other 
> platforms has become widespread, in large
> part due to the availability of the Docker for Desktop application which 
> packages a lightweight, automagically managed
> GNU/Linux virtual machine running the Docker daemon with Docker client 
> software built for the target platform.
>
> A user of Docker for Desktop on a proprietary OS can run "docker" commands 
> which transparently execute commands
> inside a GNU/Linux container, making building and testing software quite 
> convenient and reproducible without needing
> to set up cross-compile toolchains or spend time managing VM software.
>
> It makes absolute sense to me that Guix is not interested in building a 
> native distribution for the Windows or macOS
> toolchains. One of Guix System's unique strengths is its adherence to the GNU 
> FSDG and I don't think that's
> incompatible with making the Guix tools more generally available to end-user 
> devs hacking on software using a
> proprietary OS.
>
> Technically, I think we could use a similar approach to the Docker for 
> Desktop system: a "Guix for Desktop" installs
> software to create and manage a minimal Guix System virtual machine which 
> automatically updates and reconfigures
> itself, requiring no manual administration by the end-user. And it would 
> install a Guix client that connects to the Guix
> daemon running in the VM using a shared socket, enabling users to incorporate 
> Guix transparently into their workflows.
>
> I think this would be a compromise for certain, the same way it is for Emacs 
> and other GNU flagship projects that run on
> non-free systems. On the one hand, it serves to make those systems more 
> valuable, which undermines our cause. But on
> the other hand it provides a major on-ramp to free software and superior 
> build tooling, positively impacting the
> practical freedoms available to the end-users who adopt Guix.
>
> wdyt?


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao


signature.asc
Description: PGP signature