Re: Core updates status

[Efraim Flashner]

On Thu, May 09, 2024 at 11:38:07AM -0400, Maxim Cournoyer wrote:
> Hi Andreas,
> 
> Andreas Enge  writes:
> 
> > Hello,
> >
> > Am Mon, May 06, 2024 at 10:47:13AM +0200 schrieb Josselin Poiret:
> >> Maxim Cournoyer  writes:
> >> > I don't mind too much; when we re-enable the change we should add a
> >> > phase to the gnu-build-system automatically deleting/moving the libtool
> >> > archives. so that we're covered.
> >> 
> >> I agree, although we'll have to be careful since some packages might
> >> need them if they don't use pkg-config!
> >
> > I am a little bit confused by the suggestion; you mean removing all .la
> > files from all packages?
> 
> Yes!
> 
> > I thought they were there for a reason, and usually recorded the
> > dependencies. For instance, doing a "guix build mpc" and looking at
> > libmpc.la, my impression is that I see correct information.
> 
> libtool records the *transitive* dependencies, as would be useful when
> doing static builds but not shared builds, as the dependencies are
> already recorded in the RUNPATH of the built ELF binaries.  For our
> overwhelming common case (shared libraries) on GNU+Linux, these files
> are thus unnecessary and when used they lead to over-linking (due to
> listing the whole *transitive* dependencies) for shared library.  That
> in turn muddles the dependency graph (as more references get retain in
> the RUNPATH) and forces us to propagate more stuff.
> 
> > Why would
> > one want to force upstream to add a pkgconfig dependency additionally
> > to libtool? Or do I misunderstand the suggestion?
> 
> I hope my explanation above make it clear why libtool for our common
> case of building shared libraries is not useful.
> 
> It could be useful when building shared libraries or targeting some
> systems such as Android, which linker is very dumb or so I've heard. My
> suggestion is to delete them by default, or move them to a 'libtool'
> output when one is available (similarly to how we handle debug symbol
> files).
> 
> -- 
> Thanks,
> Maxim

It would be interesting to move them to a 'static' or a 'libtool' output
by default.

-- 
Efraim Flashner  רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Core updates status

[Efraim Flashner]

On Wed, May 08, 2024 at 11:03:02AM +0200, Josselin Poiret wrote:
> 
> The one thing that we need to do right now is update glibc 2.39 with all
> the fixes from the upstream release/2.39/master branch.  I don't think
> we've done this before significantly, but since we have an occasion this
> time we might as well.  We can't really use git-fetch for glibc, so imo
> the only feasible option is like what Debian does [1], which is keeping
> a diff of the 2.39 tag and the release branch and applying it as a
> patch.  We'll then probably need to add autotools to glibc builds, but
> this is doable even in commencement because we have them already
> available at that point.
> 
> The own downside of this is that the patch name will not include the
> fixed CVEs, so guix lint won't be aware that the CVEs have been patched.
> 
> [1] 
> https://salsa.debian.org/glibc-team/glibc/-/blob/sid/debian/patches/git-updates.diff
> 
> WDYT?
> 
> Best,
> -- 
> Josselin Poiret

I think that's a good idea, and probably something we should do for the
other copies of glibc we have.  We can also use the package-property
lint-hidden-cves to list the CVEs which are covered by the diff, and
that'll hide the CVEs from 'guix lint'.



-- 
Efraim Flashner  רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Core updates status

[Andreas Enge]

Thanks, Felix and Maxim, for your explanations!

Andreas

Re: Core updates status

[Maxim Cournoyer]

Hi Josselin,

Josselin Poiret  writes:

> Hi Ludo,
>
> Ludovic Courtès  writes:
>
>> I’m in favor of whatever allows us to move forward more quickly, so
>> temporarily stashing away the pkgconf changes sounds good to me.
>>
>> In that case, when time permits, could you push a ‘core-updates-new’ (?)
>> branch, (partially) rebased and without the pkgconf changes, and a
>> separate ‘wip-pkgconf’ branch?  Does that seem doable to you?
>
> I did that partially yesterday, moved the old borked core-updates to
> old-core-updates and pushed the cleaned-up version at core-updates.  I
> haven't pushed the pkgconf patches anywhere yet, but we should probably
> focus on c-u for now and worry about that later.

All pkgconf related patches except the one effecting the actual
pkg-config -> pkgconf aliasing should be safe to merge already.

-- 
Thanks,
Maxim

Re: Core updates status

[Maxim Cournoyer]

Hi Andreas,

Andreas Enge  writes:

> Hello,
>
> Am Mon, May 06, 2024 at 10:47:13AM +0200 schrieb Josselin Poiret:
>> Maxim Cournoyer  writes:
>> > I don't mind too much; when we re-enable the change we should add a
>> > phase to the gnu-build-system automatically deleting/moving the libtool
>> > archives. so that we're covered.
>> 
>> I agree, although we'll have to be careful since some packages might
>> need them if they don't use pkg-config!
>
> I am a little bit confused by the suggestion; you mean removing all .la
> files from all packages?

Yes!

> I thought they were there for a reason, and usually recorded the
> dependencies. For instance, doing a "guix build mpc" and looking at
> libmpc.la, my impression is that I see correct information.

libtool records the *transitive* dependencies, as would be useful when
doing static builds but not shared builds, as the dependencies are
already recorded in the RUNPATH of the built ELF binaries.  For our
overwhelming common case (shared libraries) on GNU+Linux, these files
are thus unnecessary and when used they lead to over-linking (due to
listing the whole *transitive* dependencies) for shared library.  That
in turn muddles the dependency graph (as more references get retain in
the RUNPATH) and forces us to propagate more stuff.

> Why would
> one want to force upstream to add a pkgconfig dependency additionally
> to libtool? Or do I misunderstand the suggestion?

I hope my explanation above make it clear why libtool for our common
case of building shared libraries is not useful.

It could be useful when building shared libraries or targeting some
systems such as Android, which linker is very dumb or so I've heard. My
suggestion is to delete them by default, or move them to a 'libtool'
output when one is available (similarly to how we handle debug symbol
files).

-- 
Thanks,
Maxim

Re: Core updates status

[Development of GNU Guix and the GNU System distribution.]

Hi Andreas,

On Wed, May 08 2024, Andreas Enge wrote:

> I am a little bit confused by the suggestion; you mean removing all
> .la files from all packages?

I don't mean to answer for Josselin but at least in Debian, which seems
to be acceptable as a reference from time to time, there has been a
long-standing practice not to ship .la files. [1]

There are several exceptions to that rule, chiefly when the files are
for dynamically loaded modules used only by a particular program. The
dynamic loader library libltdl (which ships with Libtool but is losing
popularity) requires them.

My understanding is that pkg-config [5] and pkgconf [6] do more or less
the same thing.

In one of Debian's QA tools called Lintian, we tolerated .la files but
issued fault indicators for shipping prerequisite information [2] or for
inconsistent libdirs. [3]

The exact heuristics are available here. [4]

Kind regards
Felix

[1] https://wiki.debian.org/ReleaseGoals/LAFileRemoval
[2] 
https://salsa.debian.org/lintian/lintian/-/blob/master/tags/n/non-empty-dependency_libs-in-la-file.tag
[3] 
https://salsa.debian.org/lintian/lintian/-/blob/master/tags/i/incorrect-libdir-in-la-file.tag
[4] 
https://salsa.debian.org/lintian/lintian/-/blob/master/lib/Lintian/Check/BuildSystems/Libtool/LaFile.pm
[5] https://en.wikipedia.org/wiki/Pkg-config
[6] https://github.com/pkgconf/pkgconf

Re: Core updates status

[Andreas Enge]

Hello,

Am Mon, May 06, 2024 at 10:47:13AM +0200 schrieb Josselin Poiret:
> Maxim Cournoyer  writes:
> > I don't mind too much; when we re-enable the change we should add a
> > phase to the gnu-build-system automatically deleting/moving the libtool
> > archives. so that we're covered.
> 
> I agree, although we'll have to be careful since some packages might
> need them if they don't use pkg-config!

I am a little bit confused by the suggestion; you mean removing all .la
files from all packages? I thought they were there for a reason, and
usually recorded the dependencies. For instance, doing a "guix build mpc"
and looking at libmpc.la, my impression is that I see correct information.
Why would one want to force upstream to add a pkgconfig dependency
additionally to libtool? Or do I misunderstand the suggestion?

Andreas

Re: Core updates status

[Josselin Poiret]

Hi Ludo,

Ludovic Courtès  writes:

> I’m in favor of whatever allows us to move forward more quickly, so
> temporarily stashing away the pkgconf changes sounds good to me.
>
> In that case, when time permits, could you push a ‘core-updates-new’ (?)
> branch, (partially) rebased and without the pkgconf changes, and a
> separate ‘wip-pkgconf’ branch?  Does that seem doable to you?

I did that partially yesterday, moved the old borked core-updates to
old-core-updates and pushed the cleaned-up version at core-updates.  I
haven't pushed the pkgconf patches anywhere yet, but we should probably
focus on c-u for now and worry about that later.

> It would be great if you could also explain at which commit you started
> rebasing ‘core-updates’¹ and which method/script you used.

I started rebasing at the very first commit of core-updates that wasn't
reachable from master (it is a rebase after all).  I wrote a little
script (attached) that basically filters `git rev-list` output, removing
commits for which a commit with the same name appears on master after
1.4.

#!/usr/bin/env -S guile -s
!#

(use-modules (ice-9 popen)
 (ice-9 textual-ports))

(define (check name)
  (let* ((input+output (pipe))
 (pid (spawn "git"
 (list "git"
   "log"
   "-n1"
   "v1.4.0..origin/master"
   "-F"
   (string-append "--grep=" name))
 #:output (cdr input+output
(close-port (cdr input+output))
(define res
  (let loop ((n 0))
(if (eof-object? (get-line (car input+output)))
n
(loop (+ 1 n)
(close-port (car input+output))
(waitpid pid)
(> res 0)))

(check "gnu: Add glirc.")

(let ((port (open-input-pipe "git rev-list --no-merges 
origin/master..origin/core-updates --pretty=oneline")))
  (define res
(let loop ((acc '()))
  (let ((line (get-line port)))
(if (eof-object? line)
acc
(loop (cons line acc))
  (close-pipe port)
  (for-each
   (lambda (line)
 (unless (check (string-drop line 41))
   (format #t "~a~%" line)))
   res))

;; (unless (check (string-drop line 41))
;;   (format #t "~a~%" line))

The one thing that we need to do right now is update glibc 2.39 with all
the fixes from the upstream release/2.39/master branch.  I don't think
we've done this before significantly, but since we have an occasion this
time we might as well.  We can't really use git-fetch for glibc, so imo
the only feasible option is like what Debian does [1], which is keeping
a diff of the 2.39 tag and the release branch and applying it as a
patch.  We'll then probably need to add autotools to glibc builds, but
this is doable even in commencement because we have them already
available at that point.

The own downside of this is that the patch name will not include the
fixed CVEs, so guix lint won't be aware that the CVEs have been patched.

[1] 
https://salsa.debian.org/glibc-team/glibc/-/blob/sid/debian/patches/git-updates.diff

WDYT?

Best,
-- 
Josselin Poiret


signature.asc
Description: PGP signature

Re: Core updates status

[Ludovic Courtès]

Hi Josselin and all,

Josselin Poiret  skribis:

> Maxim Cournoyer  writes:
>> Josselin Poiret  writes:

[...]

>>> I'm worried this will keep accumulating a bunch of world rebuilds,
>>> slowing down c-u some more.  I'd vote to keep the pkgconf switch for
>>> later and focus on merging the rest of what c-u has to offer.
>>
>> I don't mind too much; when we re-enable the change we should add a
>> phase to the gnu-build-system automatically deleting/moving the libtool
>> archives. so that we're covered.
>
> I agree, although we'll have to be careful since some packages might
> need them if they don't use pkg-config!

I’m in favor of whatever allows us to move forward more quickly, so
temporarily stashing away the pkgconf changes sounds good to me.

In that case, when time permits, could you push a ‘core-updates-new’ (?)
branch, (partially) rebased and without the pkgconf changes, and a
separate ‘wip-pkgconf’ branch?  Does that seem doable to you?

It would be great if you could also explain at which commit you started
rebasing ‘core-updates’¹ and which method/script you used.

Thanks for this thankless work!

Ludo’.

¹ Ideally we’d preserve commits that predate the duplicated commits.
  That way, we’d also preserve signatures as well as commit references
  that appear in commit logs.

Re: Core updates status

[Josselin Poiret]

Hi Maxim,

Maxim Cournoyer  writes:

> Hi Josselin,
>
> Josselin Poiret  writes:
>
> [...]
>
>> However, as you can see, these are non-local failures: build failures
>> have to be fixed in a dependency, which incurs a lot of rebuilding.
>> I've fixed a couple of them locally, but here's a nasty one I just got
>> stuck on: curl has a .la file with "-lnettle -lhogweed -lgssapi_krb5",
>> which leads to a build failure for... flatpak :( So fixing this properly
>> would lead to a world rebuild.
>>
>> I'm worried this will keep accumulating a bunch of world rebuilds,
>> slowing down c-u some more.  I'd vote to keep the pkgconf switch for
>> later and focus on merging the rest of what c-u has to offer.
>
> I don't mind too much; when we re-enable the change we should add a
> phase to the gnu-build-system automatically deleting/moving the libtool
> archives. so that we're covered.

I agree, although we'll have to be careful since some packages might
need them if they don't use pkg-config!

Best,
-- 
Josselin Poiret


signature.asc
Description: PGP signature

Re: Core updates status

[Maxim Cournoyer]

Hi Josselin,

Josselin Poiret  writes:

[...]

> However, as you can see, these are non-local failures: build failures
> have to be fixed in a dependency, which incurs a lot of rebuilding.
> I've fixed a couple of them locally, but here's a nasty one I just got
> stuck on: curl has a .la file with "-lnettle -lhogweed -lgssapi_krb5",
> which leads to a build failure for... flatpak :( So fixing this properly
> would lead to a world rebuild.
>
> I'm worried this will keep accumulating a bunch of world rebuilds,
> slowing down c-u some more.  I'd vote to keep the pkgconf switch for
> later and focus on merging the rest of what c-u has to offer.

I don't mind too much; when we re-enable the change we should add a
phase to the gnu-build-system automatically deleting/moving the libtool
archives. so that we're covered.

-- 
Thanks,
Maxim

Re: Core updates status

[Josselin Poiret]

Hi everyone,

I've just cleaned up the c-u branch locally (removing all the duplicated
commits) and rebased it on master since we're going to have to rewrite
history anyways.  I'm trying to build a bit with it to double check that
nothing got messed up in the process, so I haven't pushed that anywhere
yet.

Efraim Flashner  writes:

> Seeing that librsvg has rust-pango (and some others) in cargo-inputs,
> and therefore needs pango to build, I think it's safe to move pango (and
> others as needed) to propagated inputs. Make sure to make the necessary
> changes to librsvg-2.40 also.
>
> I haven't had any problems building the rust based librsvg on
> core-updates, so I'd suggest leaving that alone for now.

Basically, the issue is that libtool+pkg-config used to overlink,
leading to libtool dependencies being fulfilled without needing them to
be actually present as inputs.  Now, because there are recent commits
replacing pkg-config with pkgconf, libtool is requiring more libraries
than needed at link time.  The option then is to just remove .la
(libtool archive) files in package outputs, so that their dependents
don't try to use them (and it usually just works).

Removing them in librsvg makes guile-rsvg build again.

However, as you can see, these are non-local failures: build failures
have to be fixed in a dependency, which incurs a lot of rebuilding.
I've fixed a couple of them locally, but here's a nasty one I just got
stuck on: curl has a .la file with "-lnettle -lhogweed -lgssapi_krb5",
which leads to a build failure for... flatpak :( So fixing this properly
would lead to a world rebuild.

I'm worried this will keep accumulating a bunch of world rebuilds,
slowing down c-u some more.  I'd vote to keep the pkgconf switch for
later and focus on merging the rest of what c-u has to offer.

WDYT?
-- 
Josselin Poiret


signature.asc
Description: PGP signature

Re: Core updates status

[Efraim Flashner]

On Fri, Apr 26, 2024 at 01:56:12PM +0100, Steve George wrote:
> Hi,
> 
> On 25 Apr, Kaelyn wrote:
> > Hi,
> > 
> > On Tuesday, April 23rd, 2024 at 11:08 PM, Steve George  
> > wrote:
> (...)
> > > - guile-rsvg failing
> > > - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70537
> > > - I'm able to build librsvg@2.56.4 but not guile-rsvg
> > > - guile-rsvg@2.18.1 / guile2.2-rsvg
> > > - guile-rsvg builds on master - connected?
> > 
> > I've started looking at this build failure this morning, and my initial 
> > impressions are that a propagated-input is missing. I tried building 
> > guile-rsvg from core-updates and the build failed with three missing pango 
> > libraries (ld could not find -lpangocairo-1.0, -lpangoft2-1.0, and 
> > -lpango-1.0). I tried moving pango from the inputs to the propagated-inputs 
> > for librsvg, and that fixed the build of guile-rsvg. I'm just not sure if 
> > that is the right place to propagate it; I haven't yet traced where the 
> > pango libraries are being added to the linking command, as I've checked the 
> > pkgconfig files for librsvg, cairo, and a couple of other packages and not 
> > seen references to pango (which is what makes me think the pango 
> > propagated-input is needed elsewhere, and not actually in librsvg--but I 
> > also don't know Rust or what dependencies the rust code in librsvg has).
> 
> That's great, thanks for taking a look. Efraim mentioned that there would be 
> Rust changes for it.

Seeing that librsvg has rust-pango (and some others) in cargo-inputs,
and therefore needs pango to build, I think it's safe to move pango (and
others as needed) to propagated inputs. Make sure to make the necessary
changes to librsvg-2.40 also.

I haven't had any problems building the rust based librsvg on
core-updates, so I'd suggest leaving that alone for now.

-- 
Efraim Flashner  רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Core updates status

[Steve George]

Hi,

On 25 Apr, Kaelyn wrote:
> Hi,
> 
> On Tuesday, April 23rd, 2024 at 11:08 PM, Steve George  
> wrote:
(...)
> > - guile-rsvg failing
> > - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70537
> > - I'm able to build librsvg@2.56.4 but not guile-rsvg
> > - guile-rsvg@2.18.1 / guile2.2-rsvg
> > - guile-rsvg builds on master - connected?
> 
> I've started looking at this build failure this morning, and my initial 
> impressions are that a propagated-input is missing. I tried building 
> guile-rsvg from core-updates and the build failed with three missing pango 
> libraries (ld could not find -lpangocairo-1.0, -lpangoft2-1.0, and 
> -lpango-1.0). I tried moving pango from the inputs to the propagated-inputs 
> for librsvg, and that fixed the build of guile-rsvg. I'm just not sure if 
> that is the right place to propagate it; I haven't yet traced where the pango 
> libraries are being added to the linking command, as I've checked the 
> pkgconfig files for librsvg, cairo, and a couple of other packages and not 
> seen references to pango (which is what makes me think the pango 
> propagated-input is needed elsewhere, and not actually in librsvg--but I also 
> don't know Rust or what dependencies the rust code in librsvg has).

That's great, thanks for taking a look. Efraim mentioned that there would be 
Rust changes for it.

Steve / Futurile

Re: Core updates status

[Kaelyn]

Hi,

On Tuesday, April 23rd, 2024 at 11:08 PM, Steve George  
wrote:

> 
> 
> Hi,
> 
> We're trying to stabilise and merge core-updates, help definitely wanted!
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70456
> 
> So far the main blockers are:
> 
> - guile-rsvg failing
> - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70537
> - I'm able to build librsvg@2.56.4 but not guile-rsvg
> - guile-rsvg@2.18.1 / guile2.2-rsvg
> - guile-rsvg builds on master - connected?

I've started looking at this build failure this morning, and my initial 
impressions are that a propagated-input is missing. I tried building guile-rsvg 
from core-updates and the build failed with three missing pango libraries (ld 
could not find -lpangocairo-1.0, -lpangoft2-1.0, and -lpango-1.0). I tried 
moving pango from the inputs to the propagated-inputs for librsvg, and that 
fixed the build of guile-rsvg. I'm just not sure if that is the right place to 
propagate it; I haven't yet traced where the pango libraries are being added to 
the linking command, as I've checked the pkgconfig files for librsvg, cairo, 
and a couple of other packages and not seen references to pango (which is what 
makes me think the pango propagated-input is needed elsewhere, and not actually 
in librsvg--but I also don't know Rust or what dependencies the rust code in 
librsvg has).

Cheers,
Kaelyn
> This blocks further progress
> 
> What builds so far:
> 
> - gcc-toolchain and all the dependents from commencement.scm
> ./pre-inst-env guix build --no-substitutes gcc-toolchain
> 
> - bunch of the basic - but blocked on guile-rsvg
> ./pre-inst-env guix system --no-substitutes vm 
> gnu/system/examples/bare-bones.tmpl
> 
> Other potential issues:
> 
> - 45885 libpng non-deterministic build
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45885
> won't build due to block on pango -
> 
> - 58719 [core-updates]: build failure for file on i686
> https://ci.guix.gnu.org/build/4057809/details
> 
> - 40316 [core-updates] Nss not reproducible
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316
> confirmed
> 
> - 68270 libstdc++-boot0.x86_64 is broken
> - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68270
> 
> - 39415 [core-updates] Removing SSL patches in CMake and Kodi - help wanted
> - check if they are there and remove?
> - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39415
> 
> 
> This is building from 4a0e6e3895cefe7c2999c22e56fe9b3dbca97f55 which includes 
> the last merge from master.
> 
> Thanks,
> 
> Steve

Re: Core updates status

[Christina O'Donnell]

Hi Steve,


It would be good to confirm this one:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316


Still fails to reproduce with those changes applied.

The culprit is in nss/cmd/shlibsign/shlibsign.c:

shlibSignHMAC generates a new key-pair each time it's run:

    /* Generate a DSA key pair */
    logIt("Generate an HMAC key ... \n");
    crv = pFunctionList->C_GenerateKey(hRwSession, ,
   hmacKeyTemplate,
PR_ARRAY_SIZE(hmacKeyTemplate),
   );

Three options:
 1. Disable library signing entirely.
 2. Seed the generation to be deterministic.
 3. Drop in a HMAC key-pair and patch the code to use that instead of 
generating.


2 and 3 defeat the point of the cryptographically secure supply chain as 
the private key can be obtained deterministically, so my vote would be 
simply  to not sign the libraries (1), which would be easier to 
maintain. We're not the primary distributor and users can verify our 
distribution of nss by running `guix challenge` anyway.



It looks like Zhen Junjie applied two patches to fix NSS cross-compilation on 
Master [0]


Building everything cross-compiled to ARM now.

Kind regards,

Christina

Re: Core updates status

[Christina O'Donnell]

Okay, I'll let you know as soon as I know.

On 24/04/2024 14:17, Steve George wrote:

Hi,

You just need to checkout core-updates and then 'start building'!

It would be good to confirm this one:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316

It looks like Zhen Junjie applied two patches to fix NSS cross-compilation on 
Master [0]

Maybe master and core-updates have diverged - I haven't had time to look.

To check for reproducibility do a normal build, and if that's successful do the 
same build with `--check`

Thanks,

Steve

[0] 
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=fb86bf658a9374d41b05c5e586bfc6a3150cc3cb
  and 
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=452e7673bfeb0a14cecb8e760dda2c436aa69047


On 24 Apr, Christina O'Donnell wrote:

Hi Steve,

On 24/04/2024 07:08, Steve George wrote:

Hi,

We're trying to stabilise and merge core-updates, help definitely wanted!

I'd love to help! Any of these issues novice-friendly?

Will there be a point release after core-updates is merged?

Kind regards,

Christina

Re: Core updates status

[Steve George]

Hi,

You just need to checkout core-updates and then 'start building'!

It would be good to confirm this one:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316

It looks like Zhen Junjie applied two patches to fix NSS cross-compilation on 
Master [0]

Maybe master and core-updates have diverged - I haven't had time to look.

To check for reproducibility do a normal build, and if that's successful do the 
same build with `--check`

Thanks,

Steve 

[0] 
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=fb86bf658a9374d41b05c5e586bfc6a3150cc3cb
 and 
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=452e7673bfeb0a14cecb8e760dda2c436aa69047


On 24 Apr, Christina O'Donnell wrote:
> Hi Steve,
> 
> On 24/04/2024 07:08, Steve George wrote:
> > Hi,
> > 
> > We're trying to stabilise and merge core-updates, help definitely wanted!
> 
> I'd love to help! Any of these issues novice-friendly?
> 
> Will there be a point release after core-updates is merged?
> 
> Kind regards,
> 
> Christina
> 
> 

Re: Core updates status

[Christina O'Donnell]

Hi Steve,

On 24/04/2024 07:08, Steve George wrote:

Hi,

We're trying to stabilise and merge core-updates, help definitely wanted!


I'd love to help! Any of these issues novice-friendly?

Will there be a point release after core-updates is merged?

Kind regards,

Christina

Core updates status

[Steve George]

Hi,

We're trying to stabilise and merge core-updates, help definitely wanted!

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70456

So far the main blockers are:

- guile-rsvg failing
  - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=70537
  - I'm able to build librsvg@2.56.4 but not guile-rsvg
  - guile-rsvg@2.18.1 / guile2.2-rsvg
  - guile-rsvg builds on master - connected?

This blocks further progress 

What builds so far: 

  - gcc-toolchain and all the dependents from commencement.scm
  ./pre-inst-env guix build --no-substitutes gcc-toolchain

  - bunch of the basic - but blocked on guile-rsvg
  ./pre-inst-env guix system --no-substitutes vm 
gnu/system/examples/bare-bones.tmpl

Other potential issues:

- 45885  libpng non-deterministic build
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45885
  won't build due to block on pango - 

- 58719 [core-updates]: build failure for file on i686
  https://ci.guix.gnu.org/build/4057809/details

- 40316 [core-updates] Nss not reproducible
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=40316
  confirmed 

- 68270  libstdc++-boot0.x86_64 is broken
  - https://debbugs.gnu.org/cgi/bugreport.cgi?bug=68270

- 39415  [core-updates] Removing SSL patches in CMake and Kodi - help wanted
- check if they are there and remove?
- https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39415


This is building from  4a0e6e3895cefe7c2999c22e56fe9b3dbca97f55 which includes 
the last merge from master.

Thanks,

Steve

Re: 'core-updates' status

[Manolis Ragkousis]

On 08/26/2017 03:04 PM, Ludovic Courtès wrote:
> Leo Famulari  skribis:
> 
>> On Fri, Aug 25, 2017 at 08:34:21PM +0200, Marius Bakke wrote:
>>> 'core-updates' has finished building on x86_64 on i686, and the grafting
>>> failures should now be fixed.  Are we ready to merge this branch? :-)
>>
>> I think it's ready. There are a handful of failing packages left, but I
>> assume they will be fixed by their users after the merge :)
> 
> Seconded!  I upgraded my whole profile a couple of days ago, and I’m
> still alive.  :-)
> 
> I’ll look at the cross-compilation issue, but it’s too late to realize
> there’s a problem in this area I’m afraid.
> 
> Ludo’.
> 

It's broken both in master and core-updates, so it must be a while this
problem went unnoticed . I will file a bug report per Leo's suggestion.
I am currently trying to find when/where did it break.

Thank you,
Manolis

Re: 'core-updates' status

[Ludovic Courtès]

Leo Famulari  skribis:

> On Fri, Aug 25, 2017 at 08:34:21PM +0200, Marius Bakke wrote:
>> 'core-updates' has finished building on x86_64 on i686, and the grafting
>> failures should now be fixed.  Are we ready to merge this branch? :-)
>
> I think it's ready. There are a handful of failing packages left, but I
> assume they will be fixed by their users after the merge :)

Seconded!  I upgraded my whole profile a couple of days ago, and I’m
still alive.  :-)

I’ll look at the cross-compilation issue, but it’s too late to realize
there’s a problem in this area I’m afraid.

Ludo’.

Re: 'core-updates' status

[Leo Famulari]

On Fri, Aug 25, 2017 at 10:57:12PM +0300, Manolis Ragkousis wrote:
> On 08/25/17 21:34, Marius Bakke wrote:
> > Hello!
> > 
> > 'core-updates' has finished building on x86_64 on i686, and the grafting
> > failures should now be fixed.  Are we ready to merge this branch? :-)
> > 
> 
> Does cross-compilation work? Because I cannot cross-compile anything for
> any target I tried (i686-linux, i586-gnu).
> 
> http://paste.lisp.org/display/354303

Can you file a bug report? Do you know if this is a new regression or
has it been present for the last couple months of core-updates
development?


signature.asc
Description: PGP signature

Re: 'core-updates' status

[Manolis Ragkousis]

On 08/25/17 21:34, Marius Bakke wrote:
> Hello!
> 
> 'core-updates' has finished building on x86_64 on i686, and the grafting
> failures should now be fixed.  Are we ready to merge this branch? :-)
> 

Does cross-compilation work? Because I cannot cross-compile anything for
any target I tried (i686-linux, i586-gnu).

http://paste.lisp.org/display/354303

Manolis

Re: 'core-updates' status

[Leo Famulari]

On Fri, Aug 25, 2017 at 08:34:21PM +0200, Marius Bakke wrote:
> 'core-updates' has finished building on x86_64 on i686, and the grafting
> failures should now be fixed.  Are we ready to merge this branch? :-)

I think it's ready. There are a handful of failing packages left, but I
assume they will be fixed by their users after the merge :)


signature.asc
Description: PGP signature

Re: 'core-updates' status

[Marius Bakke]

Marius Bakke  writes:

> Marius Bakke  writes:
>
>> What do you think, are we ready to merge this branch?  There are ~2k
>> armhf packages left in the queue, but many of them have already changed
>> in 'master' and thus will need to be rebuilt anyway.
>
> I started a (hopefully final) evaluation while waiting for a consensus.
>
> Please try to `guix pull --branch=core-updates` and check that things
> are working fine on your end.  Let's merge this as soon as x86_64 and
> i686 is built out in lieu of any new regressions.

Hello!

'core-updates' has finished building on x86_64 on i686, and the grafting
failures should now be fixed.  Are we ready to merge this branch? :-)


signature.asc
Description: PGP signature

Re: 'core-updates' status

[Ludovic Courtès]

Christopher Baines  skribis:

> On Tue, 22 Aug 2017 10:34:45 +0200
> l...@gnu.org (Ludovic Courtès) wrote:
>
>> Christopher Baines  skribis:
>> 
>> > On Fri, 18 Aug 2017 23:24:29 +0200
>> > Marius Bakke  wrote:
>> >  
>> >> 'core-updates' has finished building on x86_64 and i686.  There
>> >> are a few failing packages still, but nothing too scary.  Please
>> >> try to fix some if you can!
>> >> 
>> >> https://hydra.gnu.org/eval/109753?full=1=master#tabs-now-fail  
>> >
>> > So I spotted [1] that the tailon system test is failing, but the way
>> > that it failed isn't included in the log. Is there a way to get at
>> > the log file which it generates, or, would it be good to modify the
>> > system tests to print out the log file when the test ends?  
>> 
>> We could print the log upon failure, indeed.
>> 
>> In the meantime, perhaps you can reproduce it on your machine?  :-)
>
> Unfortunately not, it passes when I run it locally. I can see the test
> it fails on, which does involve waiting for the service to start, and
> it waits for up to 50 seconds, but maybe that isn't enough, it could
> also be a different failure though.

OK.  Maybe you could try locally to:

  guix build --check ….drv --rounds=10 -K

or something.

It doesn’t sound like a blocker anyway.

Thanks,
Ludo’.

Re: 'core-updates' status

[Christopher Baines]

On Tue, 22 Aug 2017 10:34:45 +0200
l...@gnu.org (Ludovic Courtès) wrote:

> Christopher Baines  skribis:
> 
> > On Fri, 18 Aug 2017 23:24:29 +0200
> > Marius Bakke  wrote:
> >  
> >> 'core-updates' has finished building on x86_64 and i686.  There
> >> are a few failing packages still, but nothing too scary.  Please
> >> try to fix some if you can!
> >> 
> >> https://hydra.gnu.org/eval/109753?full=1=master#tabs-now-fail  
> >
> > So I spotted [1] that the tailon system test is failing, but the way
> > that it failed isn't included in the log. Is there a way to get at
> > the log file which it generates, or, would it be good to modify the
> > system tests to print out the log file when the test ends?  
> 
> We could print the log upon failure, indeed.
> 
> In the meantime, perhaps you can reproduce it on your machine?  :-)

Unfortunately not, it passes when I run it locally. I can see the test
it fails on, which does involve waiting for the service to start, and
it waits for up to 50 seconds, but maybe that isn't enough, it could
also be a different failure though.


pgp5FVEytBDsS.pgp
Description: OpenPGP digital signature

Re: 'core-updates' status

[Ludovic Courtès]

Christopher Baines  skribis:

> On Fri, 18 Aug 2017 23:24:29 +0200
> Marius Bakke  wrote:
>
>> 'core-updates' has finished building on x86_64 and i686.  There are a
>> few failing packages still, but nothing too scary.  Please try to fix
>> some if you can!
>> 
>> https://hydra.gnu.org/eval/109753?full=1=master#tabs-now-fail
>
> So I spotted [1] that the tailon system test is failing, but the way
> that it failed isn't included in the log. Is there a way to get at the
> log file which it generates, or, would it be good to modify the system
> tests to print out the log file when the test ends?

We could print the log upon failure, indeed.

In the meantime, perhaps you can reproduce it on your machine?  :-)

Thanks,
Ludo’.

Re: 'core-updates' status

[Marius Bakke]

Marius Bakke  writes:

> What do you think, are we ready to merge this branch?  There are ~2k
> armhf packages left in the queue, but many of them have already changed
> in 'master' and thus will need to be rebuilt anyway.

I started a (hopefully final) evaluation while waiting for a consensus.

Please try to `guix pull --branch=core-updates` and check that things
are working fine on your end.  Let's merge this as soon as x86_64 and
i686 is built out in lieu of any new regressions.


signature.asc
Description: PGP signature

Re: 'core-updates' status

[Efraim Flashner]

On Fri, Aug 18, 2017 at 11:24:29PM +0200, Marius Bakke wrote:
> Hello!
> 
> 'core-updates' has finished building on x86_64 and i686.  There are a
> few failing packages still, but nothing too scary.  Please try to fix
> some if you can!
> 
> https://hydra.gnu.org/eval/109753?full=1=master#tabs-now-fail
> 
> Most of the "dependency failed" armhf packages are because Hydra
> believes 'bluez' is failing, even though it has built successfully
> twice:
> 
> https://hydra.gnu.org/job/gnu/core-updates/bluez-5.45.armhf-linux
> 
> What do you think, are we ready to merge this branch?  There are ~2k
> armhf packages left in the queue, but many of them have already changed
> in 'master' and thus will need to be rebuilt anyway.

I yesterday had weechat fail to graft on core-updates:

The following derivation will be built:
   /gnu/store/9yp5dgijz8d09rm8v45lp7iwbjdmxfk5-weechat-1.9.drv
@ build-started /gnu/store/9yp5dgijz8d09rm8v45lp7iwbjdmxfk5-weechat-1.9.drv - 
aarch64-linux 
/var/log/guix/drvs/9y//p5dgijz8d09rm8v45lp7iwbjdmxfk5-weechat-1.9.drv.bz2
grafting '/gnu/store/wxaahzim320znm1pkw75616z6bzii7i2-weechat-1.9' -> 
'/gnu/store/v47ink6v2bpv3szfs2jvcz51bgciram6-weechat-1.9'...
ERROR: In procedure variable-ref: Not a variable: (#f #t #t #t #t #t #t #t #t 
#t #f #t #t #t #t #f #f #t #t #t #t #t #f #f #f #f #f #f #f #f #f #f #f #f #f 
#f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f 
#f #f #f #f #f #f #f
#f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f 
#f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f 
#f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f 
#f #f #f #f #f #f #f
#f #f)
builder for `/gnu/store/9yp5dgijz8d09rm8v45lp7iwbjdmxfk5-weechat-1.9.drv' 
failed with exit code 1


-- 
Efraim Flashner      אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: 'core-updates' status

[Christopher Baines]

On Fri, 18 Aug 2017 23:24:29 +0200
Marius Bakke  wrote:

> 'core-updates' has finished building on x86_64 and i686.  There are a
> few failing packages still, but nothing too scary.  Please try to fix
> some if you can!
> 
> https://hydra.gnu.org/eval/109753?full=1=master#tabs-now-fail

So I spotted [1] that the tailon system test is failing, but the way
that it failed isn't included in the log. Is there a way to get at the
log file which it generates, or, would it be good to modify the system
tests to print out the log file when the test ends?

1:
https://hydra.gnu.org/job/gnu/core-updates/test.tailon.x86_64-linux/all


pgpC_hbydFrTm.pgp
Description: OpenPGP digital signature

Re: 'core-updates' status

[Mark H Weaver]

Ben Woodcroft  writes:

> I went to look at blast+ on armhf, and came across this:
>
> /tmp/guix-build-blast+-2.6.0.drv-0/ncbi-blast-2.6.0+-src/c++/include/corelib/ncbistl.hpp:73:23:
> fatal error: error writing to
> /tmp/guix-build-blast+-2.6.0.drv-0/ccWOjvYh.s: No space left on device
>
> Is this symptomatic of a larger issue?

No.  One of the armhf build slaves ran out of disk space recently.  It's
fixed now, and I tried to restart most of the failed builds, but I guess
this one slipped through the cracks.  I'll restart it.

Thanks,
  Mark

Re: 'core-updates' status

[Ben Woodcroft]

Hi Marius,


On 19/08/17 07:24, Marius Bakke wrote:

Hello!

'core-updates' has finished building on x86_64 and i686.  There are a
few failing packages still, but nothing too scary.  Please try to fix
some if you can!

I went to look at blast+ on armhf, and came across this:

/tmp/guix-build-blast+-2.6.0.drv-0/ncbi-blast-2.6.0+-src/c++/include/corelib/ncbistl.hpp:73:23:
 fatal error: error writing to /tmp/guix-build-blast+-2.6.0.drv-0/ccWOjvYh.s: 
No space left on device

Is this symptomatic of a larger issue?

ben

'core-updates' status

[Marius Bakke]

Hello!

'core-updates' has finished building on x86_64 and i686.  There are a
few failing packages still, but nothing too scary.  Please try to fix
some if you can!

https://hydra.gnu.org/eval/109753?full=1=master#tabs-now-fail

Most of the "dependency failed" armhf packages are because Hydra
believes 'bluez' is failing, even though it has built successfully
twice:

https://hydra.gnu.org/job/gnu/core-updates/bluez-5.45.armhf-linux

What do you think, are we ready to merge this branch?  There are ~2k
armhf packages left in the queue, but many of them have already changed
in 'master' and thus will need to be rebuilt anyway.


signature.asc
Description: PGP signature