Re: Core-updates after the staging merge

[John Kehayias]

On Tue, Apr 18, 2023 at 12:55 AM, John Kehayias wrote:

> Oh, looks like no x86_64 builds currently on Cuirass...hopefully
> temporary?
>

Never mind, builds going through now! Okay, I really must get to bed!

Re: Core-updates after the staging merge

[John Kehayias]

Hi Andreas and Guixers,

Again, thanks Andreas for all your work on core-updates!

On Mon, Apr 17, 2023 at 11:03 AM, Andreas Enge wrote:

> Hello,
>
> just a quick update after a night of building on CI.
> Things look generally quite good on x86_64; some things are being rebuilt
> due to the recent wget update, and that should hopefully also sort out
> i686 rather quickly.
>

Packages I use all seem to be good too, except wine as there is a
samba failure for i686. Pending fix: 

> Am Sun, Apr 16, 2023 at 01:09:02PM +0200 schrieb Andreas Enge:
>> - python: It is mostly green, but with lots of red, including at least
>>   one package that prevents icecat from building. I would call for careful
>>   updates and bug fixes to move on; leaf packages can be updated to the
>>   latest version, but for important intermediate packages please be
>>   careful.
>
> Python probably is the place where work is needed most, the strip in the
> dashboard looks like it has measles. Fixing one package there could bring
> us a long way (or not - we are in plain hand resolution of dependency
> problems there, one update could also hide another one). Sometimes it is
> just a matter of trying an update to see whether a package and its
> dependents build, so even someone who knows nothing about Python, but
> has a web browser pointed at , can do some useful work
> (I am speaking from experience).
>

I took a stab at a few random packages and ended up finding out that
python-urllib3 needed an update due to our updated
python-cryptography. Only saw this through test failures of leaf
packages rather than itself (noted in upstream update that failures
with newer cryptography tend to point people to the wrong packages).
While urllib3 has quite a few dependents (> 500 rebuilds on update)
and this may break more than it fixes, I pushed the latest version. A
few other updates/build fixes for the packages I was trying as well.

In the chain of oauth/google packages I was looking at, I was left at
python-google-auth-1 (older version) as failing tests even adding in
the missing python-mock. At quick glance I'm guessing due to the newer
python-cryptography. I haven't had a chance to see how we want to work
around that, or if the older versions of the few dependents should be
kept. It is only a few packages anyway, so maybe bigger fish to hunt
first.

I'll check in tomorrow on how the urllib update built on Cuirass and
see if I can pick off any other easy python-* fixes.

Oh, looks like no x86_64 builds currently on Cuirass...hopefully
temporary?

John

Re: Time travel accident

[Vagrant Cascadian]

On 2023-04-17, Simon Tournier wrote:
> On lun., 17 avril 2023 at 15:32, Ludovic Courtès  wrote:
>
>> My plan is to write a service that makes it easy to offload a build to a
>> VM that runs with a different time (“in the past”) or something along
>> these lines to mitigate the problem.
>
> Do you mean “in the future” instead?  We need to know if the package
> will still build with a different time from the future, no?

I can see "in the past" being useful to handle builds with time-bombs
that already slipped through the cracks, and "in the future" to detect
these time-bombs while there is still time to fix them...


> Wording aside :-) What do Reproducible Builds for that?  Because
> time-bomb seems similar as timestamp…

At least for https://tests.reproducible-builds.org/debian on
amd64/x86_64, I think one of the builds runs approximately 1 year, 1
month and 1 day in the future (+397 days?), which pretty much maximizes
the chance of a difference in year, month or day, while sommewhat
minimizing detection of time bombs...

For detecting time-bombs, I would guess you want to test even further
into the future, maybe 5-10 years or so. Much longer, and you are
getting pretty close to 2038, which is an extra-special set of timebombs
that will need to be addressed at some point!

If guix someday has a long-term support release model, catching these
sorts of time-bombs as early as possible becomes even more important,
though with the current release model of once or twice a year or so, it
is a bit less problematic... although if it happens in something low on
the dependency graph, it of course gets more urgent!


live well,
  vagrant


signature.asc
Description: PGP signature

Re: gnu: julia: Fix test suite regression.

[Andreas Enge]

Good news! The latest julia build on i686 has succeeded:
   https://ci.guix.gnu.org/build/1032346/details

Andreas

Re: Core-updates after the staging merge

[Lars-Dominik Braun]

Hi Andreas,

> Well, I would see this as rather an action for a later feature branch.
> Except for removing 9.0 by building 9.4 with 9.2, since 9.0 does not
> build now anyway.

shouldn’t this snippet from 8.10 also work for 9.0?

   (modules '((guix build utils)))
   (snippet
;; collections.Iterable was moved to collections.abc in Python 3.10.
'(substitute* "testsuite/driver/testlib.py"
   (("collections\\.Iterable")
"collections.abc.Iterable")

Cheers,
Lars

Re: Core-updates after the staging merge

[Andreas Enge]

Am Mon, Apr 17, 2023 at 02:03:14PM -0400 schrieb Maxim Cournoyer:
> I think python is starting to look good here, after a few upgrades.
> It'll cause a rebuild of at least GTK+, so I'll push it a bit later
> today (during the European night).

Nice! I just merged master back, hopefully this is all compatible with
your changes (and hopefully I did not introduce many errors...).

Andreas

Re: Core-updates after the staging merge

[Andreas Enge]

Am Mon, Apr 17, 2023 at 07:47:16PM +0200 schrieb Simon Tournier:
> All in all, I am proposing to send a patch for the first path for this
> core-updates cycle and postpone this other path – not doable for this
> cycle; I will resume this story later.
> Andreas, core-updates is frozen but is the former shorter bootstrap
> chain acceptable?

Well, I would see this as rather an action for a later feature branch.
Except for removing 9.0 by building 9.4 with 9.2, since 9.0 does not
build now anyway.

Andreas

Re: Core-updates after the staging merge

[Maxim Cournoyer]

Hi,

Andreas Enge  writes:

> Hello,
>
> just a quick update after a night of building on CI.
> Things look generally quite good on x86_64; some things are being rebuilt
> due to the recent wget update, and that should hopefully also sort out
> i686 rather quickly.
>
> Am Sun, Apr 16, 2023 at 01:09:02PM +0200 schrieb Andreas Enge:
>> - python: It is mostly green, but with lots of red, including at least
>>   one package that prevents icecat from building. I would call for careful
>>   updates and bug fixes to move on; leaf packages can be updated to the
>>   latest version, but for important intermediate packages please be
>>   careful.
>
> Python probably is the place where work is needed most, the strip in the
> dashboard looks like it has measles. Fixing one package there could bring
> us a long way (or not - we are in plain hand resolution of dependency
> problems there, one update could also hide another one). Sometimes it is
> just a matter of trying an update to see whether a package and its
> dependents build, so even someone who knows nothing about Python, but
> has a web browser pointed at https://pypi.org/, can do some useful work
> (I am speaking from experience).

I think python is starting to look good here, after a few upgrades.
It'll cause a rebuild of at least GTK+, so I'll push it a bit later
today (during the European night).

-- 
Thanks,
Maxim

Re: OCaml4.07 (was Re: Graphical login broken on core-updates)

[Maxim Cournoyer]

Hi,

Simon Tournier  writes:

> Hi,
>
> Well, ocaml-4.07.1 is broken [1] on core-updates, mainly because [2]:
>
> building of 
> `/gnu/store/mc6cbczwjlsnc3hlqj3kqlyhgfc8yk0c-camlboot-0.0.0-1.45045d0.drv' 
> timed out after 3600 seconds of silence
> @ build-failed 
> /gnu/store/mc6cbczwjlsnc3hlqj3kqlyhgfc8yk0c-camlboot-0.0.0-1.45045d0.drv - 
> timeout
> cannot build derivation 
> `/gnu/store/vyhk28gzjng5va9n3k22g1qz3ikk36pl-ocaml-boot-4.07.1.drv': 1 
> dependencies couldn't be built
> cannot build derivation 
> `/gnu/store/0vwlmbqc6nq64bmwi9qnc0fvldwkflcp-ocaml-4.07.1.drv': 1 
> dependencies couldn't be built
>
>
> 1: https://ci.guix.gnu.org/build/752949/details
> 2: https://ci.guix.gnu.org/build/752949/log/raw
>
>
> Well, the package ’camlboot’ already have:
>
>   (properties
>;; 10 hours, mostly for arm, more than 1 expected even on x86_64
>`((max-silent-time . 36000)))
>
> and Lars pointed [3] that Cuirass does not honor this property.

I think it does, nowadays.  We'd have to cross-check in Cuirass source
to confirm.

-- 
Thanks,
Maxim

Re: Time travel accident

[Simon Tournier]

Hi,

On lun., 17 avril 2023 at 15:32, Ludovic Courtès  wrote:

> My plan is to write a service that makes it easy to offload a build to a
> VM that runs with a different time (“in the past”) or something along
> these lines to mitigate the problem.

Do you mean “in the future” instead?  We need to know if the package
will still build with a different time from the future, no?

Wording aside :-) What do Reproducible Builds for that?  Because
time-bomb seems similar as timestamp…


Cheers,
simon

Re: Core-updates after the staging merge

[Simon Tournier]

Hi,

On lun., 17 avril 2023 at 16:12, Lars-Dominik Braun  wrote:

> note that the information on haskell.org is not always accurate and thus
> this shorter chain may not actually work. Please give it a try and send
> a patch.

If I read correctly, the current chain is:

   7.8.4
-> 7.10.3
-> 8.0.2
-> 8.4.4
-> 8.6.5
-> 8.8.4
-> 8.10.7
-> 9.0.2; 9.2.5
-> 9.4.4 (next)

where 9.2.5 is the one used by haskell-build-system.  Instead, this
chain,

   7.8.4  (needs 7.4 at least)
-> 8.0.1  (needs 7.8 at least)
-> 8.4.4  (needs 8.0 at least)
-> 8.6.5  (needs 8.2 at least)
-> 8.10.7  (needs 8.6 at least)
-> 9.2.5  (needs 8.10 at least)

builds for me.  So It removes 7.10.3 and 8.8.4.


That said, note that the current binary root is 7.8.4 with the hope to
join with the current,

   4.08.2 (needs GCC and outputs of previous GHC)
-> 6.0(needs 4.08 at least)
-> 6.6(needs 5.04 at least)
-> 6.10.4 (needs 6.6  at least)

Well, joining 9.2.5 to 4.08.2 could be done using intermediary versions:

-> 6.10.4 (needs 6.6  at least)  | -> 6.10.4 (needs 6.6  at least)
   6.12.3 (needs 6.8  at least)  |7.2.2  (needs 6.10 at least)
   7.4.2  (needs 6.12 at least)  |7.6.3  (needs 7.0  at least)
   7.8.4  (needs 7.4 at least)   | -> 7.10.3 (needs 7.6  at least)
-> 8.0.1  (needs 7.8 at least)   |8.2.2  (needs 7.10 at least)
-> 8.4.4  (needs 8.0 at least)   | -> 8.6.5  (needs 8.2  at least)
-> 8.6.5  (needs 8.2 at least)   | -> 8.10.7 (needs 8.6 at least)
-> 8.10.7 (needs 8.6 at least)   | -> 9.2.5  (needs 8.10 at least)
-> 9.2.5  (needs 8.10 at least)

Well, one version would be win when using 7.10.3 (modulo inaccurate
information on Haskell website :-)).


All in all, I am proposing to send a patch for the first path for this
core-updates cycle and postpone this other path – not doable for this
cycle; I will resume this story later.


Andreas, core-updates is frozen but is the former shorter bootstrap
chain acceptable?


>> I mean I propose to have both: ghc-x.y (with tests) and
>> ghc.x.y/bootstrap (without tests), it would ease the maintenance of the
>> Haskell ecosystem on several architectures.  WDYT?
>
> I have a bad feeling about turning the testsuites of intermediate versions
> off. Yes, they take a long time, but then they also ensure the resulting
> compiler actually works (with high confidence) and we don’t silently
> propagate issues into the next one.

Well, I am convinced that we are doing the optimal way.  For instance,
the run is sequencial,

--8<---cut here---start->8---
   #:test-target "test"
   ;; We get a smaller number of test failures by disabling parallel test
   ;; execution.
   #:parallel-tests? #f
--8<---cut here---end--->8---

Then, for another example, the GHC testsuite is distributed separately,

https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-src.tar.xz
https://downloads.haskell.org/~ghc/8.10.7/ghc-8.10.7-testsuite.tar.xz

And Debian seems considering the GHC testsuite as a package,

https://tracker.debian.org/pkg/ghc-testsuite

Maybe we could do that.  It would allow to catch errors and not wait
ages after each GHC bootstrap toolchain completes all its testsuite.

WDYT?


Cheers,
simon

Re: guix system container and friends fail to report intermediate errors

[Ludovic Courtès]

Hi!

Vladilen Kozin  skribis:

> Hello. Is it possible to get better error reporting in the following
> example?
>
> $ sudo guix system -K -L /home/vlad/Code/fullmeta-guix/channel container
> os.scm
>
> And our os.scm imports some services defined in our channel above. Should
> there be an error in one of them, however, the stack trace only talks about
> inability to find that service symbol but otherwise fails to report any
> errors that may've happened when loading modules from our -L location above.
>
> Case in point. In one of the modules, I habitually used Clojury syntax and
> wrote:
>
> (define foo [1 2])
>
> (define foo-service
>   (list
>  (shepherd-service ... #$foo ...)))
>
> (define foo-service-type ...)
>
> Running the above OS derivation reports:
> ```
> Backtrace:
> In guix/store.scm:
> ... bt here ...
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> error: foo-service-type: unbound variable
> ```

If I understand correctly what’s happening, the module file that defines
‘foo’ and ‘foo-service’ fails to load with a “Wrong type to apply”
error.  Consequently, you end up with that “unbound variable” error in
your OS file.  Do I get it right?

Thanks,
Ludo’.

Re: gnu: julia: Fix test suite regression.

[Andreas Enge]

Am Mon, Apr 17, 2023 at 03:32:45PM +0200 schrieb Simon Tournier:
> Well, Julia builds fine for me, both on x86_64 and i686 (using the
> option -s i686-linux).  But indeed, Julia requires some memory to pass
> the test suite – more than 16GB.

Okay, then maybe we could just restart the builds. In any case, they did not
pass on berlin.

> > Should we remove those from the supported systems?
> Well, I am confused because Julia be cross-built and so i686 is part of
> the supported systems.

Not exactly cross-built, but I see what you mean. So we have machines
that build for i686, say, while apparently addressing more than 4GB of RAM.

Andreas

core-updates failures with ‘pre-inst-env’

[Development of GNU Guix and the GNU System distribution.]

Yesterday, I managed to complete a system reconfigure and 
rebooted. Most things seem to be working fine; there's an issue 
with ‘alacritty’ presumably due to a change in wlr, but I have a 
work-around for that (the workaround is Emacs). I'm also getting 
warnings about locales from Guile:


--8<---cut here---start->8---
~/src/guix-core-updates $ herd status
guile: warning: failed to install locale
--8<---cut here---end--->8---

Seems harmless so far, but I am American, so the defaults always 
work for me anyway.


More troublesome is that, while I can use the system Guix just 
fine, I can no longer use a dev Guix with ‘pre-inst-env’:


--8<---cut here---start->8---
~/src/guix-core-updates $ guix build hello
substitute: updating substitutes from 
'https://ci.guix.gnu.org'... 100.0%

0.1 MB will be downloaded:
 /gnu/store/s5pd3rnzymliafb4la5sca63j86xs0y0-hello-2.12.1
substituting 
/gnu/store/s5pd3rnzymliafb4la5sca63j86xs0y0-hello-2.12.1...
downloading from 
https://ci.guix.gnu.org/nar/lzip/s5pd3rnzymliafb4la5sca63j86xs0y0-hello-2.12.1 
...
hello-2.12.1  59KiB 
439KiB/s 00:00 ▕██▏ 100.0%


/gnu/store/s5pd3rnzymliafb4la5sca63j86xs0y0-hello-2.12.1
~/src/guix-core-updates $ ./pre-inst-env -- guix build hello
hint: Consider installing the `glibc-locales' package and defining 
`GUIX_LOCPATH', along these lines:


guix install glibc-locales
export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"

See the "Application Setup" section in the manual, for more info.

guix build: error: gcry_md_hash_buffer: Function not implemented
~/src/guix-core-updates $ herd status
--8<---cut here---end--->8---

Again there's an issue with locales for some reason, but the 
bigger problem is the “Function not implemented”, which breaks 
everything. It's not always ‘gcry_md_hash_buffer’, if I try to use 
my dev Guix with ‘home build’, this happens instead:


--8<---cut here---start->8---
~/src/guix-core-updates $ ./pre-inst-env -- guix home build 
~/guix-home/config.scm 
hint: Consider installing the `glibc-locales' package and defining 
`GUIX_LOCPATH', along these lines:


guix install glibc-locales
export GUIX_LOCPATH="$HOME/.guix-profile/lib/locale"

See the "Application Setup" section in the manual, for more info.

Backtrace:
In ice-9/threads.scm:
   390:8 19 (_ _)
In ice-9/boot-9.scm:
 3253:13 18 (_)
In ice-9/threads.scm:
   390:8 17 (_ _)
In ice-9/boot-9.scm:
 3544:20 16 (_)
  2836:4 15 (save-module-excursion #  ice-9/boot-9.scm:3545:21 ()>)

 3564:26 14 (_)
In unknown file:
 13 (primitive-load-path "guix/channels" # 7fa0b4f47380 at ice-9/boot-9.scm:3551:37 ()>)

In ice-9/boot-9.scm:
 3923:23 12 (_)
  3411:4 11 (define-module* _ #:filename _ #:pure _ #:version _ 
  #:imports _ #:exports _ #:replacements _ #:re-exports _ # _ # _ 
  # _ ?)

 3424:24 10 (_)
  222:17  9 (map1 (((git)) ((guix git)) ((guix git-authenticate)) 
  ((guix openpgp) #:select (openpgp-public-key-fingerprint #)) # 
  ?))
 3327:17  8 (resolve-interface (git) #:select _ #:hide _ #:prefix 
 _ #:renamer _ #:version _)

In ice-9/threads.scm:
   390:8  7 (_ _)
In ice-9/boot-9.scm:
 3253:13  6 (_)
In ice-9/threads.scm:
   390:8  5 (_ _)
In ice-9/boot-9.scm:
 3544:20  4 (_)
  2836:4  3 (save-module-excursion #  ice-9/boot-9.scm:3545:21 ()>)

 3564:26  2 (_)
In unknown file:
  1 (primitive-load-path "git" #  at ice-9/boot-9.scm:3551:37 ()>)

In git/bindings.scm:
66:8  0 (_ . _)

git/bindings.scm:66:8: In procedure git_libgit2_init: Function not 
implemented

--8<---cut here---end--->8---

I assume the difference in which function is found to not be 
implemented comes from whether or not I've already downloaded a 
substitute or whether I have to build it from fresh, but the 
actual problem is somewhere in the C bindings.


But why does this only happen when I try to use ‘pre-inst-env’? I 
have already done a complete bootstrap -> configure -> make clean 
-> make cycle, and the errors persist.


Any ideas?

-bjc

Re: Core-updates after the staging merge

[Lars-Dominik Braun]

Hi Simon,

> and instead we could try this shorter one:
> 
>7.8.4
> -> 8.0.2 (needs >= 7.10)
> -> 8.4.4 (needs >= 8.0)
> -> 8.8.4 (needs >= 8.4)
> -> 9.2.5 (needs >= 8.6)
> 
> WDYT?  If no objection, I will give a try.

note that the information on haskell.org is not always accurate and thus
this shorter chain may not actually work. Please give it a try and send
a patch.

> I mean I propose to have both: ghc-x.y (with tests) and
> ghc.x.y/bootstrap (without tests), it would ease the maintenance of the
> Haskell ecosystem on several architectures.  WDYT?

I have a bad feeling about turning the testsuites of intermediate versions
off. Yes, they take a long time, but then they also ensure the resulting
compiler actually works (with high confidence) and we don’t silently
propagate issues into the next one.

Cheers,
Lars

Re: Powerpc in the build farm

[Ludovic Courtès]

Hi,

Andreas Enge  skribis:

> when looking at the powerpc machines at
>https://ci.guix.gnu.org/workers
> one sees that only one slot out of two is used on guixp9, and nothing
> at all on sjd-p9.

Both are running:

  • on guixp9, with ‘--workers=2’ (the machine has 8 cores);

  • on sjd-p9, with ‘--workers=4’ (the machine has 32 cores).

This is consistent with what I see right now at
.

(Now those POWER9 machines are so fast that they can beat x86_64 even
with so few CPUs. :-))

Ludo’.

Re: herd stop hangs

[Ludovic Courtès]

Hi,

Vlad Kozin  skribis:

>> (I guess the second command is ‘sudo herd stop my-service’.)  If you
>> interrupt the ‘herd’ command, does ‘sudo herd status’ and similar
>> commands still respond?
>
> ‘sudo herd stop myservice’ exactly. Sorry about that.
> Herd continues to function. Appears that somehow SIGTERM has no effect
> or it never gets sent or it is in fact delivered and acted upon but
> shepherd fails to notice, so /var/log/messages shows subsequent
> SIGKILL. Whichever it is, herd status will show the service has been
> stopped. That’s one issue.

Could you send the relevant /var/log/messages excerpt?

How frequent is it?  If you have a recipe to reproduce the issue, I’m
most interested!

> Another is that the console where ‘sudo herd stop’ was typed required
> a CTRL C to get the prompt.

OK.

> Could it be some weird sequence of what appears in the :running slot or smth? 
> I’m using standard process constructors destructors mimicking other services

If this is a service that’s not in Guix proper, could you share the
code?  That way we’ll have everything we need to investigate.

Thanks,
Ludo’.

Re: Contributing Guix Home services

[Ludovic Courtès]

Hi!

Tanguy LE CARROUR  skribis:

> It's been quite a journey on my side! Ups. Downs. Mostly downs, though! 😅
> Thanks to Simon's unconditional technical and moral support, a **LOT**
> has changed since I sent this message. Hopefully for the better! 🤞

Heh.  :-)  While it’s fresh on your mind, it would be nice to list the
problems you ran into on your journey and see what we can do about it.

> At least now one of them [1] looks like a decent home service. Except
> for the problem with `(every khal-calendar? lst)` that I haven't figure
> out yet.
>
> [1]: 
> https://git.easter-eggs.org/bioneland/guix/-/blob/main/bioneland/home/services/khal.scm
>
>
> The one for MSMTP [2] does not contain all the available options, but all
> the configurations and serializers are there.
>
> [2]: 
> https://git.easter-eggs.org/bioneland/guix/-/blob/main/bioneland/home/services/msmtp.scm

I don’t use these two programs, but the services look nice!  The extra
bit of work you’d have to do is documentation, similar to what is done
for the other Home services.  Maybe start with msmtp to get a feel of
what that entails?

>> There’s no formal rule, but I think that what we’ve been doing so far is to 
>> ensure basic functionality of
>> the service is covered, and to provide an “escape hatch” for bits of the
>> configuration that are not covered.
>
> If by "escape hatch" you mean `extra-config`, you're right!

Yup!

> I'll try to submit a patch for one of the two mentioned above soon…ish!

Excellent, thanks!

Ludo’.

Re: gnu: julia: Fix test suite regression.

[Simon Tournier]

Hi,

On lun., 17 avril 2023 at 11:45, Andreas Enge  wrote:

> you wrote recently that Julia requires more than 4GB to build, and the 16GB
> of my laptop were indeed not enough. Now I see this on CI for i686:
> Error in testset core:
> Error During Test at 
> /tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/test/testdefs.jl:21
>   Got exception outside of a @test
>   LoadError: OutOfMemoryError()

Well, Julia builds fine for me, both on x86_64 and i686 (using the
option -s i686-linux).  But indeed, Julia requires some memory to pass
the test suite – more than 16GB.


> Can we deduce that Julia is no more (?) able to be built on 32 bit systems?

Do you mean able to be natively built on 32 bit systems?


> Should we remove those from the supported systems?

Well, I am confused because Julia be cross-built and so i686 is part of
the supported systems.

Cheers,
simon

Re: Time travel accident

[Ludovic Courtès]

Hi Konrad,

Konrad Hinsen  skribis:

> That problem has been reported elsewhere and identified as caused by an
> expired certificate:
>
>https://github.com/openssl/openssl/issues/18441
>
>
> I guess there is nothing we can do retroactively to fix this, but can we
> do something to prevent such issues in the future?

This has been discussed on several occasions, such as:

  https://issues.guix.gnu.org/58650

Unfortunately there’s no bullet-proof way to avoid these time bombs.

My plan is to write a service that makes it easy to offload a build to a
VM that runs with a different time (“in the past”) or something along
these lines to mitigate the problem.

Ludo’.

[PATCH 0/8] Remove mysql

[Efraim Flashner]

I saw that a fix was applied to mysql on core-updates to have it build
with openssl-1.1, and I figured that rather than have a version that was
several years out of date (and surely had many many security
vulnerabilities) it would be better to try to just remove it.
`mariadb:dev` provides the `mysql_config` binary that some packages
search for so I used it as a drop-in replacement. Nothing seemed broken
to me after building some packages, so to the best of my knowledge of
these disparate packages and languages all is fine.

Efraim Flashner (8):
  gnu: sysbench: Build with mariadb.
  gnu: ghc-mysql: Build with mariadb.
  gnu: neko: Build with mariadb.
  gnu: inspircd: Build with mariadb.
  gnu: ulogd: Build with mariadb.
  gnu: opensmtpd-extras: Build with mariadb.
  gnu: sbcl-clsql: Build with mariadb.
  gnu: cl-clsql: Build with mariadb.

 gnu/packages/benchmark.scm   |  9 +
 gnu/packages/haskell-xyz.scm |  4 ++--
 gnu/packages/haxe.scm|  3 ++-
 gnu/packages/irc.scm |  4 ++--
 gnu/packages/linux.scm   |  6 +++---
 gnu/packages/lisp-xyz.scm| 18 ++
 gnu/packages/mail.scm|  4 ++--
 7 files changed, 22 insertions(+), 26 deletions(-)


base-commit: 2d06dfc050114dba44e791d8decc8eaa705fee01
-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Re: Zig on core-updates

[Efraim Flashner]

On Tue, Apr 11, 2023 at 07:11:47AM +0200, Pjotr Prins wrote:
> 
> On Mon, Apr 10, 2023 at 05:56:52PM +0200, Andreas Enge wrote:
> > Am Fri, Mar 17, 2023 at 09:33:31AM + schrieb pukkamustard:
> > > A hunch: This might have something to do with Zig not properly
> > > supporting glibc 2.35: https://github.com/ziglang/zig/issues/12808
> > 
> > Thanks, this looks like a good explanation of the problem, but I do not
> > quite see what could be a solution... I see a string of issues of the form
> > "add support for targeting glibc ..."
> >https://github.com/ziglang/zig/issues/12808
> >https://github.com/ziglang/zig/issues/12809
> >https://github.com/ziglang/zig/issues/14798
> > but all of them are open.
> > 
> > We may have to do without zig in the foreseeable future...
> > 
> 
> They just added a tag for the next release. We can wait.

We may have to backport whatever fix they do, it looks like 0.11 is
going to use their new bootstrap chain, with a pre-compiled zig wasm
binary.

-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: Core-updates after the staging merge

[Simon Tournier]

On Mon, 17 Apr 2023 at 14:39, Andreas Enge  wrote:
>
> Am Mon, Apr 17, 2023 at 02:19:43PM +0200 schrieb Simon Tournier:
> > and instead we could try this shorter one:
> >7.8.4
> > -> 8.0.2 (needs >= 7.10)
>
> Here it looks like we still need 7.10.

Sorry for the typo:

-> 8.0.1 (needs >= 7.8)

Bottom of : "The
source distribution needs an installed GHC (version 7.8 at least)."

Contrary to 8.0.2;
: "The source
distribution needs an installed GHC (version 7.10 at least)."

Cheers,
simon

Re: Core-updates after the staging merge

[Andreas Enge]

Am Mon, Apr 17, 2023 at 11:03:25AM +0200 schrieb Andreas Enge:
> On aarch64 and powerpc, we are still stuck by CI problems.

Things improve! I could reenable sjd-p9 by a little "guix gc". Thanks to
Ricardo for walking me through a few cuirass steps! So we have four more
build slots on powerpc, five times what we had before. I still do not see
why only one of the two slots on guixp9 is building.

And a second aarch64 machine is online, with people working on others.

Andreas

Re: Core-updates after the staging merge

[Andreas Enge]

Am Mon, Apr 17, 2023 at 02:19:43PM +0200 schrieb Simon Tournier:
> and instead we could try this shorter one:
>7.8.4
> -> 8.0.2 (needs >= 7.10)

Here it looks like we still need 7.10.

> where ghc-x.y is a full GHC containing the complete testsuite.  I
> propose here to replace by ’ghc-x.y/bootstrap’ where the tests are
> turned off for this ghc-x.y/bootstrap.  We would not have to wait hours
> and hours… Hum, I do not know if it is a good idea. :-)

Well, it would mean that bootstrapping would somehow happen in parallel
to building the full ghc versions with tests. But is there any use for
keeping these old versions around except for bootstrapping? If not, it
might be enough to enable tests only for the last, user facing version.

Andreas

Re: Core-updates after the staging merge

[Simon Tournier]

Hi,

On lun., 17 avril 2023 at 11:56, Andreas Enge  wrote:
> Am Mon, Apr 17, 2023 at 11:03:25AM +0200 schrieb Andreas Enge:
>> - ghc is taking a long time...
>
> Actually ghc@9.0 fails its tests, which are written in Python (it looks
> like ".abc" needs to be added to collections); but since it is not needed
> for bootstrapping any more, maybe we could drop it?

Well, the current Haskell bootstrap chain is:

   7.8.4
-> 7.10.3
-> 8.0.2
-> 8.4.4
-> 8.6.5
-> 8.8.4
-> 8.10.7
-> 9.0.2; 9.2.5
-> 9.4.4 (next)

and instead we could try this shorter one:

   7.8.4
-> 8.0.2 (needs >= 7.10)
-> 8.4.4 (needs >= 8.0)
-> 8.8.4 (needs >= 8.4)
-> 9.2.5 (needs >= 8.6)

WDYT?  If no objection, I will give a try.



Moreover, the recipe for GHC looks like:

--8<---cut here---start->8---
(define-public ghc-9.0
  (package
(inherit ghc-8.10)
(name "ghc")
(version "9.0.2")
(source (origin
  (method url-fetch)
  (uri (string-append "https://www.haskell.org/ghc/dist/"; version
  "/ghc-" version "-src.tar.xz"))
  (sha256
   (base32
"15wii8can2r3dcl6jjmd50h2jvn7rlmn05zb74d2scj6cfwl43hl"
(native-inputs
 `(;; GHC 9.0.2 must be built with GHC >= 8.8
   ("ghc-bootstrap" ,ghc-8.10)
   ("ghc-testsuite"
,(origin
   (method url-fetch)
   (uri (string-append
  "https://www.haskell.org/ghc/dist/";
  version "/ghc-" version "-testsuite.tar.xz"))
   (sha256
(base32
 "1m5fzhr4gjn9ni8gxx7ag3fkbw1rspjzgv39mnfb0nkm5mw70v3s"
   ,@(filter (match-lambda
   (("ghc-bootstrap" . _) #f)
   (("ghc-testsuite" . _) #f)
   (_ #t))
 (package-native-inputs ghc-8.10
(native-search-paths
 (list (search-path-specification
(variable "GHC_PACKAGE_PATH")
(files (list (string-append "lib/ghc-" version)))
(file-pattern ".*\\.conf\\.d$")
(file-type 'directory))
--8<---cut here---end--->8---

and it is similar for ’ghc-8.10’ and other, well something like:

(native-inputs
 `(
   ("ghc-bootstrap" ,ghc-x.y)
   ("ghc-testsuite"
,(origin
   (method url-fetch)
   (uri (string-append
  "https://www.haskell.org/ghc/dist/";
  version "/ghc-" version "-testsuite.tar.xz"))

where ghc-x.y is a full GHC containing the complete testsuite.  I
propose here to replace by ’ghc-x.y/bootstrap’ where the tests are
turned off for this ghc-x.y/bootstrap.  We would not have to wait hours
and hours… Hum, I do not know if it is a good idea. :-)

I mean I propose to have both: ghc-x.y (with tests) and
ghc.x.y/bootstrap (without tests), it would ease the maintenance of the
Haskell ecosystem on several architectures.  WDYT?

Cheers,
simon

OCaml4.07 (was Re: Graphical login broken on core-updates)

[Simon Tournier]

Hi,

Well, ocaml-4.07.1 is broken [1] on core-updates, mainly because [2]:

--8<---cut here---start->8---
building of 
`/gnu/store/mc6cbczwjlsnc3hlqj3kqlyhgfc8yk0c-camlboot-0.0.0-1.45045d0.drv' 
timed out after 3600 seconds of silence
@ build-failed 
/gnu/store/mc6cbczwjlsnc3hlqj3kqlyhgfc8yk0c-camlboot-0.0.0-1.45045d0.drv - 
timeout
cannot build derivation 
`/gnu/store/vyhk28gzjng5va9n3k22g1qz3ikk36pl-ocaml-boot-4.07.1.drv': 1 
dependencies couldn't be built
cannot build derivation 
`/gnu/store/0vwlmbqc6nq64bmwi9qnc0fvldwkflcp-ocaml-4.07.1.drv': 1 dependencies 
couldn't be built
--8<---cut here---end--->8---

1: https://ci.guix.gnu.org/build/752949/details
2: https://ci.guix.gnu.org/build/752949/log/raw


Well, the package ’camlboot’ already have:

--8<---cut here---start->8---
  (properties
   ;; 10 hours, mostly for arm, more than 1 expected even on x86_64
   `((max-silent-time . 36000)))
--8<---cut here---end--->8---

and Lars pointed [3] that Cuirass does not honor this property.

WDYT?


Cheers,
simon


3: https://yhetil.org/guix/zdkvrxc77+loq...@noor.fritz.box

Re: Core-updates after the staging merge

[Andreas Enge]

Am Mon, Apr 17, 2023 at 11:03:25AM +0200 schrieb Andreas Enge:
> - ghc is taking a long time...

Actually ghc@9.0 fails its tests, which are written in Python (it looks
like ".abc" needs to be added to collections); but since it is not needed
for bootstrapping any more, maybe we could drop it?

Well, there is the dependent package ghc-next@9.4.4 that would need to be
built from 8.10 also, or maybe 9.2. Some work for the haskell team ;-)

Andreas

Re: gnu: julia: Fix test suite regression.

[Andreas Enge]

Hello Simon,

you wrote recently that Julia requires more than 4GB to build, and the 16GB
of my laptop were indeed not enough. Now I see this on CI for i686:
Error in testset core:
Error During Test at 
/tmp/guix-build-julia-1.8.3.drv-0/julia-1.8.3/test/testdefs.jl:21
  Got exception outside of a @test
  LoadError: OutOfMemoryError()

Can we deduce that Julia is no more (?) able to be built on 32 bit systems?
Should we remove those from the supported systems?

Andreas

Re: Core-updates after the staging merge

[Andreas Enge]

Hello,

just a quick update after a night of building on CI.
Things look generally quite good on x86_64; some things are being rebuilt
due to the recent wget update, and that should hopefully also sort out
i686 rather quickly.

Am Sun, Apr 16, 2023 at 01:09:02PM +0200 schrieb Andreas Enge:
> - python: It is mostly green, but with lots of red, including at least
>   one package that prevents icecat from building. I would call for careful
>   updates and bug fixes to move on; leaf packages can be updated to the
>   latest version, but for important intermediate packages please be
>   careful.

Python probably is the place where work is needed most, the strip in the
dashboard looks like it has measles. Fixing one package there could bring
us a long way (or not - we are in plain hand resolution of dependency
problems there, one update could also hide another one). Sometimes it is
just a matter of trying an update to see whether a package and its
dependents build, so even someone who knows nothing about Python, but
has a web browser pointed at https://pypi.org/, can do some useful work
(I am speaking from experience).

> - ghc is taking a long time...

This is a real nightmare. ghc@9.2 on x86_64 has been building for 6 hours
in a row:
   https://ci.guix.gnu.org/build/984270/details
and we need to build many versions one after the other during bootstrapping.
While not blocking the core-updates merge, maybe the ghc team can try and
see whether we can cut corners. Or maybe drop tests on intermediate versions
that are only required for bootstrapping?

On aarch64 and powerpc, we are still stuck by CI problems.

Andreas

Re: Core-updates after the staging merge

[Andreas Enge]

Am Mon, Apr 17, 2023 at 08:18:00AM + schrieb Guillaume Le Vaillant:
> I tried to build the 1611 dependents of sbcl on x86-64, and most of them
> build fine. I get only 6 failures, some of them because some
> dependencies like mysql or supercollider are failing to build.
> So overall, Common Lisp packages on core-updates are in a pretty good
> shape.

Great, thanks for testing! Indeed it looks as if the buildfarm just
needed to catch up.

Andreas

Re: PyQt in core-updates

[Andreas Enge]

Am Tue, Apr 04, 2023 at 09:55:37AM +0200 schrieb Lars-Dominik Braun:
> > I have just fixed calibre. It failed to build because .sip fails are
> > now in a subdirectory /lib/python3.10/site-packages/PyQt5/bindings
> > instead of /share/sip (or maybe before, they were in both directories).
> no, it was definitely in /share/sip before, but the pyproject-based
> build system does not expose any option to move it there :( If anyone
> can figure out how to move it back *and* successfully compile pyqt,
> please, change it back to /share/sip.

Well, if we want to have files somewhere else, we could always add a phase
and move them from one place to the other. I wonder how other distributions
do it.

Andreas

Re: Core-updates after the staging merge

[Guillaume Le Vaillant]

Andreas Enge  skribis:

> Hello all,
>
> the merge of staging to master, and the subsequent merge of master to
> core-updates did break a few things; but on the positive side, we are
> halfway there with getting rid of the staging and core-updates branches ;-)
> CI has almost caught up on x86_64; looking at the dashboard at
>https://ci.guix.gnu.org/eval/402403/dashboard
> shows much more red than on master, so we will need to do some more work
> before the merge. Since it is ordered alphabetically and our package names
> often start by the language, red streaks often indicate problems with
> a given programming language. There are things to work on for most of the
> teams!
>
> [...]
> - Everything common lisp related is red (cl-*, sbcl-*), but maybe there
>   is a similar thing going; or maybe not, since clisp is built.
> [...]

Hi,
I tried to build the 1611 dependents of sbcl on x86-64, and most of them
build fine. I get only 6 failures, some of them because some
dependencies like mysql or supercollider are failing to build.
So overall, Common Lisp packages on core-updates are in a pretty good
shape.


signature.asc
Description: PGP signature

Re: wget on i686 in core-updates

[Andreas Enge]

Am Sun, Apr 16, 2023 at 08:57:48PM +0200 schrieb Andreas Enge:
> As a stop-gap measure, I suggest to use a self-hosted tarball as in the
> attached commit. I have good hope that we will soon see a 1.23.4 release,
> see https://lists.gnu.org/archive/html/bug-wget/2023-04/msg2.html .

I just pushed the commit. There should be a new wget release this week:
https://lists.gnu.org/archive/html/bug-wget/2023-04/msg5.html
and then we can push the update. Having wget will enable us to see where
more work is needed on i686.

Andreas