Haskell update (was: Release 1.2.1: Cuirass failed to build Haskell 179 packages)

[Xinglu Chen]

On Tue, Mar 23 2021, zimoun wrote:

>> Maybe its time to update to a newer Stackage LTS release? :)
>
> Well, I do not know if it possible to do so before the next release.
> IMHO, let focus on what remains for the next release and let upgrade the
> Stackage LTS after.

Any updates on this?  I managed to build GHC 8.10.5 (on x86_64), but I
had to disable some tests.  I have attached the patch if anyone wants to
take a look.  :)

From 9ecfe7cb3f94b928dbd883d4b7e7c007278c0fa8 Mon Sep 17 00:00:00 2001
Message-Id: <9ecfe7cb3f94b928dbd883d4b7e7c007278c0fa8.1623177424.git.pub...@yoctocell.xyz>
From: Xinglu Chen 
Date: Tue, 18 May 2021 23:32:38 +0200
Subject: [RFC PATCH] gnu: Add ghc-8.10.

* gnu/packages/haskell.scm (ghc-8.10): New variable.
---
 gnu/packages/haskell.scm | 66 
 1 file changed, 66 insertions(+)

diff --git a/gnu/packages/haskell.scm b/gnu/packages/haskell.scm
index 09732fc594..8de3118136 100644
--- a/gnu/packages/haskell.scm
+++ b/gnu/packages/haskell.scm
@@ -657,6 +657,72 @@ interactive environment for the functional language Haskell.")
 (file-pattern ".*\\.conf\\.d$")
 (file-type 'directory))
 
+(define-public ghc-8.10
+  (package
+(inherit ghc-8.8)
+(name "ghc")
+(version "8.10.5")
+(source
+ (origin
+   (method url-fetch)
+   (uri (string-append "https://www.haskell.org/ghc/dist/;
+   version "/ghc-" version "-src.tar.xz"))
+   (sha256
+(base32 "0vq7wch0wfvy2b5dbi308lq5225vf691n95m19c9igagdvql22gi"
+(native-inputs
+ `(("ghc-bootstrap" ,ghc-8.6)
+   ("ghc-testsuite"
+,(origin
+   (method url-fetch)
+   (uri (string-append
+ "https://www.haskell.org/ghc/dist/;
+ version "/ghc-" version "-testsuite.tar.xz"))
+   (patches (search-patches "ghc-testsuite-dlopen-pie.patch"))
+   (sha256
+(base32
+ "0vcq774rfb6q8vsnh7p5clxp2qaz8ip6d2bm2ghbq53n8jl296d6"
+   ("git" ,git-minimal) ; invoked during tests
+   ,@(filter (match-lambda
+   (("ghc-bootstrap" . _) #f)
+   (("ghc-testsuite" . _) #f)
+   (_ #t))
+ (package-native-inputs ghc-8.6
+(arguments
+ (substitute-keyword-arguments (package-arguments ghc-8.6)
+   ((#:phases phases '%standard-phases)
+`(modify-phases ,phases
+   (add-after 'unpack-testsuite 'skip-more-tests
+ (lambda _
+   ;; XXX: This test fails because our ld-wrapper script
+   ;; mangles the response file passed to the linker.
+   (substitute* "testsuite/tests/hp2ps/all.T"
+ (("^test\\('T15904'") "# guix skipped: test('T15904'"))
+   ;; The following tests fail because they are unable to
+   ;; find a C compiler.
+   (substitute* "testsuite/tests/hsc2hs/all.T"
+ (("^test\\('" all)
+  (string-append "# guix skipped: " all)))
+   (substitute* "testsuite/tests/ffi/should_run/all.T"
+ (("^test\\('Capi_Ctype_00(1|2)'" all)
+  (string-append "# guix skipped: " all)))
+   (substitute* "testsuite/tests/ffi/should_run/all.T"
+ (("makefile_test, \\['Capi_Ctype_00(1|2)'\\].*" all)
+  (string-append "# guix skipped: " all)))
+   (substitute* "libraries/base/tests/IO/T12010/test.T"
+ (("^.*" all)
+  (string-append "# guix skipped: " all)))
+   ;; No idea why these are failing
+   (substitute* '("testsuite/tests/driver/T8602/T8602.T"
+  "testsuite/tests/driver/T16521/all.T")
+ (("^.*" all)
+  (string-append "# guix skipped: " all)
+(native-search-paths (list (search-path-specification
+(variable "GHC_PACKAGE_PATH")
+(files (list
+(string-append "lib/ghc-" version)))
+(file-pattern ".*\\.conf\\.d$")
+(file-type 'directory))
+
 (define-public ghc-8 ghc-8.6)
 
 (define-public ghc ghc-8)

base-commit: 503c2039a280dd52a751a6852b4157fccd1b4195
-- 
2.32.0



signature.asc
Description: PGP signature

Re: Some more rust/cargo insights

[Leo Famulari]

On Mon, Jun 07, 2021 at 08:48:11PM +0200, Leo Prikler wrote:
> (Note, though, that a Go update would need to go through staging – not
> an ideal situation either.)

I have "cheated" regarding this in the past. Although changing the Go
package should qualify for staging, it's so cheap to build Go packages
that I have sometimes decided to "just do it" on the master branch.

Re: What is the standard path for documentation?

[Paul A. Patience]

Hi Ludovic,

On Tuesday, June 8th, 2021 at 09:31, Ludovic Courtès  wrote:

> Right, share/doc/NAME appears to be more common (perhaps we should fix
> ‘install-licence-files’).

I suppose the advantage of having the version name in the path is
allowing several versions of a library to be installed simultaneously.
Although there would be only one canonical .so file (which is a link
to whichever version is the default), the lib directory could contain
.so.VERSION files.

Of course, if those packages contain include files, the method breaks
down.
In other words, having a version suffix on the doc directory would be
beneficial only when we install libraries for linking.
But binaries in Guix get their RPATH set to the right library, so this
usecase would presumably be less common than on other distros.

All this to say, probably share/doc/NAME should be the preferred convention.

> Thus, I would not install these two files, especially since they seem to
> be redundant with Guix-supported provenance tracking.

Thanks, that's what I decided to do (i.e., delete the files).

Best regards,
Paul

Re: Some more rust/cargo insights

[Hartmut Goebel]

Am 08.06.21 um 11:15 schrieb Efraim Flashner (referring to Debian)

My understanding is that they pool together all the sources that they
have and then build all the rust packages in one go.


Maybe, I have no clue. Anyhow given my experience with sequoia, I doubt 
this will reduce build-times:


When building sequoia, several crates are build up to 4 times - even 
sequoia-openpgp itself.


--
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: RISCV porting effort

[Ludovic Courtès]

Hello!

Efraim Flashner  skribis:

> Porting Guix comes with two parts: building the bootstrap-tarballs and
> adding support for the new architecture in (gnu packages bootstrap).
> Support for riscv was added to GCC with 7.1.0 so I had to bump the
> bootstrap GCC to 7 (and adjust some inputs), and guile-2.0.14 didn't
> known the endianness of riscv64 so instead of patching it I bumped the
> bootstrap guile to 3.0. I uploaded my bootstrap-tarballs to my regular
> spot after signing them and switched over to the riscv machine.
>
> Building Guix from sources is much easier thanks to the work of Vagrant,
> I was able to apt install most of the dependencies on riscv. The
> packaged version of guile-zlib was too old in 21.04 so I built that from
> source (0.1.0 is available in Debian experimental), and I also had to
> build gnutls from source for the guile-3.0 bindings. I added the riscv
> binaries to (gnu packages bootstrap) and temporarily switched the
> raw-build function to look for 3.0 guile libraries instead of 2.0.
>
> Where I'm at right now:
> Downloading using the daemon (builtin:download) fails currently, it says
> it doesn't have permissions to open-file to write downloaded files to
> the store.
> I haven't built anything using the bootstrap binaries to see if they
> actually work yet.

Exciting news!  I saw discussions about this ‘guix perform-download’
issue on IRC.  Did you eventually find what the problem is?

If the HiFive works fine, let me remind you that we have budget to buy
hardware, so we could just as well order a couple of these right away
(maybe you’ll have Guix System running by the time we receive them ;-)).

What we need is someone to order the machine(s) and to host it
afterwards, with stable connectivity.  This must be agreed upon by the
Spending Committee (currently Tobias, Ricardo, and myself; email
guix-fina...@gnu.org).  You’ll then be reimbursed by the FSF, our
current fiscal sponsor.

Any takers?  Or should we wait until you have more experience with
yours, Efraim?

Thanks,
Ludo’.

Re: Substitutes from bordeaux.guix.gnu.org

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> I think things are pretty much ready, you can find some information at
> https://bordeaux.guix.gnu.org/ . If you're up for trying out fetching
> substitutes from bordeaux.guix.gnu.org, now is a good time to do so, and
> please report back in terms of how it's working out for you.

Neat!

> If you're interested in the infrastructure behind bordeaux.guix.gnu.org,
> these are the machines which are currently involved:
>
>  - bayfront
>- running the coordinator
>- serving the substitutes
>- building things for x86_64-linux plus i686-linux
>
>  - harbourfront
>- building things for x86_64-linux plus i686-linux
>
>  - milano-guix-1
>- building things for x86_64-linux plus i686-linux
>
>  - monokuma
>- building things for aarch64-linux
>
>  - polaris
>- sometimes building things for powerpc64le-linux

Great that all this is covered.

I would love to have a POWER9 build machine in the main/default build
farm though.  Currently there’s the OSUOSL POWER9 VM but it’s not
“plugged in” (I think because it’s a machine not “under our control”
that OSUOSL is giving us access to for a limited time, IIUC).  Maybe
more of a topic for guix-sysadmin but we’ll have to coordinate on the
use of those few scarce resources.

Thanks for the update,
Ludo’.

Re: What is the standard path for documentation?

[Ludovic Courtès]

Hi Paul,

"Paul A. Patience"  skribis:

> What is the standard place to put documentation?
> The license files are automatically installed to the
> share/doc/NAME-VERSION directory
> (see install-licence-files in gnu-build-system.scm),
> but it seems like various packages install the actual
> documentation to share/doc/NAME.

Right, share/doc/NAME appears to be more common (perhaps we should fix
‘install-licence-files’).

> Also, is it alright for files in the documentation directories
> to refer to various other directories in the store?
> Deal.II installs the attached files as documentation, but
> I thought I saw the source of a Guix package make an attempt
> to strip build directory information from one of its output
> files.

Reference to the build directories are not great but OK.

However, the two log files you mention refer to build-time tools.  As a
result, if you install those, the total size reported by ‘guix size
dealii’ will be noticeably bigger because the set of “references”
(run-time dependencies) would not include things like GCC and so on,
even though they are not actually used at run time.

Thus, I would not install these two files, especially since they seem to
be redundant with Guix-supported provenance tracking.

HTH!

Ludo’.

Re: Can we integrate application-setup with the installation script?

[Ludovic Courtès]

Hi,

Raghav Gururajan  skribis:

> Quite a few people I know, who tried guix for first time on foreign
> distro, mentioned to me that they presumed the installation script to 
> cover everything, including application-setup.
>
> Is it possible to automate application-setup through the installation
> script or is there a technical issue?

That’d be nice.  I guess you can give it a try!

As an example, commit b2683a2bed424f58722c9c17c9582e3e2e19d9a3 added
support to check for the availability of nscd.  In the discussion at
 several issues were pointed out:
(1) the script cannot always guess how to install and start nscd, and
(2) it’s considered “bad style” to install stuff behind the user’s back.
This is why the script currently merely warns about a missing nscd.

Depending on the application setup bits you’d like to automate, you may
or may not encounter these issues.

HTH!

Ludo’.

Re: Idea: a meta language for (language) build systems - npm, Racket, Rust cargo

[Ludovic Courtès]

Hi!

(I’m late to the party…)

Pjotr Prins  skribis:

> As Guix takes care of the first and the last - the issue centers
> around building. The idea is to dress down these language specific
> builders, such as cargo, so you don't have all the included
> complexity. 
>
> A sexp-pack would represent the most simple build instructions to
> build a package on its own. Now, of course the current guix-builders
> solve that too. But, what I am proposing is to split out the actual
> build step into a package definition, so as to present something
> simpler to Guix.
>
> I found a cargo -> ninja converter. It is that kind of idea. Guix
> would use ninja with rustc instead of cargo. A stripped down cargo
> could potentially work too - but cargo is a complex beast.

In a way, aren’t importers addressing this use case?

In the end, they let everyone develop a package manager/build system
that suits their needs, while still allowing us to have our cake.

WDYT?

Ludo’.

Re: Cuirass badges

[Ludovic Courtès]

Hi!

Mathieu Othacehe  skribis:

> Cuirass is now able to generate badges like any respectable CI
> system. For instance:
>
> * https://ci.guix.gnu.org/jobset/master/badge generates a badge showing
>   the percentage of successful jobs for the master specification.

Fanciness.  :-)

This is not yet deployed in the current ‘cuirass’ package, right?

Thanks,
Ludo’.

Re: [PATCH RFC 0/4] Getting rid of input labels?

[Ludovic Courtès]

Hi,

Ryan Prior  skribis:

> I think there's an opportunity to avoid the need to "fall back" to the status 
> quo, though. I picture a structure for inputs that has three cases, trivially 
> decided based on data shape:
>
> - a bare symbol, eg '(tzdata glib)
>   this is translated to `(("tzdata" ,tzdata) ("glib" ,glib))
>   works the exact same if the symbol is an =origin=, you get the name of the 
> symbol quoted followed by the value of the symbol.

A “bare symbol” cannot be translated into a reference to a variable in
the way you write.

I’m leaning towards either the new style:

  (list glib tzdata)

or the old style:

  (("tzdata" ,tzdata) ("glib" ,glib))

but not a mixture of both, which would require more code to be properly
interpreted, leading to slowdowns on the hot path and generally making
it more difficult to interpret what’s in there.

Thanks,
Ludo’.

Re: “guix gc”, auto gcroots, cluster deployments

[Ludovic Courtès]

Hi,

Ricardo Wurmus  skribis:

> Ludovic Courtès  writes:
>
>> Hi!
>>
>> Ricardo Wurmus  skribis:
>>
>>> There are two problems here:
>>>
>>> 1) I don’t think “guix gc --list-dead” (or “--list-live”, or more
>>> generally “findRoots” in nix/libstore/gc.cc) should delete
>>> anything.  It should just list and not clean up.
>>
>> Maybe ‘findRoots’ could populate the list of stale roots and it’d be
>> up
>> to the caller to decide whether to delete them or not?
>
> Yes, this would be better.  It already does this for links whose
> targets exist but cannot be read.

OK.

>>> 2) For cluster installations with remote file systems perhaps
>>> there’s
>>> something else we can do to record gcroots.  We now have this

[...]

>>> […] we would
>>> record
>>> /var/guix/profiles/per-user/me/auto/8ypp8dmwnydgbsgjcms2wyb32mng0wri
>>> pointing to 
>>> /gnu/store/ap0vrfxjdj57iqdapg8q83l4f7aylqzm-profile, and
>>> then point /home/me/projects/mrg1_chipseq/.guix-profile-1-link at
>>> that.  Yes, removing
>>> /home/me/projects/mrg1_chipseq/.guix-profile-1-link would no longer
>>> free up the profile for garbage collection, but removing
>>> $(readlink /home/me/projects/mrg1_chipseq/.guix-profile-1-link)
>>> would.
>>
>> Yes, but how would per-user/me/auto/* be cleaned up?
>
> Yeah, that’s an open question.
>
> I get the appeal of having these things be cleaned up automatically
> when the link disappears, but if we added this extra layer of
> indirection for cluster deployments this would become manual.
>
> Can we make this configurable perhaps…?  On my cluster installation
> I’d rather have a cron job to erase the stuff in per-user/me/auto/* on
> my own terms, than to have “guix gc” fail to resolve links and
> consider it all garbage.

Sure, why not.  Do you have configuration options in mind?

I have troubles wrapping my head around this problem.  Should we split
it into smaller chunks in bug-guix?

Thanks,
Ludo’.

Re: Some more rust/cargo insights

[Efraim Flashner]

On Mon, Jun 07, 2021 at 06:41:41PM +0200, Hartmut Goebel wrote:
> Am 07.06.21 um 18:26 schrieb Hartmut Goebel:
> > > Another path we should checkout is to see what Debian does. My
> > > understanding is that they figured something out.  Worth a shot, but
> > > I’d rather the problem be fixed upstream. It will just take
> > > collaboration.
> > 
> > I did not check their tollchain lately, but package-earch still does not
> > show many packages 
> > .
> > Last time I check, they basically do what guix does: compile everything
> > from source - again and again.
> > 
> Just checked: Debian has quite some source packages [1], which even list
> "binary" packages (which are development packages). Anyhow it seems as if
> these are just the source again [2]
> 
> [1] 
> 
> [2]
> 
> 

My understanding is that they pool together all the sources that they
have and then build all the rust packages in one go.

-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature