Re: [PATCH RFC 0/4] Getting rid of input labels?
Hello!
Ludovic Courtès skribis:
> Here’s a proposal for a soft revolution: getting rid of input labels
> in package definitions. Instead of writing:
>
> (native-inputs
> `(("autoconf" ,autoconf)
>("automake" ,automake)
>("pkg-config" ,pkg-config)
>("guile" ,guile-3.0)))
>
> one can write:
>
> (native-inputs (list autoconf automake pkg-config guile-3.0))
I’m still looking into the feasibility of this change. For it to work,
I think it’s better if the transition to the new style is as fast as
possible. The attached script helps with that: it automatically
converts the source of packages where the conversion does not even
change the package derivation. It generates diffs like this:
@@ -280,10 +280,9 @@ during long operations.")
(base32 "106akalywfmnypzkdrhgz4n4740a8xayspybsw59kq06vz8i2qrc"
(build-system python-build-system)
(native-inputs
- `(("python-mock" ,python-mock)
- ("python-pytest" ,python-pytest)))
+ (list python-mock python-pytest))
(propagated-inputs
- `(("python-nltk" ,python-nltk-3.4)))
+ (list python-nltk-3.4))
(home-page
"https://github.com/yeraydiazdiaz/lunr.py;)
(synopsis "Full-text search library")
@@ -314,13 +313,13 @@ that best match text queries.")
(substitute* "setup.py"
(("==") ">=")))
(propagated-inputs
- `(("python-click" ,python-click)
- ("python-jinja2" ,python-jinja2)
- ("python-livereload" ,python-livereload)
- ("python-lunr" ,python-lunr)
- ("python-markdown" ,python-markdown)
- ("python-pyyaml" ,python-pyyaml)
- ("python-tornado" ,python-tornado)))
+ (list python-click
+ python-jinja2
+ python-livereload
+ python-lunr
+ python-markdown
+ python-pyyaml
+ python-tornado))
(home-page "https://www.mkdocs.org;)
(synopsis "Project documentation with Markdown")
(description "MkDocs is a static site generator geared towards building
You can run the script to modify, say, all the ‘python*’ packages:
./pre-inst-env guile simplify-package-inputs.scm \
$(./pre-inst-env guix package -A '^python' | cut -f1,2 |tr '\t' '@')
It runs in a minute and the resulting diff looks like this:
100 files changed, 4423 insertions(+), 5180 deletions(-)
The script warns when it fails to convert a package or when a comment
could not be preserved during conversion (it tries to preserve margin
comments but it’s a bit of a hack since neither ‘read’ nor
‘pretty-print’ help with that):
--8<---cut here---start->8---
gnu/packages/wxwidgets.scm:318:5: warning: python2-wxpython: input label
'wxwidgets' does not match package name, bailing out
gnu/packages/wxwidgets.scm:315:5: warning: python2-wxpython: margin comment
will be lost
gnu/packages/web.scm:6503:7: warning: python2-clf: non-trivial input, bailing
out
gnu/packages/web.scm:6503:7: warning: python-clf: non-trivial input, bailing out
gnu/packages/tryton.scm:594:5: warning: python-trytond-party: input label
'python-stnum' does not match package name, bailing out
gnu/packages/tls.scm:612:5: warning: python-acme: margin comment will be lost
gnu/packages/time.scm:435:5: warning: python-arrow: margin comment will be lost
gnu/packages/sphinx.scm:133:21: warning: python2-sphinx: computed input list,
bailing out
--8<---cut here---end--->8---
I don’t have hard figures but I think the majority of packages are
handled, which means we could do a big switch at once, or in a short
amount of time (so we can review removed comments and fix them up).
We could then forcefully convert some of the remaining cases, with the
understanding that the derivation would be different but presumably
valid; finally, there’d be the more complex cases that need to be
manually dealt with.
Thoughts?
Thanks,
Ludo’.
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2021 Ludovic Courtès
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix. If not, see
Re: [PATCH RFC 0/4] Getting rid of input labels?
Hi,
Ryan Prior skribis:
> I think there's an opportunity to avoid the need to "fall back" to the status
> quo, though. I picture a structure for inputs that has three cases, trivially
> decided based on data shape:
>
> - a bare symbol, eg '(tzdata glib)
> this is translated to `(("tzdata" ,tzdata) ("glib" ,glib))
> works the exact same if the symbol is an =origin=, you get the name of the
> symbol quoted followed by the value of the symbol.
A “bare symbol” cannot be translated into a reference to a variable in
the way you write.
I’m leaning towards either the new style:
(list glib tzdata)
or the old style:
(("tzdata" ,tzdata) ("glib" ,glib))
but not a mixture of both, which would require more code to be properly
interpreted, leading to slowdowns on the hot path and generally making
it more difficult to interpret what’s in there.
Thanks,
Ludo’.
Re: [PATCH RFC 0/4] Getting rid of input labels?
On Wednesday, May 26th, 2021 at 2:02 PM, Ludovic Courtès wrote:
> > Could the new syntax accept both variables and specifications, e.g.,
> >
> > (list "glib:bin" foo "bar@2.3")
> >
> > ?
>
> No! I mean, yes it could, but no, I don’t think that’s a good idea.
>
> :-)
>
> In terms of API, I prefer clarity; in this case, I think inputs should
>
> be a list of packages or other “lowerable” objects, rather than a list
>
> of “anything” that could be magically interpreted at run time.
I agree with this, a list of potentially ambiguous "any-type" inputs seems
fraught.
I think there's an opportunity to avoid the need to "fall back" to the status
quo, though. I picture a structure for inputs that has three cases, trivially
decided based on data shape:
- a bare symbol, eg '(tzdata glib)
this is translated to `(("tzdata" ,tzdata) ("glib" ,glib))
works the exact same if the symbol is an =origin=, you get the name of the
symbol quoted followed by the value of the symbol.
- a 2-tuple, eg '(tzdata ("gnome-lib" glib))
when we encounter a 2-tuple, we use the first value as the label and the
second as the value.
so this becomes `(("tzdata" ,tzdata) ("gnome-lib" ,glib))
- a 3-tuple, eg '(tzdata ("glib:bin" glib "bin"))
when we encounter a 3-tuple, we use the first value as the label, second as
the value, and third as the output name.
Following this convention, the inputs for most packages will be a list of bare
symbols, and packages which need custom labels and/or outputs for some inputs
can get them with little effort.
Cheers,
Ryan
Re: [PATCH RFC 0/4] Getting rid of input labels?
Ludovic Courtès schreef op wo 26-05-2021 om 15:43 [+0200]: > Hi Maxime, >[...] > In many cases, you don’t need the ability to refer to a specific input; > you just need all the inputs to contribute to search path environment > variables, and that’s enough. A “label collision” does not matter at > all in this case. > > In some cases, you do need to refer to a specific input, as in: > > [...] > In this case, there are now two options: > > 1. Arrange so that label is unique among your inputs, as is already > the case. > > 2. [...] > > Do you think there are unaddressed issues with go-ipfs-migrations? As long as there remains a possibility of overriding the ‘default‘ label generated from the package name, then everything seems ok for the (not-yet-packaged) go-ipfs-migrations. This appears to remain the case, so ok? Greetings, Maxime. signature.asc Description: This is a digitally signed message part
Re: [PATCH RFC 0/4] Getting rid of input labels?
Hello, Nicolas Goaziou skribis: > Ludovic Courtès writes: [...] >> • Packages such as ‘tzdata’ use labels to refer to non-package >> inputs. These cannot be converted to the automatic labeling >> style, or not without extra changes. > > Would it be possible to write something like > > (inputs (let ((tzcode (origin ...))) > (list ... tzcode ...))) > > ? Yes, but the problem is that the automatically-assigned label for records is “_” (a placeholder), because origins have no name, unlike packages. Thus, this phase: (replace 'unpack (lambda* (#:key source inputs #:allow-other-keys) (invoke "tar" "xvf" source) (invoke "tar" "xvf" (assoc-ref inputs "tzcode" … needs to be written differently, because there’s no “tzcode” label. One option on ‘core-updates’ is to use gexps: #~(modify-phases %standard-phases ;; … (replace 'unpack (lambda* (#:key source inputs #:allow-other-keys) (invoke "tar" "xvf" source) (invoke "tar" "xvf" #$tzcode However, this style breaks common uses of ‘inherit’ and uses of the inputs field: ‘tzcode’ here need not even be listed in ‘inputs’, and consequently, you cannot easily inherit from ‘tzdata’ and give it a different ‘tzcode’. We need to find and encourage appropriate idioms for corner cases like this. One option is the status quo: keep using labels in those rare cases. A crazier option would be to interpret input lists, when possible, as both input lists and formal parameter lists for ‘arguments’. Assume a package with: (inputs (list foo bar baz)) The ‘arguments’ field, which is currently a thunk, would magically be turned into: (lambda (foo bar baz) …) That way, arguments could refer to #$foo etc., and that’d just do the right thing when inputs are overridden. This would be reaching a level of magic that may not be wise, and it may be hard to implement. > Could the new syntax accept both variables and specifications, e.g., > >(list "glib:bin" foo "bar@2.3") > > ? No! I mean, yes it could, but no, I don’t think that’s a good idea. :-) In terms of API, I prefer clarity; in this case, I think inputs should be a list of packages or other “lowerable” objects, rather than a list of “anything” that could be magically interpreted at run time. More importantly, I think package specs are a UI feature, and that packages should be separate from UI concerns. Specs are implemented in (gnu packages), not in (guix …) for a reason: it’s a feature that makes sense in the distro, not in the core Guix machinery where there’s no “package set” to traverse. I hope that makes sense! Ludo’.
Re: [PATCH RFC 0/4] Getting rid of input labels?
Hi Maxime, Maxime Devos skribis: > Ludovic Courtès schreef op do 20-05-2021 om 16:58 [+0200]: >> Hello Guix! >> >> Here’s a proposal for a soft revolution: getting rid of input labels >> in package definitions. Instead of writing: [...] >> >> one can write: >> >> (native-inputs (list autoconf automake pkg-config guile-3.0)) >> [...] > > This concept LGTM (but I haven't looked closely at the patches), but > as noted on #guix, some issues with eliminating labels completely: > > A package definition of P may require both Q@1.0 and Q@2.0 as inputs, > in which case a ‘label collision’ would be created if we generate > labels package-name. More specifically, I'm thinking of packaging > go-ipfs-migrations (or what's its name ...). It would be a good idea > to add an (additional?) test to actually try to migrate from > go-ipfs@first-version to go-ipfs@another-version. Keep in mind that labels exist to make it easier to refer to a specific input from the build side—in a phase, configure flag, etc. In many cases, you don’t need the ability to refer to a specific input; you just need all the inputs to contribute to search path environment variables, and that’s enough. A “label collision” does not matter at all in this case. In some cases, you do need to refer to a specific input, as in: #:configure-flags (list (string-append "--with-gmp-prefix=" (assoc-ref %build-inputs "gmp"))) In this case, there are now two options: 1. Arrange so that label is unique among your inputs, as is already the case. 2. Use a gexp instead (possible on ‘core-updates’) like so: #:configure-flags #~(list (string-append "--with-gmp-prefix=" #$gmp)) or, to allow for inheritance: #:configure-flags #~(list (string-append "--with-gmp-prefix=" #$@(assoc-ref (package-inputs this-package) "gmp"))) The second variant is ugly, but we could provide helpers to make it prettier. Do you think there are unaddressed issues with go-ipfs-migrations? Thanks for your feedback! Ludo’.
Re: [PATCH RFC 0/4] Getting rid of input labels?
Hi Vincent,
Vincent Legoll skribis:
> What about
>
>> (native-inputs
>> `(,autoconf
>>("truc" ,muche)
>>"pkg-config"
>> ))
>
> i.e. allowing package objects, tuples and names, and it would DTRT ?
>
> Wouldn't something like that be possible ?
It would be possible, but I’d rather not allow for mixing styles. Not
allowing mixing style means it’s easier to check for correctness, and
the “auto labeling” code has fewer checks to make.
Ludo’.
Re: [PATCH RFC 0/4] Getting rid of input labels?
Hello,
Ludovic Courtès writes:
> Here’s a proposal for a soft revolution: getting rid of input labels
> in package definitions. Instead of writing:
>
> (native-inputs
> `(("autoconf" ,autoconf)
>("automake" ,automake)
>("pkg-config" ,pkg-config)
>("guile" ,guile-3.0)))
>
> one can write:
>
> (native-inputs (list autoconf automake pkg-config guile-3.0))
Nice! Thank you.
> • Packages such as ‘tzdata’ use labels to refer to non-package
> inputs. These cannot be converted to the automatic labeling
> style, or not without extra changes.
Would it be possible to write something like
(inputs (let ((tzcode (origin ...)))
(list ... tzcode ...)))
?
>
> • Currently, something like:
>
> (inputs (list glib))
>
> is converted to:
>
> (inputs `(("glib" ,glib)))
>
> Should it, instead, be converted to:
>
> (inputs `(("glib" ,glib)
>("glib:bin" ,glib "bin")))
>
> ? This would make the concise style strictly less
> expressive, but maybe good enough?
Could the new syntax accept both variables and specifications, e.g.,
(list "glib:bin" foo "bar@2.3")
?
Regards,
--
Nicolas Goaziou
Re: [PATCH RFC 0/4] Getting rid of input labels?
Ludovic Courtès schreef op do 20-05-2021 om 16:58 [+0200]: > Hello Guix! > > Here’s a proposal for a soft revolution: getting rid of input labels > in package definitions. Instead of writing: [...] > > one can write: > > (native-inputs (list autoconf automake pkg-config guile-3.0)) > [...] This concept LGTM (but I haven't looked closely at the patches), but as noted on #guix, some issues with eliminating labels completely: A package definition of P may require both Q@1.0 and Q@2.0 as inputs, in which case a ‘label collision’ would be created if we generate labels package-name. More specifically, I'm thinking of packaging go-ipfs-migrations (or what's its name ...). It would be a good idea to add an (additional?) test to actually try to migrate from go-ipfs@first-version to go-ipfs@another-version. Greetings, Maxime. signature.asc Description: This is a digitally signed message part
Re: [PATCH RFC 0/4] Getting rid of input labels?
Hello,
On Thu, May 20, 2021 at 5:03 PM Ludovic Courtès wrote:
> Instead of writing:
>
> (native-inputs
> `(("autoconf" ,autoconf)
>("automake" ,automake)
>("pkg-config" ,pkg-config)
>("guile" ,guile-3.0)))
>
> one can write:
>
> (native-inputs (list autoconf automake pkg-config guile-3.0))
What about
> (native-inputs
> `(,autoconf
>("truc" ,muche)
>"pkg-config"
> ))
i.e. allowing package objects, tuples and names, and it would DTRT ?
Wouldn't something like that be possible ?
--
Vincent Legoll
[PATCH RFC 0/4] Getting rid of input labels?
Hello Guix!
Here’s a proposal for a soft revolution: getting rid of input labels
in package definitions. Instead of writing:
(native-inputs
`(("autoconf" ,autoconf)
("automake" ,automake)
("pkg-config" ,pkg-config)
("guile" ,guile-3.0)))
one can write:
(native-inputs (list autoconf automake pkg-config guile-3.0))
With this patch set, this is just syntactic sugar: ‘package-inputs’
and friends still return a list of tuples. (These patches are
against ‘core-updates’ but that works equally well on ‘master’,
without a world rebuild.)
My understanding of the code is that this change adds no overhead
for packages written in the old style, and negligible overhead for
packages written in the new style (calling ‘package-name’
instead of referring to a literal string for the label.) I haven’t
tried to measure it though as it would require a massive conversion
to the new style to be really measurable.
I don’t think I need to expound on the benefits. :-)
There are issues and open questions:
• This hides labels, but they’re still visible as soon as one
fiddles with ‘package-inputs’ & co. So this lowers the barrier
to entry, but the difficulty of dealing with input tuples
does not disappear entirely.
• Both styles would be supported for a long time so contributors
would still have to know about input tuples anyway.
• There are packages that use custom labels as some sort of an
abstraction. For instance, the “kernel-headers” label is
associated either with ‘linux-libre-headers’ or with
‘hurd-headers’. In this case, the simplified style would
use the package name as the label, which isn’t appropriate,
or at least would require adjustments in packages that
rely on this.
• There are packages with same-named but different inputs,
and they rely on having a different input label.
• Some packages rely on labels that differ from the package
name (this is what the ‘guix lint -c input-labels’ patch
detects). For instance, commencement.scm has things like:
`(("guile" ,%bootstrap-guile))
Automatic labeling would convert it to:
`(("guile-bootstrap" ,%bootstrap-guile))
Not necessarily a problem: we can keep the old style for these.
Common Lisp packages typically lack the “cl-” prefix in
input labels but most likely they don’t actually refer to
those labels, so we should be fine.
• Packages such as ‘tzdata’ use labels to refer to non-package
inputs. These cannot be converted to the automatic labeling
style, or not without extra changes.
• Currently, something like:
(inputs (list glib))
is converted to:
(inputs `(("glib" ,glib)))
Should it, instead, be converted to:
(inputs `(("glib" ,glib)
("glib:bin" ,glib "bin")))
? This would make the concise style strictly less
expressive, but maybe good enough?
Ludovic Courtès (4):
records: Support field sanitizers.
DRAFT packages: Allow inputs to be plain package lists.
DRAFT gnu: Change inputs of core packages to plain lists.
DRAFT lint: Add 'input-labels' checker.
gnu/packages/base.scm | 30 +++
gnu/packages/guile.scm | 87 ++
gnu/packages/mes.scm | 23 ---
guix/lint.scm | 35 +
guix/packages.scm | 35 +++--
guix/records.scm | 65 ---
tests/records.scm | 38 ++
7 files changed, 198 insertions(+), 115 deletions(-)
--
2.31.1
