Python's native-inputs
Hi Guix, On some languages, there are a lot of unused native-inputs that are development & linting dependencies much more than packages that are actually used to build or test a package (I'm thinking at least Python and Rust). These fall in the category of tools "useful" at run time, but unecessary at build time. Is there a clear policy about their removal? I've seen the case of pre-commit in Python, and I've commited a series yesterday regarding pylint, but there are a whole lot of them in Python, at least : - black and extensions - codecov - coveralls - coverage - pytest-cov - pycodestyle - mypy and extensions - isort and extensions - flake8 and extensions - possibly virtualenv and extensions and I'm probably forgetting some. These packages make a lot of sense when considering things like `guix shell -D` but they are hampering some progress on Python packages since they are everywhere and a small update in their inputs rebuilds the whole python world (even though it has NO influence on the functionality of any other package). What are the guidelines in this case? I can propose a huge patch series (currently ~300 patches, and not finished), to remove them, lint against them and remove them from the importer as a default, but that's a big decision to make. IMO we should have a dev-inputs field to handle these cases, but that's even more work. -- Best regards, Nicolas Graves
Re: bug#63267: gcc-toolchain is missing libstdc++.so
Hi everyone, Apologies for the long delay on this. On Tue, May 09, 2023 at 07:07 PM, Simon Tournier wrote: > Hi, > > I am proposing patch#63393 [1] which adds the output lib to > gcc-toolchain. Well, quoting the comment: > > ;; The main raison d'être of this "meta-package" is (1) to conveniently > ;; install everything that we need, and (2) to make sure ld-wrapper > comes > ;; before Binutils' ld in the user's profile. > > I think, this package gcc-toolchain should be the visible package and > battery included. > > WDYT? > > 1: https://issues.guix.gnu.org/issue/63393 > > Cheers, > simon I've just pushed, as b47ae1ecc43baaf726701ab2d2f810ecfaa75428, the patches from that issue (the first from simon, the second my simplified one just adding gcc:lib to gcc-toolchain). This should fix the original bug here, so I am closing. However, it was raised here and in 63393 about alternatives in how we use gcc-toolchain outputs. As well as the issue I ran into about make-libstdc++. So, if anyone would like to change anything from the new status quo, please open a new issue. At least now we are working from a better default I would say, with gcc-toolchain including the libraries from gcc:lib. Thanks all, John
Re: No default OpenJDK version?
Makes sense, we can do that.
Le 16 avril 2024 23:50:54 GMT+02:00, Vagrant Cascadian
a écrit :
>On 2024-04-16, Julien Lepiller wrote:
>> Currently, most java packages use the implicit jdk from the build
>> system (ant- or maven-build-system), which is… icedtea@8. We still
>> have quite a lot of old packages that don't build with openjdk9, so
>> I'm not sure when we can update the default jdk…
>
>But for the packages that explicitly need to pull in openjdk:
>
> git grep 'openjdk[1-9]' | grep -v define-public | grep -v java.scm | nl
> 1 gnu/packages/android.scm:(native-inputs (list openjdk12))
> 2 gnu/packages/apl.scm:(inputs (list bash-minimal openjdk18))
> 3 gnu/packages/apl.scm:(native-inputs (list `(,openjdk18 "jdk") zip))
> 4 gnu/packages/axoloti.scm: `(("openjdk" ,openjdk11 "jdk")
> 5 gnu/packages/bioconductor.scm: (list (list openjdk11 "jdk")
> 6 gnu/packages/bioinformatics.scm: #:jdk openjdk11))
> 7 gnu/packages/cran.scm: ("jdk" ,openjdk11 "jdk")
> 8 gnu/packages/diffoscope.scm: (list `(,openjdk12 "jdk")
> 9 gnu/packages/emacs-xyz.scm: (list emacs-ecukes emacs-espuds
> emacs-undercover openjdk9))
>10 gnu/packages/geo.scm: openjdk11))
>11 gnu/packages/geo.scm: #:jdk ,openjdk11
>12 gnu/packages/groovy.scm: #:jdk ,openjdk9
>13 gnu/packages/groovy.scm: #:jdk ,openjdk9
>14 gnu/packages/groovy.scm: #:jdk ,openjdk9
>15 gnu/packages/groovy.scm: #:jdk ,openjdk9
>16 gnu/packages/groovy.scm: #:jdk ,openjdk9
>17 gnu/packages/gstreamer.scm: ("openjdk" ,openjdk14)
>18 gnu/packages/gstreamer.scm: ("openjdk:jdk" ,openjdk14 "jdk")
>19 gnu/packages/java-compression.scm: #:jdk ,openjdk9
>20 gnu/packages/kodi.scm: openjdk9 ;like
> groovy
>21 gnu/packages/mpi.scm: `(("jdk" ,openjdk11 "jdk")
>22 gnu/packages/pep.scm: `(,openjdk9 "jdk") which yml2))
>
>
>It is not a crazy number of packages, but in cases like diffoscope and
>enjarify, which seem fairly compatible with arbitrary versions, it would
>be nice to have an unversioned option to specify?
>
>Maybe I confused things by talking about the "default" version, I guess
>I was wondering if it would make sense for an "openjdk-latest" or
>"openjdk-lts" or whatever color you want, just as long as it does not
>require specifying (and more importantly, keeping track of) the version
>of openjdk needed.
>
>live well,
> vagrant
Re: No default OpenJDK version?
On 2024-04-16, Julien Lepiller wrote:
> Currently, most java packages use the implicit jdk from the build
> system (ant- or maven-build-system), which is… icedtea@8. We still
> have quite a lot of old packages that don't build with openjdk9, so
> I'm not sure when we can update the default jdk…
But for the packages that explicitly need to pull in openjdk:
git grep 'openjdk[1-9]' | grep -v define-public | grep -v java.scm | nl
1 gnu/packages/android.scm:(native-inputs (list openjdk12))
2 gnu/packages/apl.scm:(inputs (list bash-minimal openjdk18))
3 gnu/packages/apl.scm:(native-inputs (list `(,openjdk18 "jdk") zip))
4 gnu/packages/axoloti.scm: `(("openjdk" ,openjdk11 "jdk")
5 gnu/packages/bioconductor.scm: (list (list openjdk11 "jdk")
6 gnu/packages/bioinformatics.scm: #:jdk openjdk11))
7 gnu/packages/cran.scm: ("jdk" ,openjdk11 "jdk")
8 gnu/packages/diffoscope.scm: (list `(,openjdk12 "jdk")
9 gnu/packages/emacs-xyz.scm: (list emacs-ecukes emacs-espuds
emacs-undercover openjdk9))
10 gnu/packages/geo.scm: openjdk11))
11 gnu/packages/geo.scm: #:jdk ,openjdk11
12 gnu/packages/groovy.scm: #:jdk ,openjdk9
13 gnu/packages/groovy.scm: #:jdk ,openjdk9
14 gnu/packages/groovy.scm: #:jdk ,openjdk9
15 gnu/packages/groovy.scm: #:jdk ,openjdk9
16 gnu/packages/groovy.scm: #:jdk ,openjdk9
17 gnu/packages/gstreamer.scm: ("openjdk" ,openjdk14)
18 gnu/packages/gstreamer.scm: ("openjdk:jdk" ,openjdk14 "jdk")
19 gnu/packages/java-compression.scm: #:jdk ,openjdk9
20 gnu/packages/kodi.scm: openjdk9 ;like
groovy
21 gnu/packages/mpi.scm: `(("jdk" ,openjdk11 "jdk")
22 gnu/packages/pep.scm: `(,openjdk9 "jdk") which yml2))
It is not a crazy number of packages, but in cases like diffoscope and
enjarify, which seem fairly compatible with arbitrary versions, it would
be nice to have an unversioned option to specify?
Maybe I confused things by talking about the "default" version, I guess
I was wondering if it would make sense for an "openjdk-latest" or
"openjdk-lts" or whatever color you want, just as long as it does not
require specifying (and more importantly, keeping track of) the version
of openjdk needed.
live well,
vagrant
signature.asc
Description: PGP signature
Re: No default OpenJDK version?
> Currently, most java packages use the implicit jdk from the build > system (ant- or maven-build-system), which is… icedtea@8. We still > have quite a lot of old packages that don't build with openjdk9, so > I'm not sure when we can update the default jdk… does that prevent the introduction of a newer default JDK and annotating the old java dependency on the (hopefully few) packages that fail to build with a newer default? -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “Malthus was right. It's hard to see how the solar system could support much more than 10^28 people or the universe more than 10^50.” — John McCarthy (1927–2011), father of Lisp
Re: No default OpenJDK version?
Currently, most java packages use the implicit jdk from the build system (ant- or maven-build-system), which is… icedtea@8. We still have quite a lot of old packages that don't build with openjdk9, so I'm not sure when we can update the default jdk… Le 16 avril 2024 22:25:33 GMT+02:00, Vagrant Cascadian a écrit : >When recently taking a look at diffoscope, I was reminded that there is >effectively no default openjdk version, you have to pick a specific >version for each package definition... > >At some time in diffoscope's history, that was openjdk@12. > >But there are quite a few versions to choose from: > > guix package -A openjdk | sort -V > openjdk 9.181 out,jdk,doc gnu/packages/java.scm:869:2 > openjdk 10.46 out,jdk,doc gnu/packages/java.scm:1140:2 > openjdk 11.0.22 out,jdk,doc gnu/packages/java.scm:1218:2 > openjdk 12.33 out,jdk,doc gnu/packages/java.scm:1536:2 > openjdk 13.0.14 out,jdk,doc gnu/packages/java.scm:1576:2 > openjdk 14.0.2 out,jdk,doc gnu/packages/java.scm:1583:2 > openjdk 15.0.10 out,jdk,doc gnu/packages/java.scm:1598:2 > openjdk 16.0.2 out,jdk,doc gnu/packages/java.scm:1617:2 > openjdk 17.0.10 out,jdk,doc gnu/packages/java.scm:1625:2 > openjdk 18.0.2.1out,jdk,doc gnu/packages/java.scm:1642:2 > openjdk 19.0.2 out,jdk,doc gnu/packages/java.scm:1646:2 > openjdk 20.0.2 out,jdk,doc gnu/packages/java.scm:1663:2 > openjdk 21.0.2 out,jdk,doc gnu/packages/java.scm:1667:2 > >Some packages may only work with a specific era of openjdk, but I >suspect many of the packages in guix just picked whatever version >happened to be present when it was added to guix. > >Which makes it hard to know when to update the openjdk dependency... > >In the diffoscope case, it seems to have work fine with openjdk@21, with >the only result being that some openjdk-version-specific tests pass and >some are skipped as a one-for-one trade compared to the old openjdk@12. > >Alternately, I would be tempted to switch to openjdk@17, which is the >current default in Debian, so has a little more testing behind it... > >Though there is a bit of a perverse incentive to stick with the oldest >version that still works, due to openjdk having a very long bootstrap >chain of itself... > >And then the question gets to be of diffoscope's dependencies, what >versions of openjdk do they pull in (notably enjarify, which uses >openjdk@12, although that also seems to work ok with openjdk@21)? > > >Would it make sense to have an openjdk "default" version, so packages >could instead depend on that, and only need to specify a version if >needed for some particular reason? Or is compatibility across openjdk >versions troublesome enough that it really always needs to be handled on >a case-by-case basis? > > >live well, > vagrant
No default OpenJDK version?
When recently taking a look at diffoscope, I was reminded that there is effectively no default openjdk version, you have to pick a specific version for each package definition... At some time in diffoscope's history, that was openjdk@12. But there are quite a few versions to choose from: guix package -A openjdk | sort -V openjdk 9.181 out,jdk,doc gnu/packages/java.scm:869:2 openjdk 10.46 out,jdk,doc gnu/packages/java.scm:1140:2 openjdk 11.0.22 out,jdk,doc gnu/packages/java.scm:1218:2 openjdk 12.33 out,jdk,doc gnu/packages/java.scm:1536:2 openjdk 13.0.14 out,jdk,doc gnu/packages/java.scm:1576:2 openjdk 14.0.2 out,jdk,doc gnu/packages/java.scm:1583:2 openjdk 15.0.10 out,jdk,doc gnu/packages/java.scm:1598:2 openjdk 16.0.2 out,jdk,doc gnu/packages/java.scm:1617:2 openjdk 17.0.10 out,jdk,doc gnu/packages/java.scm:1625:2 openjdk 18.0.2.1out,jdk,doc gnu/packages/java.scm:1642:2 openjdk 19.0.2 out,jdk,doc gnu/packages/java.scm:1646:2 openjdk 20.0.2 out,jdk,doc gnu/packages/java.scm:1663:2 openjdk 21.0.2 out,jdk,doc gnu/packages/java.scm:1667:2 Some packages may only work with a specific era of openjdk, but I suspect many of the packages in guix just picked whatever version happened to be present when it was added to guix. Which makes it hard to know when to update the openjdk dependency... In the diffoscope case, it seems to have work fine with openjdk@21, with the only result being that some openjdk-version-specific tests pass and some are skipped as a one-for-one trade compared to the old openjdk@12. Alternately, I would be tempted to switch to openjdk@17, which is the current default in Debian, so has a little more testing behind it... Though there is a bit of a perverse incentive to stick with the oldest version that still works, due to openjdk having a very long bootstrap chain of itself... And then the question gets to be of diffoscope's dependencies, what versions of openjdk do they pull in (notably enjarify, which uses openjdk@12, although that also seems to work ok with openjdk@21)? Would it make sense to have an openjdk "default" version, so packages could instead depend on that, and only need to specify a version if needed for some particular reason? Or is compatibility across openjdk versions troublesome enough that it really always needs to be handled on a case-by-case basis? live well, vagrant signature.asc Description: PGP signature
Re: Functional package interface
In the code there, the function propagate-packages will indeed try and
propagate every package through the graph node.
I looked at your code, it's comprehensive. I wasn't sure such a thing
was possible with input rewriting, although I think there's still a lot
of benefit to the functional approach, like this:
- some packages are present under different names (emacs has
emacs-minimal, emacs, emacs-pgtk, emacs-wide-int...) and that is not
handled by this function.
It should handle it pretty elegantly, since those packages inherit from
emacs, if you replace emacs they'll inherit from your replacement while
keeping their unique traits.
- propagating some packages that are used in build-systems will trigger
massive rebuilds (e.g. go, emacs, python, ocaml, texlive), you'll
probably have to be smart about what you want to avoid to propagate, but
it's great because guix can already hide a package to
package-inputs-rewriting using (hidden-package {package-name}). There are some
examples in the series.
No idea how, but I guess I should implement a #:mask-hidden? argument
to handle this. I have more basic issues for now, but that's a good
point.
- this has a cost. I haven't timed it yet, but the slowdown is
noticeable, though still quite usable.
I'm doing a couple stupid things, it's improved with the last few
commits, but there's still a long way to go. If there's a cost to
syntax rule abuse, you were probably feeling that.
- you of course have to get the packages into the function from a
profile. Here's it's done in RDE's code, but that should be quite
doable in Guix too.
Yeah, the implementation model is kinda strange. Having to put extra
code in every single module in the entire repo isn't a great
prerequisite for anything. It could be the point that prevents this
from manifesting. The mile long butt-ugly list of inputs that it needs
doesn't help.
Thanks for the tips. Work is still ongoing, but recursive propagation
now works when giving one package-set another as an input, so the
intended use-case is now possible.
- sc
Re: Functional package interface
Hi Nicolas Isn't what you're trying to do already in Guix? Have a look at package-inputs-rewrite right there : https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html I want to have this but more powerful, I'd like to do something like define an operating system with all instances of a package replaced. Maybe something like that is already possible, but I haven't figured out a way. As far as I know, with package-input-rewriting you would still have to manually add it to all the services in your operating-system definition. Or just use module-set! and manage the environment state some other way. You also only have access to the input fields of packages, this is something parameterize packages tackles too, being able to add inputs that go anywhere in the package definition. Build phases for example, to enable common compile options across multiple packages more easily. I've also been working on general package propagation in RDE, it's still a work in progress, but it's not far from it : https://lists.sr.ht/~abcdw/rde-devel/patches/49956 I'll take a look. I knew there had to be more people working on this idea, but it's not always easy to find these projects. - sc
Re: Functional package interface
On 2024-04-16 01:17, spacecadet wrote:
> Hi Nicolas
>
>> Isn't what you're trying to do already in Guix? Have a look at
>> package-inputs-rewrite right there :
>> https://guix.gnu.org/manual/devel/en/html_node/Defining-Package-Variants.html
>
> I want to have this but more powerful, I'd like to do something like
> define an operating system with all instances of a package replaced.
> Maybe something like that is already possible, but I haven't figured
> out a way.
>
> As far as I know, with package-input-rewriting you would still have to
> manually add it to all the services in your operating-system
> definition. Or just use module-set! and manage the environment state
> some other way.
>
> You also only have access to the input fields of packages, this is
> something parameterize packages tackles too, being able to add inputs
> that go anywhere in the package definition. Build phases for example,
> to enable common compile options across multiple packages more easily.
>
>> I've also been working on general package propagation in RDE, it's still
>> a work in progress, but it's not far from it :
>> https://lists.sr.ht/~abcdw/rde-devel/patches/49956
>
> I'll take a look. I knew there had to be more people working on this idea,
> but it's not always easy to find these projects.
In the code there, the function propagate-packages will indeed try and
propagate every package through the graph node.
Four tricky points though :
- propagating some packages that are used in build-systems will trigger
massive rebuilds (e.g. go, emacs, python, ocaml, texlive), you'll
probably have to be smart about what you want to avoid to propagate, but
it's great because guix can already hide a package to
package-inputs-rewriting using (hidden-package {package-name}). There are some
examples in the series.
- some packages are present under different names (emacs has
emacs-minimal, emacs, emacs-pgtk, emacs-wide-int...) and that is not
handled by this function.
- this has a cost. I haven't timed it yet, but the slowdown is
noticeable, though still quite usable.
- you of course have to get the packages into the function from a
profile. Here's it's done in RDE's code, but that should be quite
doable in Guix too.
>
> - sc
--
Best regards,
Nicolas Graves
Re: Fix grammar and markup (was Re: Feedback of the GNU Guix manual)
Hi Matt. When triple checking, I read in “info "(texinfo)@pxref"”:
In past versions of Texinfo, it was not allowed to write punctuation
after a '@pxref', so it could be used _only_ before a right parenthesis.
This is no longer the case. The effect of '@pxref{NODE-NAME}' is
similar to that of 'see @ref{NODE-NAME}'. However, in many circumstance
the latter is preferrable, as this makes it clear in the Info output
that the word "see" should be present.
Because of this last sentence, my tendency would now be to use @pxref
only for parentheses. (Texinfo is really confusing.) Should I just cut
these changes:
(Linux Services): Use @pxref to start phrase.
(Configuring the Shell): Use @pxref to start phrase.
or is there more that should go in this patch?
Matt writes:
> On Mon, 15 Apr 2024 14:58:50 +0200 pelzflorian (Florian Pelz) wrote
> ---
> > I believe the Emacs errors are historical; looking at Emacs 29.3 as
> > packaged in Guix, Emacs-Info displays xref properly as See. Display
> > errors had existed, but in the past only.
>
> I reported it to emacs-devel (it's addressed now) and the issue isn't
> technically @xrefs; it's the compilation of @xref and @ref, '*Note'
> and '*note' respectively. Emacs looks at the context around these and
> renders them as 'See' or 'see' according to what's nearby. […]
Ohh thank you and them for the fix.
Regards,
Florian
