[PATCH] display grafted package

[zimoun]

Hi Mark and Andreas

Quoting from .

On Tue, 23 Mar 2021 at 19:42, Mark H Weaver  wrote:

> How about changing "guix package -A" and "guix package -s" to display
> information about the package's replacement, if it has one?
>
> Alternatively, those commands could somehow explicitly indicate that the
> package has been grafted, and show the version number of the
> replacement, in such a way that is less confusing to users.

The attached patch does that only for ’package->recutils’ (show and
search).  For instance, note the ’replaced’ field for the grafted
package.  (Obviously, it could be any other word than ’replaced’
compatible with the recutils record.)

--8<---cut here---start->8---
$ time ./pre-inst-env guix show zstd
name: zstd
version: 1.4.9
outputs: out lib static
systems: x86_64-linux i686-linux
dependencies: 
location: gnu/packages/compression.scm:1473:2
homepage: https://facebook.github.io/zstd/
license: Modified BSD, FreeBSD, GPL 2, GPL 3+, Expat, Public Domain, Zlib
synopsis: Zstandard real-time compression algorithm  
description: Zstandard (`zstd') is a lossless compression algorithm that
+ combines very fast operation with a compression ratio comparable to that of
+ zlib.  In most scenarios, both compression and decompression can be performed
+ in ‘real time’.  The compressor can be configured to provide the most suitable
+ trade-off between compression ratio and speed, without affecting decompression
+ speed.

name: zstd
version: 1.4.4
replaced: 1.4.9
outputs: out lib static
systems: x86_64-linux i686-linux
dependencies: 
location: gnu/packages/compression.scm:1402:2
homepage: https://facebook.github.io/zstd/
license: Modified BSD, FreeBSD, GPL 2, GPL 3+, Expat, Public Domain, Zlib
synopsis: Zstandard real-time compression algorithm  
description: Zstandard (`zstd') is a lossless compression algorithm that
+ combines very fast operation with a compression ratio comparable to that of
+ zlib.  In most scenarios, both compression and decompression can be performed
+ in ‘real time’.  The compressor can be configured to provide the most suitable
+ trade-off between compression ratio and speed, without affecting decompression
+ speed.

real0m0.822s
user0m1.039s
sys 0m0.057s
--8<---cut here---end--->8---

On my machine, it slows down from 0.5s to 0.8s; I do not know why.


How display such information for ’package -A’?  The (selected) output
looks like:

--8<---cut here---start->8---
zstd1.4.9   out,lib,static  gnu/packages/compression.scm:1473:2
zstd1.4.4   out,lib,static  gnu/packages/compression.scm:1402:2
--8<---cut here---end--->8---

And it appears to me hard to add another field or add something to the
’1.4.4’ line.  I mean, it would probably break some script.  Maybe some
people use this format to pipe.  I do not know.

WDYT?

Cheers,
simon

diff --git a/guix/ui.scm b/guix/ui.scm
index 7fbd4c63a2..b6497f5e5c 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -1528,9 +1528,18 @@ HYPERLINKS? is true, emit hyperlink escape sequences when appropriate."
   (define (package

Needed: tooling to detect references to buggy */stable packages (was: Re: [PATCHES] ImageMagick security updates without grafting)

[Mark H Weaver]

Earlier, I wrote:
> One thing to be very careful about is to only use 'gtk-doc/stable',
> 'dblatex/stable', and 'imagemagick/stable' in native-inputs, and
> moreover to make sure that no references to these */stable packages
> remain in any package outputs.
>
> Of course, if any package retains references to its 'native-inputs',
> that's always a bug, but I wouldn't be surprised if such bugs exist in
> Guix.  Such bugs might be relatively harmless now (except when
> cross-compiling), but they could become a security bug if a package
> retains a reference to 'imagemagick/stable'.

It occurs to me that we will need some tooling to ensure that no
references to these buggy "*/stable" packages end up in package outputs
that users actually use.  Otherwise, it is likely that sooner or later,
a runtime reference to one of these buggy packages will sneak in to our
systems.

An initial idea is that these "*/stable" packages could have a package
property (perhaps named something like 'build-time-only') that indicates
that references to its outputs should not occur within the outputs of
any other package that does not have that same property.

We'd also need to somehow ensure that users don't install these
'build-time-only' packages directly, at least not without an additional
option (e.g. --force-unsafe-build-time-only) to override it.

Additionally, it might be good to issue warnings if 'build-time-only'
packages are not hidden, or if they are found within the 'inputs' or
'propagated-inputs' fields of any package that's not also
'build-time-only'.  Both of these last two checks have loopholes,
however, so they are not reliable indicators.

Thoughts?  Other proposals?

 Regards,
   Mark

Re: [PATCHES] ImageMagick security updates without grafting

[Maxime Devos]

On Sun, 2021-03-28 at 17:37 -0400, Mark H Weaver wrote:
> Maxime Devos  writes:
> 
> > On Sat, 2021-03-27 at 20:01 -0400, Mark H Weaver wrote:
> > > [...]
> > > Maxime wrote:
> > > > What does ‘guix refresh --list-dependent imagemagick@6.9.11-48’
> > > > output now?
> > > When I last checked, it reported on the order of 2400 dependent package
> > > rebuilds.
> > 
> > I should have written imagemagick@6.9.12-4 here.
> 
> On my Guix system, after applying my recent patch set, "guix refresh -l
> imagemagick" (which refers to imagemagick@6.9.12-4) reports 603
> dependent packages.

I had a similar number (after "guix pull").  Looks good.

> I see that, according to our guidelines, since this number is greater
> than 300, it implies that updates to 'imagemagick' should not be done on
> the 'master' branch.
> 
> On the other hand, for what it's worth, on my own GNOME system, the
> number of rebuilds from my patch set was quite minimal, and *far* less
> than the number of rebuilds than I usually need to do when updating my
> system to the latest 'master' after just a few days.
> 
> I should say that I'm fully in support of having guidelines like this to
> limit the number of rebuilds on 'master'.  It's especially important to
> me since I never use substitutes, and build everything locally on my
> (rather old) Thinkpad X200.
> 
> That said, _number_ of dependent packages is not a good measure of what
> we should be trying to minimize.  I can build hundreds of 'python-*'
> packages in the time it takes to build a single package like 'webkitgtk'
> or 'icecat'.
> 
> A better measure might try to estimate the total amount of *build time*
> suffered by _all_ Guix users, as a result of updating a given package.
> That would depend on both (1) the estimated _time_ needed to build the
> dependent packages, and (2) the estimated number of users of those
> dependent packages, perhaps based on download statistics from our
> substitute servers.

That seems a good idea, but something to discuss in a new thread
(or a bug report to make sure it is not forgotten).

> > You missed a few packages:
> > in gnu/packages/mate.scm: search for "gtk-doc".
> > Also, the (gnu packages imagemagick) import seems
> > unused.
> 
> I did not attempt to comprehensively change all 'native-inputs'
> references of 'gtk-doc' to 'gtk-doc/stable'.  I stopped when the number
> of rebuilds on my own GNOME system became quite minimal.  That's why the
> summary line of commit 9dea1618755891526f708aa335b4136c1302d16e ends
> with the words "selected packages".

I have began writing a patch that changes *all* references of
gtk-doc to gtk-doc/stable (in native-inputs only).

> However, I see now that we should continue working on this, at least
> until we can update 'imagemagick' on 'master' without violating our
> guidelines.
> 
> > Looking at the package graph, many packages depend on imagemagick
> > through python-sphinx, so it may be worthwile to define a
> > python-sphinx/stable and use it instead of python-sphinx in the
> > native-inputs.
> > 
> > I suggest
> >  guix graph --type=reverse-package imagemagick@6.9.12-4 | dot -Tpdf > a.pdf
> > 
> > to find out if there are more uses for imagemagick/stable.
> 
> That's a good idea.  Would you like to work on it?

Yes.

> One thing to be very careful about is to only use 'gtk-doc/stable',
> 'dblatex/stable', and 'imagemagick/stable' in native-inputs, and
> moreover to make sure that no references to these */stable packages
> remain in any package outputs.
> 
> Of course, if any package retains references to its 'native-inputs',
> that's always a bug, but I wouldn't be surprised if such bugs exist in
> Guix.  Such bugs might be relatively harmless now (except when
> cross-compiling), but they could become a security bug if a package
> retains a reference to 'imagemagick/stable'.

I'll be careful!

> On my own system and user profile, which includes GNOME, I'm glad to
> report that I have *no* references to 'imagemagick' at all, not even to
> its newest release, and that's my strong preference.

Note to self, before I forget how to test this:

guix build $PACKAGES
# maybe guix build $PACKAGES --no-grafts?
guix graph --type=references $PACKAGES
# ^ look in output for "imagemagick".

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: [PATCHES] ImageMagick security updates without grafting

[Mark H Weaver]

Maxime Devos  writes:

> On Sat, 2021-03-27 at 20:01 -0400, Mark H Weaver wrote:
>> [...]
>> Maxime wrote:
>> > What does ‘guix refresh --list-dependent imagemagick@6.9.11-48’
>> > output now?
>
>> When I last checked, it reported on the order of 2400 dependent package
>> rebuilds.
>
> I should have written imagemagick@6.9.12-4 here.

On my Guix system, after applying my recent patch set, "guix refresh -l
imagemagick" (which refers to imagemagick@6.9.12-4) reports 603
dependent packages.

I see that, according to our guidelines, since this number is greater
than 300, it implies that updates to 'imagemagick' should not be done on
the 'master' branch.

On the other hand, for what it's worth, on my own GNOME system, the
number of rebuilds from my patch set was quite minimal, and *far* less
than the number of rebuilds than I usually need to do when updating my
system to the latest 'master' after just a few days.

I should say that I'm fully in support of having guidelines like this to
limit the number of rebuilds on 'master'.  It's especially important to
me since I never use substitutes, and build everything locally on my
(rather old) Thinkpad X200.

That said, _number_ of dependent packages is not a good measure of what
we should be trying to minimize.  I can build hundreds of 'python-*'
packages in the time it takes to build a single package like 'webkitgtk'
or 'icecat'.

A better measure might try to estimate the total amount of *build time*
suffered by _all_ Guix users, as a result of updating a given package.
That would depend on both (1) the estimated _time_ needed to build the
dependent packages, and (2) the estimated number of users of those
dependent packages, perhaps based on download statistics from our
substitute servers.

>> > If it there are many dependent packages, could some
>> > of them use imagemagick/stable, dblatex/stable or gtk-doc/stable
>> > as well?
>> 
>> Yes, that's exactly the purpose of this patch set.  Although at present,
>> the only user of 'imagemagick/stable' is 'dblatex/stable', and the only
>> user of 'dblatex/stable' is 'gtk-doc/stable'.
>
> You missed a few packages:
> in gnu/packages/mate.scm: search for "gtk-doc".
> Also, the (gnu packages imagemagick) import seems
> unused.

I did not attempt to comprehensively change all 'native-inputs'
references of 'gtk-doc' to 'gtk-doc/stable'.  I stopped when the number
of rebuilds on my own GNOME system became quite minimal.  That's why the
summary line of commit 9dea1618755891526f708aa335b4136c1302d16e ends
with the words "selected packages".

However, I see now that we should continue working on this, at least
until we can update 'imagemagick' on 'master' without violating our
guidelines.

> Looking at the package graph, many packages depend on imagemagick
> through python-sphinx, so it may be worthwile to define a
> python-sphinx/stable and use it instead of python-sphinx in the
> native-inputs.
>
> I suggest
>  guix graph --type=reverse-package imagemagick@6.9.12-4 | dot -Tpdf > a.pdf
>
> to find out if there are more uses for imagemagick/stable.

That's a good idea.  Would you like to work on it?

One thing to be very careful about is to only use 'gtk-doc/stable',
'dblatex/stable', and 'imagemagick/stable' in native-inputs, and
moreover to make sure that no references to these */stable packages
remain in any package outputs.

Of course, if any package retains references to its 'native-inputs',
that's always a bug, but I wouldn't be surprised if such bugs exist in
Guix.  Such bugs might be relatively harmless now (except when
cross-compiling), but they could become a security bug if a package
retains a reference to 'imagemagick/stable'.

On my own system and user profile, which includes GNOME, I'm glad to
report that I have *no* references to 'imagemagick' at all, not even to
its newest release, and that's my strong preference.

 Regards,
   Mark

Re: GNOME 40

[Mark H Weaver]

Léo Le Bouter  writes:

> If anyone is curious of the work or wants to participate, we are
> working there: 
> https://git.guix-patches.cbaines.net/guix-patches/log/?h=wip-gnome-40
>
> The branch is based on core-updates and we will rebase it every now and
> then, as well as merging patches to official core-updates as we feel it
> is necessary.
>
> cbaines gave us access to the git repo, it's not official so it's more
> flexible,

How is it more flexible than a "wip-*" branch on Savannah?

 Thanks,
   Mark

Re: Document our WIP

[Vincent Legoll]

Hello,

Mathieu, thanks for the additions.

Can someone having some experience with pinebook-pro
double-check this item ?

Vagrant, you seem to have some good results, can you add
something about them ?

Any other subjects worth mentionning there ?

https://libreplanet.org/wiki/Group:Guix/WorkInProgress

-- 
Vincent Legoll

Re: GNOME 40

[Léo Le Bouter]

If anyone is curious of the work or wants to participate, we are
working there: 
https://git.guix-patches.cbaines.net/guix-patches/log/?h=wip-gnome-40

The branch is based on core-updates and we will rebase it every now and
then, as well as merging patches to official core-updates as we feel it
is necessary.

cbaines gave us access to the git repo, it's not official so it's more
flexible, right now the access cbaines gives is all in and trustful,
but in the future they plan on getting branch access control with 
https://gitolite.com/gitolite/ so that people can be given access to
wip branches with associated continuous integration infrastructure for
development.

If anyone wants to participate, please reach to cbaines about access.

Léo


signature.asc
Description: This is a digitally signed message part

GNOME 40

[Raghav Gururajan]

Hello Guix!

I am starting this thread for the work on GNOME 40.

Currently, Léo Le Bouter  and I are working together on this. 
Any one is welcome to join. :-)


P.S.
Anyone replying to this thread, please `reply-all` or manually `CC` all 
participants.


Thank you!

Regards,
RG.



OpenPGP_signature
Description: OpenPGP digital signature

Re: Packaging

[Maxime Devos]

Please keep guix-devel@gnu.org in CC: or To:.

On Sun, 2021-03-28 at 10:18 +, mecqor labi wrote:
> Thanks for your help,
> The Shadowsocks package does not work in Guix at all, and this has been 
> reported over the developers of Guix, but it has never been solved.
> About Tor, this is not what I want, speed of the program is very slow, I have 
> problem in upload files and other ...
> 
> [...]

Guix has a package importer for rust crates:
  "guix import crate shadowsocks"

I imported the shadowsocks crate and some dependencies (see attachment).
(To build: "guix build -f d.scm")  Some dependencies are still missing,
but I hope this is enough to get you started with writing the package
definition.

Greetings,
Maxime

(use-modules (gnu packages crates-io)
((guix licenses) #:prefix license:)
 (guix build-system cargo)
 (guix packages)
 (guix download)
 (guix git-download))

(define-public rust-spin-0.9
  (package
(name "rust-spin")
(version "0.9.0")
(source
  (origin
(method url-fetch)
(uri (crate-uri "spin" version))
(file-name
  (string-append name "-" version ".tar.gz"))
(sha256
  (base32
"0y2406b02rkn1p1mfcb9vn6mxy1nb222k2gfq5b2lcw1rfcbyyxq"
(build-system cargo-build-system)
(arguments
  `(#:cargo-inputs
(("rust-lock-api" ,rust-lock-api-0.4
(home-page
  "https://github.com/mvdnes/spin-rs.git;)
(synopsis
  "Spin-based synchronization primitives")
(description
  "Spin-based synchronization primitives")
(license license:expat)))

(define-public rust-notify-5
  (package
(name "rust-notify")
(version "5.0.0-pre.6")
(source
  (origin
(method url-fetch)
(uri (crate-uri "notify" version))
(file-name
  (string-append name "-" version ".tar.gz"))
(sha256
  (base32
"1hzlm8lscdbrd2gsniikqv4cv0g0w10yfpp61b0bkv9l6jwq5zg5"
(build-system cargo-build-system)
(arguments
  `(#:cargo-inputs
(("rust-anymap" ,rust-anymap-0.12)
 ("rust-bitflags" ,rust-bitflags-1)
 ("rust-crossbeam-channel"
  ,rust-crossbeam-channel-0.5)
 ("rust-filetime" ,rust-filetime-0.2)
 ("rust-fsevent" ,rust-fsevent-2)
 ("rust-fsevent-sys" ,rust-fsevent-sys-3)
 ("rust-inotify" ,rust-inotify-0.9)
 ("rust-libc" ,rust-libc-0.2)
 ("rust-mio" ,rust-mio-0.7)
 ("rust-serde" ,rust-serde-1)
 ("rust-walkdir" ,rust-walkdir-2)
 ("rust-winapi" ,rust-winapi-0.3))
#:cargo-development-inputs
(("rust-serde-json" ,rust-serde-json-1
(home-page "https://github.com/notify-rs/notify;)
(synopsis
  "Cross-platform filesystem notification library")
(description
  "Cross-platform filesystem notification library")
(license license:cc0)))

(define-public rust-nix-0.20
  (package
(name "rust-nix")
(version "0.20.0")
(source
  (origin
(method url-fetch)
(uri (crate-uri "nix" version))
(file-name
  (string-append name "-" version ".tar.gz"))
(sha256
  (base32
"12n1syfd967hblrcrrqk63a4s1h4hsybfqwblh71rihvv8cli6zs"
(build-system cargo-build-system)
(arguments
  `(#:cargo-inputs
(("rust-bitflags" ,rust-bitflags-1)
 ("rust-cc" ,rust-cc-1)
 ("rust-cfg-if" ,rust-cfg-if-1)
 ("rust-libc" ,rust-libc-0.2))
#:cargo-development-inputs
(("rust-bytes" ,rust-bytes-0.4)
 ;; TODO update to 0.5
 ("rust-caps" ,rust-caps-0.3)
 ("rust-lazy-static" ,rust-lazy-static-1)
 ("rust-rand" ,rust-rand-0.6)
 ("rust-semver" ,rust-semver-0.9)
 ("rust-sysctl" ,rust-sysctl-0.1)
 ("rust-tempfile" ,rust-tempfile-3
(home-page "https://github.com/nix-rust/nix;)
(synopsis "Rust friendly bindings to *nix APIs")
(description
  "Rust friendly bindings to *nix APIs")
(license license:expat)))

(define-public rust-byte-string-1
  (package
(name "rust-byte-string")
(version "1.0.0")
(source
  (origin
(method url-fetch)
(uri (crate-uri "byte-string" version))
(file-name
  (string-append name "-" version ".tar.gz"))
(sha256
  (base32
"1vdll2i5gdx98d3wqbrq8c2l61jqqhzlri7d3hskm35a0mxdxahi"
(build-system cargo-build-system)
(home-page
  "https://github.com/FraGag/byte_string;)
(synopsis
  "Wrapper types for outputting byte strings (b\"Hello\") using the Debug ({:?}) format.")
(description
  "Wrapper types for outputting byte strings (b\"Hello\") using the Debug ({:?}) format.")
(license (list license:expat license:asl2.0

(define-public rust-bloomfilter-1
  (package
(name "rust-bloomfilter")
(version "1.0.5")
(source
  (origin
(method url-fetch)

Re: [PATCHES] ImageMagick security updates without grafting

[Maxime Devos]

On Sat, 2021-03-27 at 20:01 -0400, Mark H Weaver wrote:
> [...]
> Maxime wrote:
> > What does ‘guix refresh --list-dependent imagemagick@6.9.11-48’
> > output now?

> When I last checked, it reported on the order of 2400 dependent package
> rebuilds.

I should have written imagemagick@6.9.12-4 here.

> > If it there are many dependent packages, could some
> > of them use imagemagick/stable, dblatex/stable or gtk-doc/stable
> > as well?
> 
> Yes, that's exactly the purpose of this patch set.  Although at present,
> the only user of 'imagemagick/stable' is 'dblatex/stable', and the only
> user of 'dblatex/stable' is 'gtk-doc/stable'.

You missed a few packages:
in gnu/packages/mate.scm: search for "gtk-doc".
Also, the (gnu packages imagemagick) import seems
unused.

Looking at the package graph, many packages depend on imagemagick
through python-sphinx, so it may be worthwile to define a
python-sphinx/stable and use it instead of python-sphinx in the
native-inputs.

I suggest
 guix graph --type=reverse-package imagemagick@6.9.12-4 | dot -Tpdf > a.pdf

to find out if there are more uses for imagemagick/stable.

Greetings,
Maxime.


signature.asc
Description: This is a digitally signed message part

Re: Guix Packaging Meetup Sunday, March 28, 2021 from 2:00 PM EST (7:00 PM UTC)

[jgart]

Hi Guixers!

I wanted to remind you that LibreMiami is hosting another Guix Packaging Meetup 
this Sunday.

Here is the event link: 
https://keskonfai.fr/events/ac68371f-fdbe-4171-99b2-1788a29e272c

Everyone is invited to participate. We'll be working from a kanban board:

https://board.disroot.org/project/guix-packaging/kanban

I already added a few kanban entries for programs we can package during the 
meetup. Feel free to
suggest packages you'd like to work on as well. 

We had the first guix packaging meetup last month and it was a great turn out. 
Some regular guix
contributors came. We packaged go-github-com-operatorfoundation-ed25519 and 
contributed it back to upstream (see
a2ece4d).

Hope to see you there!

best regards,

jgart

libremiami.org
mumble.libremiami.org
search.libremiami.org