Re: hint: Run `guix search ... | less' to view all the results

[Bengt Richter]

Hi zimoun, Jan,

On +2020-04-26 11:38:01 +0200, zimoun wrote:
> Dear,
> 
> On Sun, 26 Apr 2020 at 10:35, Jan Synacek  wrote:
> 
> > Seriously? Are you seriously forcing your users to either run emacs (or at 
> > least
> > to set the env variable) or use pipes to get the entire search result?
> 
> It is "known" that Guix should respect the PAGER variable [1,2] and it
> is already a feature request [3].
> 
> [1] https://lists.gnu.org/archive/html/guix-devel/2020-02/msg00039.html
> [2] https://lists.gnu.org/archive/html/help-guix/2020-02/msg00150.html
> [3] https://lists.gnu.org/archive/html/help-guix/2020-02/msg00154.html
> 
> 
> > That's just... backwards. Also, it feels like as if the author of that code 
> > sort
> > of assumed that whoever runs the command is stupid enough not to be able to 
> > deal
> > with long output. I'm sure that it wasn't meant like that.
> 
> The manual recommands to use "guix search" in combination with
> 'recsels' (see [4] '--search' paragraph).
> 
> Therefore, the current philosophy of searching is:
> 
>   1) guix search  | recsel -P name,synopsis | grep 
>   2) guix show 
> 
> I agree we could discuss that... as it was started for example see
> this thread [5].
> 
> [4] https://guix.gnu.org/manual/devel/en/guix.html#Invoking-guix-package
> [5] https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00141.html
> 
> 
> > Pretty please, fix this. Don't force your users into usage patterns that 
> > might
> > be completely foreign to them. Don't truncate output from programs by 
> > default.
>

I had been assuming it had just been allowed to scroll off
screen due to unimpeded output, as IIUC Jan wants. Is it
actually truncated?

I believe in KISS for primitives, and I think the less they
look at how they are being used the better for their design.
Otherwise the implementation is implicitly getting ad hoc
inputs from the environment, and incrementally it will grow
messy.

At the higher level, I think systems can be too eager to
help, which can be really annoying to an advanced user, but
really helpful to a noob.

So maybe Jan could be satisfied by a preference setting
(USER_LEVEL expert) in that regard. Many apps have such
features, E.g. emacs will want to take you into a tutorial
until you tell it to stop pestering you.

I think preference similarities will be noticed, and
gradually factored out of apps and put into system and user
preference configs, like themes, for better and worse. PAGER
is probably an example of that.

But I'm wondering whether environment variables per se
really ever should be used for global preference info. ISTM
it becomes an overused "top-level" (define ...) name space.
Hygiene becomes a problem. NOT_SURE_PREFIXES_SCALE_WELL_ ... :)

> Thank you for sharing your opinion.
> 
> After a discussion [6] on other opinions than yours, another strategy
> is implemented [7] and yes the default behaviour could be discussed
> (see bottom [8] and elswhere I am too lazy to find them :-)).
> 
> [6] https://issues.guix.gnu.org/issue/35551#11
> [7] https://issues.guix.gnu.org/issue/36390
> [8] https://lists.gnu.org/archive/html/guix-devel/2020-02/msg00378.html
> 
> 
> > Here's my suggestion: Don't make idiosyncratic decisions like the one I
> > described above. I don't think that your target audience is people 
> > completely
> > new to linux-like systems. I'm definitely not new. And I'm used to tools 
> > that
> > just simply (remember the KISS principle?) give me output when I run them.
> 
> Your suggestion is to implement "Guix honours $PAGER", right?
> And to test on another environment variable name than $INSIDE_EMACS, right?
>

I do think the name smells fishy, unless it's emacs
privately avoiding a recursion problem. IMO if an app needs
to know *who* is calling it, to adjust its behaviour, that's
a design entanglement. If the app can behave in alternate
ways, that can be part of its API, which emacs or any other
caller can make use of.

OTOH if a shape-shifting app wants to change personalities
for different known callers, why not ask the system (ps) who
is calling, and not mess with the caller at all?

I'm not sure who is setting $INSIDE_EMACS, but right now I
am in some sense "inside emacs" (and a lot more) and I can
ask ps[1] to find out[2], so why is $INSIDE_EMACS being used
to pass that info bit? Or is there more to it?
I may be totally misunderstanding the issue, but I think
the general points are valid.

[1]
--8<---cut here---start->8---
#!/usr/bin/bash
# ~/bin/pidparents

pid=${1:-$$}#this process if no pid specified as $1

while [ $(($pid)) -gt 0 ]; do
  ps h -p $pid -o comm,tt,pid,stat,args
  pid=$(ps -q $pid -o ppid=)
done
--8<---cut here---end--->8---

[2]
--8<---cut here---start->8---
pidparents  ? 6727 Ss   /usr/bin/bash /home/bokr/bin/pidparents
emacs   pts/0 6556 Sl+  emacs 

unexpected reproducibility of reproducible blog post?

[Leo Prikler]

Hi simon,

I've executed your commands, et voilà
yours: /gnu/store/klisfr3a4wxb9dc5sgibb45kky72kg65-docker-pack.tar
mine:  /gnu/store/klisfr3a4wxb9dc5sgibb45kky72kg65-docker-pack.tar
Unsurprisingly, this did not change when adding channels -- though, if
you were to add your personal channel with some override for gcc-
toolchain, things might be different.

I don't know, what configuration exactly went into the blog post, but I
assume, it is not the same as for the time-machine experiments before. 
Since the prefix `guix time-machine --channels=guix-version-for-
reproduction.txt --` appears to be missing from the command, that hash
is therefore probably not indicative of anything.

I think the larger problem here is that, while Guix itself is
reproducible, Guix + org-mode (specifically the latter) is not. 
Particularly, looking at the source[1,2], it appears as if all code
blocks were evaluated once, but evaluating them again in a new
environment would bring different results.  In other words, you'd have
to use `guix time-machine` inside `guix time-machine` to get a truly
reproducibly org-mode file, or else come up with a smart way of
dynamically updating the hash in the source blocks themselves.

All the best,
Leo

[1] https://lists.gnu.org/archive/html/guix-devel/2020-01/msg00106.html
[2] 
https://github.com/khinsen/reproducibility-with-guix/blob/master/reproducibility-with-guix.org

Guix System video review on YouTube

[Jonathan Brielmaier]

Hi fellow guix hackers,

Guix System got a video review on YouTube:
https://www.youtube.com/watch?v=IKsXecNJ_nE

As it's in German and was quite a little bit negative (title: "Guix
System distribution - a disaster"), I watched it and noted every issue
he stumbled across.

Although the reviewer was not really happy with the Guix System
distribution, he was quite pleased with the package manager. So he would
recommend his viewers to try Guix on a foreign distro, but not our distro.

For the issues I could reproduce, I already filed bugs:
---
XFCE: web browser -> choose preferred application, no browser preinstalled
  -> http://issues.guix.gnu.org/issue/40880
website: http://guix.gnu.org/screenshots/virtual-machine/ screenshot
shows os-config.scm people search for it, but can't find it, because its
named configuration.scm not os-config.scm
  -> http://issues.guix.gnu.org/issue/40882
thunar: Browse Network -> failed to open
  -> http://issues.guix.gnu.org/issue/40884
thunar: sftp does not work, as well as in pcmanfm-qt
  -> http://issues.guix.gnu.org/issue/40885

Some bugs are only reproducible when you run the Guix QEMU image
(1.1.0). When you install XFCE via the installer, they don't appear.
That's strange.
---
thunar: Icons ^ and down/home are missing
XFCE: no network-manager installed by default (seems not so important on
a QEMU image)

He mentioned lots of other stuff, which feels more like
enhancements/changes and not like bugs. So here is an (incomplete) list
of his findings:
---
* There is no /etc/os-release file. I think it was proposed a while ago,
but the patch was rejected.
* When starting Icecat for the first time, it show 6 top sites,
including trisquel.info and gnu.org, but not guix.gnu.org :)
* While installing packages via `guix install` you can't scroll in the
terminal, you always get reset to the bottom.
* For users coming from other init systems it's hard to find the herd
services. Maybe aliases for `service` or `systemctl`.
-> network-manager-applet: after installing how to start the network
manager service? As suggested by the UI of nm-applet ("NetworkManager
not running").
* guix show/search does not show if a package is installed.
* guix package -I does not show the packaged installed via the
config.scm/services, this can be very confusing. How can you show all
installed packages?
* `guix search ... | less can be confusing at the beginning.
* Finding firefox is little difficult as it's named Icecat. Maybe we can
create a dummy package which then links to the icecat package.
* Guix has multiple plasma/KDE applications but not the KDE/plasma
desktop itself. It's not yet packaged.
* Multi user package concept not clear (root as different packages then
normal user).
* XFCE: has no video/music player installed by default.

Good night
Jonathan

Re: package requests

[Catonano]

Il giorno sab 25 apr 2020 alle ore 15:06 Tobias Geerinckx-Rice 
ha scritto:

> Hi Catonano,
>
> Catonano 写道：
> > Say that some people file bugs in the Guix issue tracker
> > requesting the
> > packaging of some software projects
> >
> > Can a list of such requests be seen in any way ?
>
> […]
>
> > Is there any specific tagging scheme to be used for package
> > requests filing ?
>
> Not that I know of.  It's all very ad-hoc and things get lost &
> forgotten.
>
>  is the closest
> thing we have to an ‘official’ list.
>
> While a structured approach would be much better than our wiki
> page, I prefer that wiki page over *our* bug tracker.
>

don't you like Debbugs ?

I can see why

Have you seen this, anyway ?

https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00238.html

Sirgazil proposed to send a patch to allow Debbugs to tag package requests
so that they could be easily searched

I'd be fine with either solution, as long as it's accepted by the wider
community

Re: package requests

[Catonano]

Il giorno sab 25 apr 2020 alle ore 15:52 sirgazil  ha
scritto:

>   On Sat, 25 Apr 2020 11:38:07 + Catonano 
> wrote 
>  > Say that some people file bugs in the Guix issue tracker requesting the
> packaging of some software projects
>  > Can a list of such requests be seen in any way ?
>  > I remember reading a thread about this some time ago but I can't find
> it anymore and I don't remember how it ended, if it ended
>
> Maybe this one?
> https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00231.html


yes, thanks. This is exactly the thread I saw

Re: Prototype tool for building derivations

[Christopher Baines]

zimoun  writes:

> Hi Chris,
>
> On Fri, 17 Apr 2020 at 22:22, Christopher Baines  wrote:
>
>> Just let me know if you have any questions or comments!
>
> From what I understand of both your prototype and build systems, you
> should interesting to read this paper [1]: "Build systems à la carte:
> Theory and practice". It needs some imagination if you are not
> familiar with Haskell notations. And you should interested by section
> 4 about Schedulers and section 5 about Rebuilders; especially Table 2
> p.28, and also subsections 8.2 about Parallelism and 8.4 about Could
> implementation.
>
> [1] https://doi.org/10.1017/S095679682088
>
>
> Thank you for the initiative and sharing your perspectives.

Thanks, yeah, I'll have a read of the scheduling sections as that could
be useful!

Thanks,

Chris


signature.asc
Description: PGP signature

Re: Prototype tool for building derivations

[Christopher Baines]

Ludovic Courtès  writes:

> Hello!
>
> Christopher Baines  skribis:
>
>> In terms of getting to a point where Guix packages build reliably and
>> reproducibly, I think more testing is what's going to help. By taking
>> packages and building them more, on a wide variety of hardware and
>> software configurations, we'll get data on what works, what doesn't, and
>> where improvements and fixes can be made.
>>
>> It's very much a prototype, but I've pushed some code up here [1] now,
>> the README.org file [2] contains usage instructions as well as a
>> description of the architecture.
>>
>> 1: https://git.cbaines.net/guix/build-coordinator/
>> 2: https://git.cbaines.net/guix/build-coordinator/about/
>>
>> So far, I've mostly done the boring stuff, but I'm excited about what
>> this could support.
>
> Neat!  I like the architecture and the fact that it’s easy to track down
> what was built and where.  (guix scripts offload) currently doesn’t save
> that info.

Thanks :)

> A note about the usage as explained in the README: be sure to register
> GC roots for derivations before passing them around processes.  :-)

Yeah, there's a lot to be desired in terms of this kind of polish.

> Having an HTTP interface is really nice (I recently had someone ask
> whether one could coordinate offloading over HTTP rather than SSH, this
> is helpful in some contexts.)
>
> That also makes me wonder whether we could implement some of the store
> RPCs over HTTP (I think Nix does something along these lines for
> substitutes nowadays).  Because the coordinator interface is in fact
> close to a subset of the daemon’s RPC interface.

I have been thinking about accessing the daemon using HTTP, however I'm
still not sure if this would be better as a core feature, or a bridge
that talks to the daemon on one end, handles authentication and talks
HTTP on the other.

...

>> I think it would be good to get point where there are many different
>> individuals and groups providing independent sources of Guix packages,
>> such that users can have a high level of confidence that the substitutes
>> they're getting correspond to the source code. Getting there will be
>> easier if substitute servers are easy to operate, and part of that I
>> think comes down to how easy it is to see what's going on. With the
>> current daemon implementation, I'm not sure how to get much data out
>> (this could be possible, I haven't looked very closely). This approach
>> however where the scheduling is done outside the daemon makes the
>> information more accessible.
>
> That’s a worthy goal!  I’m not sure the coordinator is necessarily
> helping directly there, because it’s another component (or two!) to set
> up, in addition to ‘guix publish’ and something like Cuirass that
> monitors a channel and builds it.

Indeed, there's a real risk in solving problems by building new things.

One thing I've been thinking about with the build-coordinator is that it
would be neat if it was easy to build around and extend. So I've added
some "hooks" which mean you can just add Guile code that runs when
builds succeed/fail or when an input can't be found.

I've been experimenting in the last few days with adding the necessary
options to the build-coordinator so that it can be used to build things
for providing substitutes. This is in the form of some options in the
build-coordinator, a script [3] to query data.guix.gnu.org for
derivations (this has downsides, but it was a simple place to start) and
a build-success hook [4] that performs a similar function as guix
publish, but without a daemon, just moving the nar file in to the right
place, and generating a corresponding narinfo file.

3: 
https://git.cbaines.net/guix/build-coordinator/tree/scripts/guix-build-coordinator-queue-builds-from-guix-data-service.in
4: 
https://git.cbaines.net/guix/build-coordinator/tree/guix-build-coordinator/hooks.scm?id=43312f9d977aac0f35bc5ce9b63e81cd5116d980#n46

Now this approach is far less mature than Cuirass + guix publish, but I
think it has some advantages such as not requiring everything you want
to provide substitutes for to be in a single store. Additionally, while
you don't get the file level deduplication when you have compressed nar
files, I think that this approach will reduce the disk space
requirements compared to Cuirass + guix publish, as I think you'd
probably have the compressed nar plus the item in the store in that
case.

> However I think it could be a way to restructure (guix scripts offload)
> and/or Cuirass: they could talk to the coordinator instead of doing
> their own thing.

That's an interesting idea.

> In fact, I think it would be quite easy to reimplement (guix scripts
> offload) using the coordinator (see ‘process-request’ for the protocol
> with the daemon), and it would be interesting to see how that works.

Yeah, I'll take a look.

>> I think some of the design decisions here are quite short sighted. I
>> think it would be better if some of this 

Re: 14/42: gnu: Add netdde.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> guix-comm...@gnu.org skribis:
>
>> commit 5aedec90651e7db389732554247d25419cfce755
>> Author: Ricardo Wurmus 
>> AuthorDate: Tue Apr 14 12:46:00 2020 +0200
>>
>> gnu: Add netdde.
>> 
>> * gnu/packages/hurd.scm (netdde): New variable.
>
> LGTM!

Pushed to core-updates; rebased and reset wip-hurd-vm!
Greetings,
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: 08/42: gnu: guix: Update to eeb87326d8.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>> gnu: guix: Update to eeb87326d8.
>> 
>> * gnu/packages/package-management.scm (guix): Update to 
>> 1.1.0-1.eeb87326d8.
>
> Note: when merging, make sure it refers to a commit from ‘core-updates’,
> not one from the WIP branch.

Yes!...pushed to core-updates!
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: 07/42: gnu: guix: Cross-build fix: override compressors.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> guix-comm...@gnu.org skribis:
>
>> commit eeb87326d891e0f88f5eebf767d97b45c3e6
>> Author: Jan (janneke) Nieuwenhuizen 
>> AuthorDate: Fri Apr 10 21:24:19 2020 +0200
>>
>> gnu: guix: Cross-build fix: override compressors.
>> 
>> * gnu/packages/package-management.scm (guix)[arguments]: When 
>> cross-compiling,
>> add `fixup-compressors' stage.
>> [inputs]: When cross-compiling, add `xz'.
>
> Nice!
>
>> +  (lambda* (#:key inputs #:allow-other-keys)
>> +;; Use host compressors.
>> +(let ((bzip2 (assoc-ref inputs "bzip2"))
>> +  (gzip (assoc-ref inputs "gzip"))
>> +  (xz (assoc-ref inputs "xz")))
>> +  (substitute* "guix/config.scm"
>> +(("/gnu/store/.*/bzip2")
>> + (string-append bzip2 "/bin/bzip2"))
>> +(("/gnu/store/.*/gzip") gzip
>> + (string-append gzip "/bin/gzip"))
>> +(("/gnu/store/.*/xz")
>> + (string-append xz "/bin/xz")))
>
> Here’s a way to avoid hard-coding the store name:
>
>   (substitute* "guix/config.scm"
> (("\".*/bin/gzip\"")
>  (string-append "\"" gzip "/bin/gzip\""))
> …)

Ah, nice.  Change them to

   (("\"[^\"]*/bin/bzip2")
 (string-append "\"" bzip2 "/bin/bzip2"))

> Otherwise LGTM!

Thanks!

> Perhaps eventually we should add ‘--with-host-gzip’ and similar
> configure options.

Okay, I'll give that a go later.
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: 07/42: gnu: guix: Cross-build fix: override compressors.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

> guix-comm...@gnu.org skribis:
>
>> commit eeb87326d891e0f88f5eebf767d97b45c3e6
>> Author: Jan (janneke) Nieuwenhuizen 
>> AuthorDate: Fri Apr 10 21:24:19 2020 +0200
>>
>> gnu: guix: Cross-build fix: override compressors.
>> 
>> * gnu/packages/package-management.scm (guix)[arguments]: When 
>> cross-compiling,
>> add `fixup-compressors' stage.
>> [inputs]: When cross-compiling, add `xz'.
>
> Nice!
>
>> +  (lambda* (#:key inputs #:allow-other-keys)
>> +;; Use host compressors.
>> +(let ((bzip2 (assoc-ref inputs "bzip2"))
>> +  (gzip (assoc-ref inputs "gzip"))
>> +  (xz (assoc-ref inputs "xz")))
>> +  (substitute* "guix/config.scm"
>> +(("/gnu/store/.*/bzip2")
>> + (string-append bzip2 "/bin/bzip2"))
>> +(("/gnu/store/.*/gzip") gzip
>> + (string-append gzip "/bin/gzip"))
>> +(("/gnu/store/.*/xz")
>> + (string-append xz "/bin/xz")))
>
> Here’s a way to avoid hard-coding the store name:
>
>   (substitute* "guix/config.scm"
> (("\".*/bin/gzip\"")
>  (string-append "\"" gzip "/bin/gzip\""))
> …)

Ah, nice.  Change them to

   (("\"[^\"]*/bin/bzip2")
 (string-append "\"" bzip2 "/bin/bzip2"))

> Otherwise LGTM!

Thanks!

> Perhaps eventually we should add ‘--with-host-gzip’ and similar
> configure options.

Okay, I'll give that a go later.
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: 04/42: gnu: guix: Cross-build workaround: Run native guile for version.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>> gnu: guix: Cross-build workaround: Run native guile for version.
>
> Good catch, LGTM!

Thanks; pushed to core-updates.
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: 04/36: gnu: guix: Do not copy bootstrap-guile when cross-compiling.

[Jan Nieuwenhuizen]

Ludovic Courtès writes:

>> gnu: guix: Do not copy bootstrap-guile when cross-compiling.
>> 
>> * gnu/packages/package-management.scm (guix)[arguments]: When 
>> cross-compiling,
>> skip copy-bootstrap-guile phase; needed for tests only.
...
>> -(add-before 'check 'copy-bootstrap-guile
>> +,@(if (%current-target-system)
>> +  '()
>> +  `((add-before 'check 'copy-bootstrap-guile
>>(lambda* (#:key system inputs #:allow-other-keys)
>
> Perhaps it would be clearer by avoiding the unquote-splicing dance:
>
>   (lambda* (#:key system target inputs #:allow-other-keys)
> (unless target
>   …)
> #t)

Ah, yes.  No need to be careful not to trigger a rebuild.

> Otherwise LGTM!

Thanks, pushed to core-updates.
janneke

-- 
Jan Nieuwenhuizen  | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: Medium-term road map

[Christopher Baines]

zimoun  writes:

> 2. Search on all the packages included in Guix since the Big Bang.
>
> It is difficult to find the Guix commit where one package goes in and
> the commit where it goes out. The Guix Data Service (GDS) helps a lot
> for that! But AFAIK, it is not possible from the CLI and I do not find
> it handy when I need it (just because I need to open a webbrowser,
> etc.).
>
> IMHO, there is 3 questions:
>  - how to build such full historical index locally? Even if it is expensive
>  - how to fetch it from GDS? Or any other substitue?

It's far from "the Big Bang", but the data.guix.gnu.org instance of the
Guix Data Service has data going back to the start of 2019, with some
big gaps that are still being filled in. Once that's happened, it'll be
feasible to start looking at going further back.

On your point about fetching data, the Guix Data Service can return data
in JSON, and it's been mentioned that there could be something like the
(guix ci) module, but to access the Guix Data Service. Once the data is
available, it would at least be feasible to use that for a command line
interface.

Chris


signature.asc
Description: PGP signature

Re: Medium-term road map

[Christopher Baines]

Ludovic Courtès  writes:

> We released 1.1.0, but what’s coming next?  What would you like to see?

Thanks for starting this thread Ludo, I love this kind of thinking :)

> There are many exciting things being developed and great ideas floating
> around.  For myself, I feel like focusing on “consolidating” what we
> have in the coming weeks.  Here are the areas I hope to focus on (and
> embarking as many of you as possible :-)):
>
>   1. Authentication.  I want to finally provide a standardized mechanism
>  to allow channels to be authenticated upon ‘guix pull’.  “make
>  authenticate” was a first milestone, let’s get it done.  See
>  .

Sounds useful!

>   2. Performance.  There are many things we can improve there, first and
>  foremost: the “Computing derivation” part of ‘guix pull’, Guile’s
>  compiler terrible time and space requirements, further optimizing
>  core operations like ‘package-derivation’, as well as low-level
>  stuff as described at .

I remember saying that the Guix Data Service times various things, and
that it could time the "Computing derivation" bit, but it doesn't store
that data currently. While the performance is dependent on what else is
going on at the same time, it might be interesting to store the data to
see if there's a trend going forward.

>  Related to that is the question of substitute availability, another
>  major hindrance to usability.  We should address this both in the
>  build farm (reducing the
>  time-to-push-to-time-of-substitute-availability, tracking random
>  build failures), and on the client side (can we provide ways for
>  users to pull to a commit that won’t require them to build
>  everything from source, while not compromising on their security?).

Substitute availability as well as building things is something I've
been interested in recently. With your guidance, I think Bayfront is
working more reliably now, although like Berlin there is a disk space
issue to be worked on.

The Guix Data Service has some awareness of substitute availablility,
but the data is not as up to date as it could be, and there isn't a
great way of looking at it either. I hope to address that over the next
few weeks.

In terms of building things, I'm working on making it possible to use
the "Guix Build Coordinator" thing I've been building [1] to provide
substitues, and I'll hopefully send an email out about that soon.

1: https://lists.gnu.org/archive/html/guix-devel/2020-04/msg00323.html

>   3. G-exps.  We should really finish the migration to gexps, as in the
>  ‘wip-build-system-gexp’ branch, and adjust our APIs accordingly.
>
>   4. User interface.  Let’s get our act together with ‘guix shell’ and
>  ‘guix run-script’, and let’s address other annoyances that
>  newcomers keep stumbling upon!
>
> Thoughts?

The other things on my mind are:

Patch submission and review, some information in [2]. I at least want to
do some more work on the Guix Data Service to make the comparisons more
useful for reviewing patches, as well as seeing if I can get the Guix
Build Coordinator to build the affected things, and report back. Then
there are all the issues around submitting packages, like how do I know
if someone is working on this already, or how do I know this hasn't been
updated on core-updates, and how can actually submitting patches be made
easier.

2: https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00476.html

More sophisticated policies on accepting substitutes, this means being
able to say things like "I trust substitutes if there signed by all
these keys/entities". While most of the work on reproducible builds has
been on getting things to build reproducibly, Guix could lead the way on
doing the user facing side of this. A majority of the substitutes for
x86_64-linux match between Bayfront and Berlin, so we already have a
minimially viable "rebuilder" setup to actually make use of this.

Thanks,

Chris


signature.asc
Description: PGP signature

Re: rav1e AV1 encoder

[Efraim Flashner]

On Sun, Apr 26, 2020 at 01:48:00PM -0400, Leo Famulari wrote:
> On Sun, Apr 26, 2020 at 08:05:34PM +0300, Efraim Flashner wrote:
> > I squashed all the cargo commits together and pushed them. I didn't push
> > the one with adding rav1e to ffmpeg, I wasn't sure how much that would
> > add to 'guix size'.
> 
> It's okay to delete the wip-rav1e branch now, right?

It should be ok. We have the patches already for the crate importer and
if you don't need the code for adding rav1e to ffmpeg then there's
nothing else we need there.


-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: rav1e AV1 encoder

[Leo Famulari]

On Sun, Apr 26, 2020 at 08:05:34PM +0300, Efraim Flashner wrote:
> I squashed all the cargo commits together and pushed them. I didn't push
> the one with adding rav1e to ffmpeg, I wasn't sure how much that would
> add to 'guix size'.

It's okay to delete the wip-rav1e branch now, right?


signature.asc
Description: PGP signature

Re: rav1e AV1 encoder

[Leo Famulari]

On Sun, Apr 26, 2020 at 08:05:34PM +0300, Efraim Flashner wrote:
> I squashed all the cargo commits together and pushed them. I didn't push
> the one with adding rav1e to ffmpeg, I wasn't sure how much that would
> add to 'guix size'.

Amazing, thank you!!!

I intended the FFmpeg commit as a "demo", since we don't usually package
unreleased versions of packages, especially important things like
FFmpeg. We can add rav1e when FFmpeg officially supports it.

unexpected reproducibility of reproducible blog post?

[zimoun]

Dear,

Based on the nice blog post [1], instead of really travelling I just
travel in time. :-)
If I read correctly and if I did not do any mistake, the final hash is
not the same now than before. It is not what I was expecting.

Expected output (blog post):
/gnu/store/iqn9yyvi8im18g7y9f064lw9s9knxp0w-docker-pack.tar

Returned output:
/gnu/store/klisfr3a4wxb9dc5sgibb45kky72kg65-docker-pack.tar

Has the file 'guix-version-for-reproduction.txt' been tracked?
Is really the commit 769b96b62e8c09b078f73adc09fb860505920f8f used to
produce the Docker image listed in the blog post?


Thank you in advance.

All the best,
simon


--8<---cut here---start->8---
guix describe
Generation 11   Apr 26 2020 19:24:23(current)
  guix ca4b558
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: ca4b55882a0f6b4ba46253485afb82ec000f8fc2
--8<---cut here---end--->8---

--8<---cut here---start->8---
  guix time-machine --commit=769b96b62e8c09b078f73adc09fb860505920f8f \
   --  pack -f docker -C none \
   -S /bin=bin -S /lib=lib -S /share=share -S /etc=etc \
   gcc-toolchain
--8<---cut here---end--->8---


[1] https://guix.gnu.org/blog/2020/reproducible-computations-with-guix/

Re: rav1e AV1 encoder

[Efraim Flashner]

I squashed all the cargo commits together and pushed them. I didn't push
the one with adding rav1e to ffmpeg, I wasn't sure how much that would
add to 'guix size'.


-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: 14/42: gnu: Add netdde.

[Ludovic Courtès]

guix-comm...@gnu.org skribis:

> commit 5aedec90651e7db389732554247d25419cfce755
> Author: Ricardo Wurmus 
> AuthorDate: Tue Apr 14 12:46:00 2020 +0200
>
> gnu: Add netdde.
> 
> * gnu/packages/hurd.scm (netdde): New variable.

LGTM!

Re: 07/42: gnu: guix: Cross-build fix: override compressors.

[Ludovic Courtès]

guix-comm...@gnu.org skribis:

> commit eeb87326d891e0f88f5eebf767d97b45c3e6
> Author: Jan (janneke) Nieuwenhuizen 
> AuthorDate: Fri Apr 10 21:24:19 2020 +0200
>
> gnu: guix: Cross-build fix: override compressors.
> 
> * gnu/packages/package-management.scm (guix)[arguments]: When 
> cross-compiling,
> add `fixup-compressors' stage.
> [inputs]: When cross-compiling, add `xz'.

Nice!

> +  (lambda* (#:key inputs #:allow-other-keys)
> +;; Use host compressors.
> +(let ((bzip2 (assoc-ref inputs "bzip2"))
> +  (gzip (assoc-ref inputs "gzip"))
> +  (xz (assoc-ref inputs "xz")))
> +  (substitute* "guix/config.scm"
> +(("/gnu/store/.*/bzip2")
> + (string-append bzip2 "/bin/bzip2"))
> +(("/gnu/store/.*/gzip") gzip
> + (string-append gzip "/bin/gzip"))
> +(("/gnu/store/.*/xz")
> + (string-append xz "/bin/xz")))

Here’s a way to avoid hard-coding the store name:

  (substitute* "guix/config.scm"
(("\".*/bin/gzip\"")
 (string-append "\"" gzip "/bin/gzip\""))
…)

Otherwise LGTM!

Perhaps eventually we should add ‘--with-host-gzip’ and similar
configure options.

Ludo’.

Re: 04/42: gnu: guix: Cross-build workaround: Run native guile for version.

[Ludovic Courtès]

guix-comm...@gnu.org skribis:

> commit 4c7c89f97a0d49bc678251bf3ab452dcab85522a
> Author: Jan (janneke) Nieuwenhuizen 
> AuthorDate: Fri Apr 24 11:21:48 2020 +0200
>
> gnu: guix: Cross-build workaround: Run native guile for version.
> 
> We need to take some care here, the native guile is not necessarily 
> always the
> same version as the host guile.
> 
> * gnu/packages/package-management.scm (guix)[arguments]: Do not attempt 
> to run
> host guile for getting effective-version; resort to native guile.

Good catch, LGTM!

Ludo’.

Re: 08/42: gnu: guix: Update to eeb87326d8.

[Ludovic Courtès]

guix-comm...@gnu.org skribis:

> commit ab24528e322e0b55fdaadbd90f435371a8553f15
> Author: Jan (janneke) Nieuwenhuizen 
> AuthorDate: Fri Apr 10 14:30:35 2020 +0200
>
> gnu: guix: Update to eeb87326d8.
> 
> * gnu/packages/package-management.scm (guix): Update to 
> 1.1.0-1.eeb87326d8.

Note: when merging, make sure it refers to a commit from ‘core-updates’,
not one from the WIP branch.

Ludo’.

Re: Medium-term road map

[zimoun]

Hi Ludo,

On Sat, 25 Apr 2020 at 15:38, Ludovic Courtès  wrote:

>   4. User interface.  Let’s get our act together with ‘guix shell’ and
>  ‘guix run-script’, and let’s address other annoyances that
>  newcomers keep stumbling upon!

Adding some items to the whishlist. :-)

1. Access to the internal checkout without knowing the very internals.

When using "guix time-machine", I am regularly navigating in the Git
history. Therefore, I have to clone , going from the
 to , run Git commands, going back, etc. Well, it
is not handy since the checkout already exists (in
'~/.cache/guix//').

Something like "guix git  " should be useful. Even
if I am not clear on how to do with several channels. Maybe:

guix git  --  
guix git -C chan.scm -- --no-pager log --oneline


Considering this new subcommand, it should become easy to add Git tags
and so ease the navigation (for reference [1]).

[1] https://lists.gnu.org/archive/html/guix-devel/2019-11/msg00513.html


2. Search on all the packages included in Guix since the Big Bang.

It is difficult to find the Guix commit where one package goes in and
the commit where it goes out. The Guix Data Service (GDS) helps a lot
for that! But AFAIK, it is not possible from the CLI and I do not find
it handy when I need it (just because I need to open a webbrowser,
etc.).

IMHO, there is 3 questions:
 - how to build such full historical index locally? Even if it is expensive
 - how to fetch it from GDS? Or any other substitue?
 - how to append new packages locally? For example with packages from channels.

The bonus question is feeding SWH to be "guix time-machine" bullet-proof. :-)


Cheers,
simon

Re: 04/36: gnu: guix: Do not copy bootstrap-guile when cross-compiling.

[Ludovic Courtès]

Hi!

guix-comm...@gnu.org skribis:

> commit f1bf6c5d7dfae3c1fc28e417c2f0940b0b010087
> Author: Jan (janneke) Nieuwenhuizen 
> AuthorDate: Fri Apr 24 08:54:43 2020 +0200
>
> gnu: guix: Do not copy bootstrap-guile when cross-compiling.
> 
> * gnu/packages/package-management.scm (guix)[arguments]: When 
> cross-compiling,
> skip copy-bootstrap-guile phase; needed for tests only.

[...]

> -(add-before 'check 'copy-bootstrap-guile
> +,@(if (%current-target-system)
> +  '()
> +  `((add-before 'check 'copy-bootstrap-guile
>(lambda* (#:key system inputs #:allow-other-keys)

Perhaps it would be clearer by avoiding the unquote-splicing dance:

  (lambda* (#:key system target inputs #:allow-other-keys)
(unless target
  …)
#t)

Otherwise LGTM!

Ludo’.

Re: Changing rust variable names to follow cargo guide?

[Efraim Flashner]

On Sat, Apr 25, 2020 at 11:17:25AM +0200, Hartmut Goebel wrote:
> Hi,
> 
> in Cargo (to my experience) most versions are defined as "caret
> versions, specifying the minimum version. According to the Cargo Book
> these version dependence [1] are defined like this (my words):
> 
> Newer version of a package can be used, as long as the left-most
> non-zero digit in the version keeps the same. This means: If a package
> requires ^1.3, it might also use 1.4, 1.5, etc. Whereas a package
> requiring ^0.5.7 must only use 0.5.8, etc. but not 0.6.x
> 
> This it should be save to update e.g. unicode-segmentation 1.3 to 1.6 -
> as long as no package requires an exact version.
> 
> I propose the following:
> 
>  1.
> 
> Variables names for rust packages shall only define the left-most
> non-zero digit (and any leading zeros, of course).
> 
> Examples: rust-url-1, rust-url-2, rust-bytes-0.4, rust-bytes-0.5
> 
>  2.
> 
> Packages using caret version requirements shall refer to the
> respective variable.
> 
> Example: requirements "url ^1.5" -> rust-url-1; "bytes ^0.4.8" ->
> rust-bytes-0.4
> 
>  3.
> 
> If a package requires e specific version, the variable shall contain
> this specific version and a "alias define" shall refer to this
> package if it is the highes "caret" version in guix.
> 
> Example 1: Some package requires "bakery =1.2", with "bakery" not
> being in guix yet. Then the new package "bakery@1.2.3" will be
> defined as "rust-bakery-1.2" and additionally "(define-public
> rust-bakery-1 rust-bakery-1.2")
> Example 2: Later, if some package requires  "bakery ^1.3",
> "rust-bakery-1" will (directly) define "bakery@1.3.0".
> 
> Above rule shall go into the packaging guide in the manual. Variables
> shall be renamed accordingly.
> 
> Rational:
> 
>  1. Using only the left-most non-zero digit in the variable name follows
> the version scheme intended by Cargo.
>  2. Referring to variable names build like this eases updating, since
> only the package itself needs to be updated, not the all occurrences
> of the variable name.
>  3. Distinguishing in dependencies between variable names build like
> this and more specific ones makes it easy to spot whether some
> package requires a *specific* version of another package, or whether
> it is (expected to be) save to upgrade the other package.
> 
> WDTY?
> 
> [1]
> https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#caret-requirements
> 

I like this idea. It makes updating rust crates easier and removes a
bunch of extra ones that aren't actually needed.

-- 
Efraim Flashner  אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted


signature.asc
Description: PGP signature

Re: hint: Run `guix search ... | less' to view all the results

[zimoun]

Dear,

On Sun, 26 Apr 2020 at 10:35, Jan Synacek  wrote:

> Seriously? Are you seriously forcing your users to either run emacs (or at 
> least
> to set the env variable) or use pipes to get the entire search result?

It is "known" that Guix should respect the PAGER variable [1,2] and it
is already a feature request [3].

[1] https://lists.gnu.org/archive/html/guix-devel/2020-02/msg00039.html
[2] https://lists.gnu.org/archive/html/help-guix/2020-02/msg00150.html
[3] https://lists.gnu.org/archive/html/help-guix/2020-02/msg00154.html


> That's just... backwards. Also, it feels like as if the author of that code 
> sort
> of assumed that whoever runs the command is stupid enough not to be able to 
> deal
> with long output. I'm sure that it wasn't meant like that.

The manual recommands to use "guix search" in combination with
'recsels' (see [4] '--search' paragraph).

Therefore, the current philosophy of searching is:

  1) guix search  | recsel -P name,synopsis | grep 
  2) guix show 

I agree we could discuss that... as it was started for example see
this thread [5].

[4] https://guix.gnu.org/manual/devel/en/guix.html#Invoking-guix-package
[5] https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00141.html


> Pretty please, fix this. Don't force your users into usage patterns that might
> be completely foreign to them. Don't truncate output from programs by default.

Thank you for sharing your opinion.

After a discussion [6] on other opinions than yours, another strategy
is implemented [7] and yes the default behaviour could be discussed
(see bottom [8] and elswhere I am too lazy to find them :-)).

[6] https://issues.guix.gnu.org/issue/35551#11
[7] https://issues.guix.gnu.org/issue/36390
[8] https://lists.gnu.org/archive/html/guix-devel/2020-02/msg00378.html


> Here's my suggestion: Don't make idiosyncratic decisions like the one I
> described above. I don't think that your target audience is people completely
> new to linux-like systems. I'm definitely not new. And I'm used to tools that
> just simply (remember the KISS principle?) give me output when I run them.

Your suggestion is to implement "Guix honours $PAGER", right?
And to test on another environment variable name than $INSIDE_EMACS, right?


Thank you for your feedback.

All the best,
simon

hint: Run `guix search ... | less' to view all the results

[Jan Synacek]

Hello,

I'm going to give honest feedback, since I still care about some things.

I decided to check the new guix-1.1.0 and one of the first things that I ran 
was:

$ guix package -s firefox

The resulting message was infuriating to say the least. The code says:

(if (and (not (getenv "INSIDE_EMACS"))
...
  ...
  (display-hint (format #f (G_ "Run @code{~a ... | less} \
to view all the results.")
command)))

Seriously? Are you seriously forcing your users to either run emacs (or at least
to set the env variable) or use pipes to get the entire search result? 
That's just... backwards. Also, it feels like as if the author of that code sort
of assumed that whoever runs the command is stupid enough not to be able to deal
with long output. I'm sure that it wasn't meant like that.

Pretty please, fix this. Don't force your users into usage patterns that might
be completely foreign to them. Don't truncate output from programs by default.

There is a thread on guix-devel [1] titled "Medium-term road map". It contains:

> What would you like to see?

and

> ... Here are the areas I hope to focus on (and
> embarking as many of you as possible ...

and

> ... User interface.  Let’s get our act together with ‘guix shell’ and
> ‘guix run-script’, and let’s address other annoyances that
> newcomers keep stumbling upon! ...

I'm selectively quoting parts that I consider lead to the same direction - more
guix users and possibly developers.

Here's my suggestion: Don't make idiosyncratic decisions like the one I
described above. I don't think that your target audience is people completely
new to linux-like systems. I'm definitely not new. And I'm used to tools that
just simply (remember the KISS principle?) give me output when I run them.

Please, keep making guix (and guile) a great tool.

[1] https://lists.gnu.org/archive/html/guix-devel/2020-04/msg00400.html

Regards,
Jan