Re: 02/02: gnu: libical: Add upstream patch to preserve API compatibility.

[Marius Bakke]

Ludovic Courtès  writes:

> Howdy!
>
> guix-comm...@gnu.org skribis:
>
>> +   (list (origin
>> +   (method url-fetch)
>> +   (uri (string-append
>> + "https://github.com/libical/libical/commit/;
>> + 
>> "ae394010c889e4c185160da5e81527849f9de350.patch"))
>
> IIRC, those generated patches are not stable, like generated tarballs,
> no?  What are the available options?

I haven't seen GitHub change such generated patch files yet[*], but I do
agree it's "fragile", because they can be difficult to reproduce when
GitHub inevitably disappears, or _do_ change the format.

I wonder if it's feasible to use Guile-Git to generate the patches
instead, at the expense of having to download the entire repository
for the purpose of extracting one or more (substitutable) patches.

[*] There was one instance where it had changed, but that was a merge
commit(!), which is a pretty extreme case (whoops):
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=7670efefe4fb4aca12cb19ea5d89ff37c48e3ea6


signature.asc
Description: PGP signature

Re: Wisdom regarding packaging proxysql

[Leo Famulari]

On Wed, Feb 05, 2020 at 04:59:01PM +0100, Ellen Papsch wrote:
> The first is the rather unflexible Makefile based build system. It
> would require some patching on Guix side. For example, the install
> phase installs into a hard coded prefix (/usr). I played with the
> thought of adding a meson build, but it would require forking and I
> don't want to force something on upstream. 

It's not uncommon to see hard-coded installation prefixes. What else
would need to be changed? Is it doable?

> mariadb-connector-c is patched quite extensively, in ways that add
> specific proxysql behavior documented in their wiki and relied upon by
> users. For example, ma_password_c.patch changes the hashing behavior of
> passwords, allowing users to specify passwords in a custom hash
> presentation. I would keep this library bundled, because it constitutes
> a fork, given the modifications.

Agreed, this is a fork.

> The situation with jemalloc is better, only two small patches are
> applied. issue823.patch solves a performance issue observed in a
> benchmark. Authors of jemalloc declined the patch, noting that it
> optimizes something they do not really want to support[0].
> issue2358.patch fixes a bug which is also fixed upstream and slated for
> the next release[1]. A minor proxysql feature is affected[2].
> 
> I'm inclined to use the jemalloc from Guix, although create a
> customized version just for proxysql with the two patches applied.  If
> I don't apply the first one, the main proxysql auhtor will personally
> haunt me (he seems to value performance above all). The second patch
> unbreaks an application feature.

I think that's the right idea: use the upstream patches on our jemalloc
package.

Re: Install script supporting sysV init?

[Jan]

On Wed, 05 Feb 2020 14:44:28 -0500
Joshua Branson  wrote:

> I've got an old ThinkPad T400.  The motherboard supports 8GB of RAM.
> I think I bought it off of ebay for $50-80.  Does your old thinkpad's
> motherboard support more RAM?
You mean a new Thinkpad T400, mine is T60 and supports max 3GB RAM :)
> --
> Joshua Branson
> Sent from Emacs and Gnus


Jan Wielkiewicz

Re: Install script supporting sysV init?

[Joshua Branson]

I've got an old ThinkPad T400.  The motherboard supports 8GB of RAM.  I
think I bought it off of ebay for $50-80.  Does your old thinkpad's
motherboard support more RAM?

--
Joshua Branson
Sent from Emacs and Gnus

Wisdom regarding packaging proxysql

[Ellen Papsch]

Hello Guix,

I'm currently packaging proxysql, of course with the goal of getting it
into Guix upstream as well. There are two hurdles, which I thought I
should discuss here.

The first is the rather unflexible Makefile based build system. It
would require some patching on Guix side. For example, the install
phase installs into a hard coded prefix (/usr). I played with the
thought of adding a meson build, but it would require forking and I
don't want to force something on upstream. 

The second hurdle is the extensive bundling of dependencies, all in all
16 libraries. I looked closely which libraries are modified in any way.
Luckily, 14 libraries are not modified and can be added directly as
inputs, of which 3 will need to be packaged (clickhouse-cpp,
libhttpserver, libinjection). Remaining ones are jemalloc and mariadb-
connector-c.

mariadb-connector-c is patched quite extensively, in ways that add
specific proxysql behavior documented in their wiki and relied upon by
users. For example, ma_password_c.patch changes the hashing behavior of
passwords, allowing users to specify passwords in a custom hash
presentation. I would keep this library bundled, because it constitutes
a fork, given the modifications.

The situation with jemalloc is better, only two small patches are
applied. issue823.patch solves a performance issue observed in a
benchmark. Authors of jemalloc declined the patch, noting that it
optimizes something they do not really want to support[0].
issue2358.patch fixes a bug which is also fixed upstream and slated for
the next release[1]. A minor proxysql feature is affected[2].

I'm inclined to use the jemalloc from Guix, although create a
customized version just for proxysql with the two patches applied.  If
I don't apply the first one, the main proxysql auhtor will personally
haunt me (he seems to value performance above all). The second patch
unbreaks an application feature.

Please let me in on your wisdom ;-)

Best regards
Ellen

[0] https://github.com/jemalloc/jemalloc/pull/523
[1] https://github.com/jemalloc/jemalloc/issues/1605
[2] https://github.com/sysown/proxysql/issues/2358

Re: [PATCH] utils: Handle errors in worker threads.

[Christopher Baines]

Ludovic Courtès  writes:

> Hi,
>
> Christopher Baines  skribis:
>
>> Previously, if an error occurred, the worker fiber simply never sends a
>> reply. In the case of HTTP requests to Cuirass, where an exception occurs 
>> when
>> performing a database query, the fiber handling the request blocks as it 
>> never
>> gets a response. I think that this has the potential to cause the process to
>> hit file descriptor limits, as the connections are never responded to.
>>
>> This is fixed by responding with the details of the exception, and then
>> throwing it within the fiber that made the call.
>>
>> * src/cuirass/utils.scm (make-worker-thread-channel): Catch exceptions when
>> calling proc.
>> (call-with-worker-thread): Handle receiving exceptions from the worker 
>> thread.
>
> Good catch!
>
>> + (put-message reply
>> +  (catch
>> +#t
>
> Please put #t on the same line as ‘catch’.  :-)

Ok, I've made this change and pushed as
bb225189fd56d89ec8be926dda269295ccbfe918.

>> +(lambda ()
>> +  (apply proc args))
>> +(lambda (key . args)
>> +  (cons* 'worker-thread-error key 
>> args))
>
> As discussed with others at the Guix Days, it’s probably a good idea to
> distinguish “remote” exceptions from local exceptions like you did (and
> unlike what ‘inferior-eval’ does).
>
> LGTM!

I don't quite follow what you're saying here, so feel free to elaborate,
but it also didn't sound like there was a problem to fix, so I've gone
ahead and pushed.

Thanks for taking a look,

Chris


signature.asc
Description: PGP signature

Re: Another update on the Guix Data Service

[Christopher Baines]

Ludovic Courtès  writes:

> Christopher Baines  skribis:
>
>> There's now a page to compare two derivations [4], it's not a
>> particularly clear comparison yet, but can hopefully be improved in the
>> future. This is linked to from the derivation history pages [5].
>>
>> 4: 
>> http://data.guix.gnu.org/compare/derivation?base_derivation=/gnu/store/j3n1y6ak2hxs838lm6xx3akljf0xh5qd-guix-1.0.1-10.41b4b71.drv_derivation=/gnu/store/x6zncwgmmhf1lq7p805ixhaqfhwdxybk-guix-1.0.1-11.f38eabe.drv
>> 5: 
>> http://data.guix.gnu.org/repository/1/branch/master/package/guix/derivation-history
>
> I didn’t know about this and it’s really cool!  It’s often necessary to
> compare derivations when you want to understand what’s going on.
>
> Do you think there’s derivation comparison code out there that could
> make it into (guix derivations) or similar, with an eye on perhaps
> having a CLI derivation comparison tool using the same code in the
> future?

I'm not sure any of the Guix Data Service code is worth moving across,
but it might be worth trying to make something better that uses the
records in Guix, and then using that in the Guix Data Service.

>> Provide narinfo and nar files for derivations, and the required source
>> files ([12] for example). This means the Guix Data Service can be used
>> as a substitute server for derivations, for example [13].
>>
>> 12: http://data.guix.gnu.org/l9kznkzkwj1hilbx39nrnlkpl1s6rxz1.narinfo
>> 13: guix build --substitute-urls="http://data.guix.gnu.org 
>> https://ci.guix.gnu.org; 
>> /gnu/store/l9kznkzkwj1hilbx39nrnlkpl1s6rxz1-hello-2.10.drv
>
> Well done.  :-)
>
> I’m interested in making sure we can share the relevant code between
> (guix scripts publish) and the Data Service if that’s not already the
> case.

I don't think I had any big problems duplicating code in (guix scripts
publish), however, it might be worth looking at perhaps trying to
identify similarities if there are cases where duplication can be
avoided.

>> In particular, I'm still thinking about;
>>
>>  - Automated code review for Guix patches, which I was working on when I
>>started working on the Guix Data Service, and most of the
>>functionality can be beneficial there.
>
> Yeah, I think at the Guix Days you showed how to take advantage of the
> Data Service while reviewing patches; perhaps we should write down a
> tutorial or something.

Yep, I'm definately hoping it's getting close to the point where it can
be useful.

>>  - The Guix Weekly News project I was working on recently uses data from
>>the Guix Data Service, and that's what motivated storing the channel
>>news entries.
>
> Yup, that looked nice!  What’s missing, actually?

I haven't sorted anything out to automatically update the data, and
"publish" the news articles.

I also need to improve the large lists of packages, as that isn't very
useful.


signature.asc
Description: PGP signature

Re: WIP gnu: poetry: Update to 1.0.3.

[Tanguy Le Carrour]

Hi Gui, hi Gábor!

Le 02/04, Tanguy Le Carrour a écrit :
> Le 02/03, Gábor Boskovits a écrit :
> > Tanguy Le Carrour  ezt írta (időpont: 2020.
> > febr. 3., H, 16:31):
> > > I'm working on updating Poetry to version 1.0.3 and I have a problem
> > > with the version of a dependency.
> > >
> […]
> > > [1]: attempt to patch setup.py
> > > -(arguments `(#:tests? #f)); tests depend on dbus 
> > > service
> > > +(arguments
> > > + `(#:tests? #f ;; Pypi does not have tests.
> > > +   #:phases
> > > +   (modify-phases %standard-phases
> > > + (add-after 'unpack 'change-dependencies
> > > +   (lambda _
> > > + ;; Guix has version 21.0.0 of python-keyring
> > > + (invoke "sed" "-i" "-e"
> > > + "s/keyring>=20.0.1,<21.0.0/keyring>=20.0.1,<22.0.0/"
> > > + "setup.py")
> > > + #t)
> > >
> >
> > What did not work in this case?
> The `setup.py` file was not modified!
> > Also, do you think that you could replace invoke sed with substitute*?
>
> I'll definitively give it a try! Thanks!

This worked perfectly! Thanks!

I've just submitted the first 3 patches (trivial updates) required to update
Poetry, but I'm still struggling with the update of python-cachy. I've a
`fakeredis: unable to find libc or equivalent` error, so for the time
being, I deactivated the tests.

I might have to start a new thread on guix-devel, if I fail to figure out
a way to make it work.

I'll also have to work on my commit message for poetry as quite a lot is
happening:

```
WIP gnu: poetry: Update to 1.0.3.

* gnu/packages/python-xyz.scm (poetry): Update to 1.0.3.
[propagated-inputs] remove python-glob2, add python-clikit,
add python-importlib-metadata, remove python-msgpack, remove
python-pathlib2, add python-keyring.
[home-page] update URL.
[arguments/#:phases/modify-phases] remove patch.
```


Thanks again for your help!

-- 
Tanguy

Re: Guix size reduction work group

[zimoun]

Hi Ludo,

On Wed, 5 Feb 2020 at 16:18, Ludovic Courtès  wrote:

> > - Improve the tooling.  In my experience, guix graph is quickly unusable
> >   with a high number of nodes.  Maybe d3.js could be leveraged to add a
> >   filtering system, or a way to click on nodes to hide them and all
> >   their children.
>
> ‘guix size’ is key here: it’s a profiler, exactly what we need IMO.
> WDYT?

It is hard to examine the graph with "guix size".
Sometimes, I am doing: "guix graph foo | grep"; especially with 'bag'
and friends. (I do not want to speak for Mathieu but he told me that
he is doing the same time to time. :-))

I agree that using "guix size" is already enough to find the culprits.


> > - How do we compare to Nix?
>
> A few years back we were doing better because we used separate outputs
> in key places where Nixpkgs didn’t.  Later on Nixpkgs had a large part
> of its packages split in several outputs (more than we do).  From what I
> heard, it wasn’t as fruitful as they had hoped it would be in terms of
> closure size, but it might still be better than what we have, dunno.

IMHO, there is 2 directions: add more outputs or duplicate package variants.

For example Emacs:

for pkg in emacs-xwidgets emacs emacs-no-x-toolkit emacs-no-x emacs-minimal;
do
  echo $pkg
 guix size $pkg | tail -n1
 echo ""
done

emacs-xwidgets
total: 1222.4 MiB

emacs
total: 1003.7 MiB

emacs-no-x-toolkit
total: 522.6 MiB

emacs-no-x
total: 275.6 MiB

emacs-minimal
total: 204.2 MiB


And from my opinion, not enough packages are variant-ed.


> The thing is, I think it’s something that requires constant care, every
> time we add a package or modify an existing one.  It’s very easy to lose
> benefits that had been previously obtained through hard work!

I have never thought, neither tried but is it possible to find and/or
build all the packages that 'inherit' from a specific one?


Cheers,
simon

Guix Days + FOSDEM → thumbs up everyone!

[Ludovic Courtès]

Hello Guix!

The 4 days in Brussels (Guix Days + FOSDEM) have been intense as always,
and I’m only catching up with email now (as you might have noticed ;-)).

I’d like to thank Pjotr and Manolis for organizing the Guix Days and the
Minimalist Language track again this year, as well as Andreas for
chairing the Guix Days all along!  Thanks also to the 35+ participants
who showed up at the Guix Days and who got involved in all these
sessions.  I believe Andreas has been gathering the notes that everyone
took so hopefully we can share that soon!

It’s also amazing and plain awesome that quite a few people presented
their work on Guix in various devrooms!  I think it’s all been a
productive way to both get more people exposed to Guix and to get useful
feedback.  I guess most videos are on-line now if you want to catch up!
Follow the links at
.

I had a great time meeting everyone and it’s definitely been a great
boost; I feel energized and all, woohoo!  I hope it was just as nice for
y’all!  Thumbs up, Guix!

Cheers,
Ludo’.

Re: use-modules in gexps

[Ricardo Wurmus]

Ludovic Courtès  writes:

>> I wondered if perhaps we could add a G-expression compiler for module
>> imports, so that instead of
>>
>> (with-imported-modules (source-module-closure '((my module)))
>>   #~(begin
>>  (use-modules (my module))
>>  …))
>>
>> we would do
>>
>>   #~(begin
>>  #$(use-modules (my module))
>>  …)
>>
>> which would have the effect of adding (my module) — or the module
>> closure – to the list of modules needed by the current G-expression.
>
> I think the syntax should be different because #$/ungexp has a different
> meaning.  However, that would leave open the question of how to select
> the subset of the module’s closure you want to import, etc.
>
>> Can a G-expression compiler help accumulate state (e.g. by making
>> G-expressions values in the state monad) or can it only be compiled down
>> to a value at the current location?
>
> The latter.  What state would you want to accumulate?

When I wrote “state” I had the list of modules encountered in mind, so
that it would be possible to collect all marked instances of
“use-module” and hoist them to the top, in effect doing automatically
what is achieved manually by using “with-imported-modules” with a list
of modules.

--
Ricardo

Re: Guix size reduction work group

[Ludovic Courtès]

Hi!

Pierre Neidhardt  skribis:

> Shall we start a work group to fix the issue?
>
> - Write a blog article to explain the issue and a detailed process on
>   how to fix it.  (Embed it to the manual.)

The “Submitting Patches” section mentions closure size specifically.  Is
there anything you think we should add there?

> - Improve the tooling.  In my experience, guix graph is quickly unusable
>   with a high number of nodes.  Maybe d3.js could be leveraged to add a
>   filtering system, or a way to click on nodes to hide them and all
>   their children.

‘guix size’ is key here: it’s a profiler, exactly what we need IMO.
WDYT?

> - How do we compare to Nix?

A few years back we were doing better because we used separate outputs
in key places where Nixpkgs didn’t.  Later on Nixpkgs had a large part
of its packages split in several outputs (more than we do).  From what I
heard, it wasn’t as fruitful as they had hoped it would be in terms of
closure size, but it might still be better than what we have, dunno.

The thing is, I think it’s something that requires constant care, every
time we add a package or modify an existing one.  It’s very easy to lose
benefits that had been previously obtained through hard work!

At any rate, I agree we need to improve!

Ludo’.

Re: Guix search, colors and INSIDE_EMACS

[Ludovic Courtès]

Hi!

Pierre Neidhardt  skribis:

> In my experience, colors worked perfectly before the INSIDE_EMACS switch
> was introduced.  I don't understand what this change tried to fix.
> Maybe one fix broke something else.

‘INSIDE_EMACS’ has always been honored by (guix colors).  Also, it’s
honored similar by all GNU tools and in particular Coreutils, which I
think is overall a good thing as Ricardo explained!

Now, it’s true that that there are cases where even shell-mode +
guix-build-log-mode doesn’t give you as much highlighting as ‘guix’
outside Emacs (for example for errors or highlights in all the
diagnostics.)  Perhaps we “just” need to enhance comint somehow!  :-)

Ludo’.

Re: [PATCH] utils: Handle errors in worker threads.

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> Previously, if an error occurred, the worker fiber simply never sends a
> reply. In the case of HTTP requests to Cuirass, where an exception occurs when
> performing a database query, the fiber handling the request blocks as it never
> gets a response. I think that this has the potential to cause the process to
> hit file descriptor limits, as the connections are never responded to.
>
> This is fixed by responding with the details of the exception, and then
> throwing it within the fiber that made the call.
>
> * src/cuirass/utils.scm (make-worker-thread-channel): Catch exceptions when
> calling proc.
> (call-with-worker-thread): Handle receiving exceptions from the worker thread.

Good catch!

> + (put-message reply
> +  (catch
> +#t

Please put #t on the same line as ‘catch’.  :-)

> +(lambda ()
> +  (apply proc args))
> +(lambda (key . args)
> +  (cons* 'worker-thread-error key 
> args))

As discussed with others at the Guix Days, it’s probably a good idea to
distinguish “remote” exceptions from local exceptions like you did (and
unlike what ‘inferior-eval’ does).

LGTM!

Ludo’.

Re: use-modules in gexps

[Ludovic Courtès]

Hello!

Ricardo Wurmus  skribis:

> I just watched the excellent FOSDEM talk about G-expressions by Chris
> Marusich.  One person asked at the end about why it is necessary to use
> “with-imported-modules” when inside of the G-expression “use-modules” is
> used.

Chris and I discussed it (I think the paper mentions it).  There are
several reasons:

  1. Scope and importing are two different things.  The user needs fine
 grain control over what’s imported and what’s not (for example, you
 generally don’t want to import Guile-provided modules; sometimes
 you want to important a module’s closure, sometimes not, because
 you know some of the modules are provided by some other means.)

  2. ‘use-modules’ and ‘define-module’ are meant to be top-level forms,
 but a gexp can be non-top-level, and thus they do not have a
 ‘use-modules’ form.

 For example, the ‘start’ and ‘stop’ fields of 
 are non-top-level, which is why there’s an additional ‘modules’
 field there.  All the ‘modules’ field are appended to form a
 ‘use-modules’ form in the resulting shepherd config file.

  3. Guile has several ways to bring modules in scope: ‘use-modules’,
 ‘define-module’, and both of these support #:select, #:prefix,
 etc.  These features should remain available.

So I took the conservative approach of not trying to merge these two
related but different features.  The downside is that it looks
redundant, but the upside is that there’s no loss of expressivity.

> I wondered if perhaps we could add a G-expression compiler for module
> imports, so that instead of
>
> (with-imported-modules (source-module-closure '((my module)))
>   #~(begin
>  (use-modules (my module))
>  …))
>
> we would do
>
>   #~(begin
>  #$(use-modules (my module))
>  …)
>
> which would have the effect of adding (my module) — or the module
> closure – to the list of modules needed by the current G-expression.

I think the syntax should be different because #$/ungexp has a different
meaning.  However, that would leave open the question of how to select
the subset of the module’s closure you want to import, etc.

> Can a G-expression compiler help accumulate state (e.g. by making
> G-expressions values in the state monad) or can it only be compiled down
> to a value at the current location?

The latter.  What state would you want to accumulate?

Really happy to see the positive effects of Chris’ talk!  :-)

Ludo’.

Re: nesting with-imported-modules

[Ludovic Courtès]

Hello!

Ricardo Wurmus  skribis:

> it seems that it is impossible to nest with-imported-modules.  A gexp
> that is wrapped in multiple layers of with-imported-modules won’t depend
> on the list of all mentioned modules but only on the outermost.
>
> This is because with-imported-modules sets the current-imported-modules
> parameter without checking if the parameter already has a value.
>
> Should nesting be supported?  It seems useful.

Is it?  :-)

My impression is that one would always write:

  (with-imported-modules … #~(…))

or possibly combining multiple expressions:

  (define e1 (with-imported-modules … #~(…)))
  (define e2 (with-imported-modules … #~(… #$e1 …)))

in which case modules are appended.

I couldn’t think of a use case where one would literally write:

  (with-imported-modules …
(with-imported-modules …
  #~(…)))

Perhaps I’m missing some pattern where it could be useful though?

Thanks,
Ludo’.

Re: Thoughts on stateful services in Guix

[Ludovic Courtès]

Hi Alex,

Thanks for sharing your thoughts and this tricky but important use case!

Alex Sassmannshausen  skribis:

> 3 A generalisation: the Stateful-Service Service
> 
>
>   State dumping and restoration *should* be generalisable.
>
>   It should normally consist of one or more operations of:
>   • running a special program to dump data, encrypting the dump, and
> storing it, together with the service revision hash, in a known
> location.
>   • tarring up a directory tree, encrypting the archive and storing it,
> together wtih the service revision hash, in a known location.
>
>   These operations should be able to be provided by a daemon, managed by
>   shepherd, which can be configured with a gpg key for encryption, a
>   mechanism for decryption (interactive or programmatic), a location for
>   storing/retrieving dumps (local or remote?), and a DSL for mapping
>   data dump / data restoration program invocations or file-system
>   locations to data dumps it already knows of.

Back in my Nix days, Wouter den Breejen worked on “S-Nix”, whose goal
was to explore how Nix could be extended to support state¹.  Back then,
I think the experiment was not considered fruitful, in part due to its
complexity, the implementation’s reliance on ext3cow, things like that.

There may still be good ideas to take from that, although it’s probably
more reasonable to start with something simple and practical, trying to
address a couple of concrete use cases before trying to generalize.
What you propose above sounds like it could be a good start!

Ludo’.

¹ “Managing state in a purely functional deployment model”, master
  thesis,
  .
  I couldn’t find a copy on-line so here’s one from my attic:
  .

Re: Thoughts on stateful services in Guix

[Giovanni Biscuolo]

Hi Alex,

I'm very interested in deploying "as stateless as possible" services
with Guix, thank you for sharing your thougts on this!

I'm still non able to contribute code fot this task... I'm working on it

Alex Sassmannshausen  writes:

> As a result of FOSDEM conversations today I felt inspired to put some
> thoughts on paper about an area where I think we currently run into
> complications.

I also think that deploying services in a functional way (ala Guix
packages to be clear) is _complicated_ and... it's a problem

to be clear: this complication it's *absolutely not* Guix specific, but
it's something that becomes very clear when you want to use the
declarative approach to "everithing" once you've got Guix :-O

(and after you were deluded by the promises from tools like Puppet and
alike)

> Not sure if it is appropriate to write this blog-post-ish contribution
> here, but as I don't have a blog, and it's about Guix development, I
> figured it might be OK.

maybe with a little help by someone more qualified than me this should
become a Guix blog post... or better be generalized ("how to write
staletess services") and included in the cookbook

anyway, I think your analisys is very helpful in discussing how Guix can
help fighting the "state dragon"

I've some comment and questions

> Best wishes,
>
> Alex
>
> 1 Introduction
> ══

[...]

>   Disciplined developers can implement many services in such a way that
>   their statefulness is delegated to other dedicated services.  In this
>   way the problem of statefulness can be isolated.  However, many
>   existing useful services /have not/ been implemented in such a way.

do you mean upstream?

[...]

from now on you used Drupal as an example: I do not know how Drupal
works, but I know other problematic web services :-)

> 2.3.1 Enter the state dragon
> 
>
>   But hang-on… if Mysql service is a dependency, and we store state in a
>   Mysql DB,

in this regard some time ago I found
http://gfxmonk.net/2015/01/03/nixos-and-stateless-deployment.html (see
"Lumps of impurity") a good explanation on what should be state and what
not; this is the relevant part:

--8<---cut here---start->8---
NixOS keeps the “OS” parts of the machine stateless, meaning that the
only state you need to manage is that which you create yourself. From
the top of my head (not an exhaustive list):

- /var, /tmp, /run: stateful, unmanaged

- each user’s $HOME directory: stateful, unmanaged (rarely used on a server, 
though)

- Users & Groups: stateless

- Installed software: stateless

- Installed services (using systemd): stateless

- All program configuration (i.e all of /etc): stateless

- Kernel & kernel modules, grub configuration: stateless (but requires a reboot 
to activate)

- Disk mounts: stateless

By “stateless”, I mean that the given item is generated entirely from
the pure nix expression of the system, and isn’t affected by any
previous state.
--8<---cut here---end--->8---

I was not able to find a similar description in some Guix
document... but it's OT now :-)

(...and software config stored in $HOME should be stateless, also;
guix-home-manager [1] is a _very_ interesting work on this topic)

storing state it's obviously the job of many services, what's bad it's
some software (Drupal?  Discourse for sure, Wordpress for sure) store
(part of?) configuration as status; some software provides a CLI tool
(usually as a afterthougt, e.g. wp-cli) to imperatively configure it,
but it's always treated as status

in other words: some (usually web) services "embeds" config with status
and provide one or more interface to configure, others clearly separates
config from status leaving the "burden" of writing configuration
(usually in /etc) to sysadmins... and that software is "easy" to
integrate as a stateless (and functional) Guix service

my question is: can we work around this very bad design choiche?  can we
patch the package and have stateless config for, e.g., Drupal?

I see this someway similar to "de-blobbing" our packages: how hard it's
to do depends on upstream... unfortunately not all developers agrees
that stateful config is a problem (in a similar way not all agrees that
distributing binary-blobs it's a problem)

>   what happens when we upgrade to a newer version of Drupal,
>   with a different schema?  Would this service instantiation change the
>   state of our system?  Could we still roll-back?

oh yes, schema migration of status is also a problem to manage when we
upgrade software... and usually we do it "by hand" (making backups of
status, ecc.)

schema migration is usual when status is stored in a database,
theoretically schema migration could be applyed also when storing state
in XML (YAML?) in files, but I've never seen this use case

your proposal (below) is interesting!

>   And remember that broken symlink we introduced in the package?  That
>   

Re: Store channel specification in profile

[Ludovic Courtès]

Hello!

Pierre Neidhardt  skribis:

> Ideas that are not that good on second thought:
>
> - Create a profile specification file from CLI options,
>   like --export PROFILE or --convert MANIFEST.
>
>   The problem is that those are extra steps that the user would have to run
>   manually.  We can save those extra steps by dumping the specification file
>   automatically into the profile.
>
> The Plan©:
>
> On every profile installation, we generate a "specifications.scm" file 
> alongside
> the internal "manifest".

One thing to keep in mind, though, is that if the ‘specifications.scm’
is part of the profile, it must be future-proof.  That is, the APIs it
uses must essentially be guaranteed to remain forever.  That’s a very
strong constraint.

In contrast, versioned data formats like the famous ‘manifest’ file
don’t have this problem at all, but they’re less directly usable from
the CLI.

> Problems:
>
> - There may be too many provenances, we need to add a CLI flag to
>   ignore provenance.

Like Konrad wrote, just write if there are too many of them.

> Proposed format for "specifications.scm": we can reuse
> `specifications->manifest`.  Each entry is either or string, in which case it
> acts as before, or a list, with the following self-explanatory elements:
>
> (specifications->manifest
>  '(("my-package"
> #:outputs '("out")
> #:version "2.3"
> #:channel (channel
>(name 'guix)
>(branch "master")
>(url "https://git.savannah.gnu.org/git/guix.git;)
>(commit
> "704719edade1368f798c9301f3a8197a0df5c930")))
>("my-package2")
>"old-style-package"))

As a rule of thumb, I think ‘specifications->manifest’ must remain what
it is: it should take a specification as is currently defined and return
a manifest.  I think that if we need a new concept, we should not
overload an existing one.

We also need to distinguish between APIs, which should use first-class
objects (, etc.), and data formats, which are plain sexps.
(The above example is a mixture of both and in fact looks very similar
to what’s already in the ‘manifest’ file.)  But then again, that means
relying on a larger chunk of the API.

So, all in all, I think I’d rather see it implemented as ‘guix package
--export’ or similar.  The generated Scheme file could use all the parts
of the current API (and we could adjust the generator as we change the
API).  So it could generate something similar to the example in the
manual (info "(guix) Inferiors"):

--8<---cut here---start->8---
(use-modules (guix inferior) (guix channels)
 (srfi srfi-1))   ;for 'first'

(define channels
  ;; This is the old revision from which we want to
  ;; extract guile-json.
  (list (channel
 (name 'guix)
 (url "https://git.savannah.gnu.org/git/guix.git;)
 (commit
  "65956ad3526ba09e1f7a40722c96c6ef7c0936fe"

(define inferior
  ;; An inferior representing the above revision.
  (inferior-for-channels channels))

;; Now create a manifest with the current "guile" package
;; and the old "guile-json" package.
(packages->manifest
 (list (first (lookup-inferior-packages inferior "guile-json"))
   (specification->package "guile")))
--8<---cut here---end--->8---

There could also be an option to generate a “symbolic” manifest: one
that would install packages of the same name, but not resorting to
inferiors.  The result would not be a faithful translation of what’s
actually in the store, but an approximation thereof that could be useful
for people transitioning.  In fact it would be easier to start by
prototyping this before going further.

As far as faithfulness is concerned, we should also keep in mind that we
don’t always have all the information needed to reconstruct what’s in a
profile.  For example, we lack information about the package
transformation options that were used (if any), and we lack information
about arbitrary user code that might have been used to generate manifest
entries.

Thus, as we design this, I think we must keep in mind that the result is
necessarily an approximation.

> A somewhat unrelated propostion: To avoid further confusion between the 
> internal
> "manifest" and the user-facing "--manifest", we could rename the internal
> manifest to $profile/internal-maifest.

Renaming ‘manifest’ is not really an option because, as we discussed,
it’s a contract over time: today’s Guix can interact with a profile
created 5 years ago just fine.

Thanks for sharing your thoughts!

Ludo’.

Re: Another update on the Guix Data Service

[Ludovic Courtès]

Hi!

I’m late to the party and learned about most of these things in the
meantime at the Guix Days, and this is all very cool!

Christopher Baines  skribis:

> There's now a page to compare two derivations [4], it's not a
> particularly clear comparison yet, but can hopefully be improved in the
> future. This is linked to from the derivation history pages [5].
>
> 4: 
> http://data.guix.gnu.org/compare/derivation?base_derivation=/gnu/store/j3n1y6ak2hxs838lm6xx3akljf0xh5qd-guix-1.0.1-10.41b4b71.drv_derivation=/gnu/store/x6zncwgmmhf1lq7p805ixhaqfhwdxybk-guix-1.0.1-11.f38eabe.drv
> 5: 
> http://data.guix.gnu.org/repository/1/branch/master/package/guix/derivation-history

I didn’t know about this and it’s really cool!  It’s often necessary to
compare derivations when you want to understand what’s going on.

Do you think there’s derivation comparison code out there that could
make it into (guix derivations) or similar, with an eye on perhaps
having a CLI derivation comparison tool using the same code in the
future?

> Provide narinfo and nar files for derivations, and the required source
> files ([12] for example). This means the Guix Data Service can be used
> as a substitute server for derivations, for example [13].
>
> 12: http://data.guix.gnu.org/l9kznkzkwj1hilbx39nrnlkpl1s6rxz1.narinfo
> 13: guix build --substitute-urls="http://data.guix.gnu.org 
> https://ci.guix.gnu.org; 
> /gnu/store/l9kznkzkwj1hilbx39nrnlkpl1s6rxz1-hello-2.10.drv

Well done.  :-)

I’m interested in making sure we can share the relevant code between
(guix scripts publish) and the Data Service if that’s not already the
case.

> In particular, I'm still thinking about;
>
>  - Automated code review for Guix patches, which I was working on when I
>started working on the Guix Data Service, and most of the
>functionality can be beneficial there.

Yeah, I think at the Guix Days you showed how to take advantage of the
Data Service while reviewing patches; perhaps we should write down a
tutorial or something.

>  - The Guix Weekly News project I was working on recently uses data from
>the Guix Data Service, and that's what motivated storing the channel
>news entries.

Yup, that looked nice!  What’s missing, actually?

>  - Trust in builds, once the data about package build reproducibility is
>more complete, hopefully that will be informative and show where
>build reproducibility issues can be fixed.
>
>  - Improving the speed in which substitutes are made available.
>
>  - Explaining the data that underpins Guix to a technical audience, like
>derivations and how they work.

Nice!

Thanks,
Ludo’.

Outreachy timeline and status report

[Gábor Boskovits]

The timeline is available here:
https://www.outreachy.org/apply/project-selection/

Every date refers to 4 p.m. UTC that day.

We kindly request proposals to be available by the 11th of February,
so that it will be possible to promote them on the Outreachy Twitter
chat.

Please remind prospective applicants, that the initial application
deadline is the 25th of February.

The contribution period is between March 5, April 7. Please note that
registering a contribution is a mandatory requirement for an applicant
to be selected as an intern.
Reminder to mentors: the contribution period is usually the most busy
period of the mentoring activity, as handling multiple contribution
request at the same time might be necessary.
Including information in patches that they are related to outreachy
might help reviewers to give these some priority. You can use the
--subject-prefix option of git-format-patch for that.

Please do not notify interns about the selection result before the
official announcement, as this is forbidden by Outreachy policy.
Announcement expected at April 27.

Current status:

There are currently two proposals:

2020 May to 2020 August round - GNU Guix - Integration of desktop
environments into GNU Guix

Mentoring: Danny Milosavljevic

You can co-mentor using:
https://www.outreachy.org/outreachy-may-2020-internship-round/communities/gnu-guix/integration-of-desktop-environments-into-gnu-guix//cfp/mentor/submit/

and

2020 May to 2020 August round - GNU Guix - Create Netlink bindings in Guile

Mentoring: Gábor Boskovits

You can co-mentor using:
https://www.outreachy.org/outreachy-may-2020-internship-round/communities/gnu-guix/create-netlink-bindings-in-guile/cfp/mentor/submit/

You can add new proposals using
https://www.outreachy.org/outreachy-may-2020-internship-round/communities/gnu-guix/submit-project/

Best regards,
g_bor
-- 
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21

Re: 23/36: gnu: Add libkdepim.

[Hartmut Goebel]

Am 04.02.20 um 23:29 schrieb Ludovic Courtès:
> This must get us close to full coverage of the core of KDE, no?

Well, nor t yet. I have yet another approx. 100 packages hanging around.


>> +(synopsis "Libraries for common kdepim apps")
>> +(description "Libraries for common kdepim apps.")
> I know it’s no fun, but could you try to follow the guidelines for
> synopses and descriptions at:

Whet exactly is your point? I assume it is that summary and description
are quite meaningless. I agree on this.

I'm afraid, there is nothing to say about some of these packages. They
are used by KDE PIM only, there is no Readme and no description. Debian
also does not provide more infos.

The only change which makes sense IMHO is to add something like  "This
library is used by KDE PIM" to the description. (Or change the
description to only state this.)

WDYT?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |