Re: A friendlier API for operating-system declarations

[Hartmut Goebel]

Hi,

Am 19.02.24 um 23:25 schrieb antlers:

(define (os-with-yubi parent users*)
   (modify-record parent
 (groups -> (cons (user-group (name "plugdev")) <>))
 (users  -> (map (lambda (user)
   (if (member (user-account-name user)
   users*)
   (modify-record user
 (supplementary-groups -> (cons "plugdev" <>)))
   user))
 <>))
 (services => (append <> (list
   (service pcscd-service-type)
   (simple-service 'u2f-udev-rules udev-service-type
   (list (specification->package "libu2f-host")))
   (simple-service 'yubi-udev-rules udev-service-type
   (list (specification->package 
"yubikey-personalization"


I'd appreciate such a mechanism, as it fits my mental model of how to 
compose a system out of reusable components (much like "roles" in 
Ansible resp. Debop).


--
Regards
Hartmut Goebel

| Hartmut Goebel  | h.goe...@crazy-compilers.com   |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: A friendlier API for operating-system declarations

[Liliana Marie Prikler]

Am Sonntag, dem 25.02.2024 um 13:27 -0800 schrieb antlers:
> Thanks for the detailed response c:
> 
> The blocker is that AFAIK we can't tell if a record-field was defined
> as thunk-ed or delay-ed outside of the expansion of the
> `define-record-type*' form that defines it, nor programatically
> access it's getter/setter (the pair `(guix records)` defines, which
> accounts for eg. thunk-ing, and is distinct from the underlying pair
> returned by eg. `record-accessor').
You might want to have a look at match-record or even define your
modify-record in terms of it.  IIUC it ought to handle exactly the
problem you're talking about while also making it slightly easier for
you to structure your updating code.

Cheers

Re: A friendlier API for operating-system declarations

[antlers]

Thanks for the detailed response c:

The blocker is that AFAIK we can't tell if a record-field was defined as
thunk-ed or delay-ed outside of the expansion of the `define-record-type*'
form that defines it, nor programatically access it's getter/setter (the pair
`(guix records)` defines, which accounts for eg. thunk-ing, and is distinct 
from the
underlying pair returned by eg. `record-accessor').

Re: A friendlier API for operating-system declarations

[Liliana Marie Prikler]

Am Sonntag, dem 25.02.2024 um 10:49 -0800 schrieb antlers:
> 
> 1.) It leaks `(guix records)`'s thunked/delayed field abstraction to
> the end-user, which will confuse and regularly bite absolutely anyone
> who tries to use it.
> 
> 2.) Ideally we'd check that modified fields exist at compile-time.
> This could require users to provide the record-type, which is not
> always public, but could also be optional.
You can get the record type of a record at run-time, so at least you
can synthesize a check that gives you a helpful message if that's what
you're gunning for.

> 3) The `cut`-like transformation is recursive and, while it has
> well-defined and tested behavior, the semantics are non-standard. I
> don't believe that the syntax achieves its goals without this
> mechanism, but there are remaining sub-concerns to address before I'd
> consider it ready to formalize and document.
Perhaps we can make it like modify-services where we take a value
updater instead of evaluating the forms directly?  This would fix both
#3 and #1 (since we'd unwrap the thunked/delayed field silently).
That would also make it so that we can use a single set of arrows (=>)
for both.

> 4) The syntax expands modifications of multiple fields into recursive
> expansions (ie: calls to the record constructor) when it would be
> more efficient (especially for thunked/delayed fields) to accumulate
> transformations by target field and `compose' them over a single call
> to the record constructor. This hasn't been an issue in practice, but
> would be the "right thing to do".
That's a minor detail, but it might be a syntax-rules → syntax-case
change.

> If I do raise the possibility of up-streaming it would be with these
> issues addressed, and would still have substantial cause to warrant
> discretionary discussion then. Really a scratching-my-own-itch
> solution atm.
Of course it'd go through the proper review channels first, but from
personal experience talk over at guix-devel is less likely to cause the
change you want over submitting an actual patch :)

Cheers

Re: A friendlier API for operating-system declarations

[antlers]

> I don't believe that the syntax achieves its goals without this
> mechanism

I just realized my example doesn't actually include a use of recursive
cut after all (whoops), and finding only 2 uses in my code-base, am open to
considering SRFI-26 sufficient. This helps somewhat, but was not the blocking
issue.

Re: A friendlier API for operating-system declarations

[antlers]

> That looks like a nice syntax indeed. Is the code behind it small
> enough to include it in (guix records)?

Small enough? Yes (<100 LOC, inc. a handful of comments and tests).
Suitable? No, I introduced it as "moderately-cursed" for a reason >u<
But I appreciate the sentiment c:



1.) It leaks `(guix records)`'s thunked/delayed field abstraction to
the end-user, which will confuse and regularly bite absolutely anyone
who tries to use it.

2.) Ideally we'd check that modified fields exist at compile-time.
This could require users to provide the record-type, which is not
always public, but could also be optional.

3) The `cut`-like transformation is recursive and, while it has
well-defined and tested behavior, the semantics are non-standard. I
don't believe that the syntax achieves its goals without this
mechanism, but there are remaining sub-concerns to address before I'd
consider it ready to formalize and document.

4) The syntax expands modifications of multiple fields into recursive
expansions (ie: calls to the record constructor) when it would be more
efficient (especially for thunked/delayed fields) to accumulate
transformations by target field and `compose' them over a single call
to the record constructor. This hasn't been an issue in practice, but
would be the "right thing to do".



#3 and #4 are a light refactoring and feedback-round away (#3 could be
 controversial), but #1 and #2 require deeper understanding and
 (especially for #1) modification of `(guix records)' than I'm
 comfortable with at the moment.

If I do raise the possibility of up-streaming it would be with these
issues addressed, and would still have substantial cause to warrant
discretionary discussion then. Really a scratching-my-own-itch solution
atm.

Re: A friendlier API for operating-system declarations

[Liliana Marie Prikler]

Am Montag, dem 19.02.2024 um 14:25 -0800 schrieb antlers:
> ```
> (define (os-with-yubi parent users*)
>   (modify-record parent
>     (groups -> (cons (user-group (name "plugdev")) <>))
>     (users  -> (map (lambda (user)
>   (if (member (user-account-name user)
>   users*)
>   (modify-record user
>     (supplementary-groups -> (cons "plugdev"
> <>)))
>   user))
>     <>))
>     (services => (append <> (list
>   (service pcscd-service-type)
>   (simple-service 'u2f-udev-rules udev-service-type
>   (list (specification->package "libu2f-host")))
>   (simple-service 'yubi-udev-rules udev-service-type
>   (list (specification->package "yubikey-
> personalization"
> ```
That looks like a nice syntax indeed.  Is the code behind it small
enough to include it in (guix records)?  If so, would you care to
submit it via the usual contribution channels?

Cheers

Re: A friendlier API for operating-system declarations

[antlers]

Hi!

Just wanted to say that I really admire your take on end-user service 
configuration in the Beaver Labs channel.

I gravitated towards composing functions over `operating-systems` myself, 
though my config is probably only ""notable"" for the moderately-cursed 
`modify-record` macro that I use to derive/configure/wrap the services[1]:

```
(define (os-with-yubi parent users*)
  (modify-record parent
(groups -> (cons (user-group (name "plugdev")) <>))
(users  -> (map (lambda (user)
  (if (member (user-account-name user)
  users*)
  (modify-record user
(supplementary-groups -> (cons "plugdev" <>)))
  user))
<>))
(services => (append <> (list
  (service pcscd-service-type)
  (simple-service 'u2f-udev-rules udev-service-type
  (list (specification->package "libu2f-host")))
  (simple-service 'yubi-udev-rules udev-service-type
  (list (specification->package 
"yubikey-personalization"
```

It's like if `modify-services` was generalized over any kind of record, but 
instead of using pre-defined verbs like `remove`, the `body` component of each 
`(field-name -> body)` clause is wrapped in an implicit SRFI-26 `cut`-like[2] 
form to create an anonymous function that's applied to the field's value. It's 
a super leaky abstraction because I use a different symbol for `->` depending 
on whether that record-field is a plain value, a thunk, or a `delay`-ed form 
(and it could be implemented more efficiently), but it greatly reduces the 
length and indentation level of repeatedly nested, inherited record variations.

```
((compose os-with-yubi
  [...])
   [operative-system])
```

I only wrote a handful of top-level `operating-system transformation` 
functions, and IIRC I only composed them at that top level; I think the way 
that you've broken them up into smaller forms and composed them out of each 
other though deeper, standard-functional composition gives you that same 
additive benefit over would-be nested forms, with each definition roughly 
matching up to one my "anonymous" invocations.

What I still dwell on is whether there's a way to further minimize the 
code-volume of including additional functionality (as was pondered in a prior 
response, and as `modify-record` does in obsoleting `modify-services`'s verbs), 
and how best to avoid order-dependencies and expose inherit 
inter-service-configuration dependencies and conflicts. Tropin's RDE uses an 
emacs or systemd-esque `provides`/`requires` system which is satisfying, but 
introduces implicit standardization on the symbols associated with software 
roles and builds a significant graph of them, which I feel adds to the "bulk" 
of the (still very elegant) solution and embraces the need to wrap every 
service and transformation into their cohesive system-- that's part of what's 
kept me on plain Guix with my bandaid-solutions (in the spirit of learning the 
standard approach before exploring larger systems built on top of it).

Anyway, I like your take, just fount it today and got to thinking-- thanks for 
putting it out there~

1: From: 
https://github.com/AutumnalAntlers/old-guix-config/blob/main/modules/antlers/systems/transformations/yubi.scm
2: Like `cut`, but deeper: see the `<>` symbol nested deep within in the 
`users` clause of the Yubi example.

Re: A friendlier API for operating-system declarations

[antlers]

Oh, thank you! Don't study too closely-- I'm quite the novice myself,
having come upon your work in search of better solutions, and the
drawbacks I described are rather notable; but I appreciate that you
see what I was going for.

(I'd be much happier with it if I could interrogate `(guix record)`
structures for the `plain | thunked | delayed` distinction and hide
that from the user, as that's what really prevents me from
recommending wider use, but understanding of the module eludes me.)

On Mon, Feb 19, 2024, at 5:53 PM, Edouard Klein wrote:
> This is awesome. I need to study your macro to understand how you wrote
> it. I've only written a handful of macro in my life, and came to lisp
> via common lisp, from which I've learned quite a lot apprently bad
> habits, so I'm lost in scheme-land.
>
> I'll study your system, thant _you_ for your reply !
>
> Cheers,
>
> Edouard.
>
>
> antlers  writes:
>
>> Hi!
>>
>> Just wanted to say that I really admire your take on end-user service 
>> configuration in the Beaver Labs channel.
>>
>> I gravitated towards composing functions over `operating-systems` myself, 
>> though
>> my config is probably only ""notable"" for the moderately-cursed 
>> `modify-record`
>> macro that I use to derive/configure/wrap the services[1]:
>>
>> ```
>> (define (os-with-yubi parent users*)
>>   (modify-record parent
>> (groups -> (cons (user-group (name "plugdev")) <>))
>> (users  -> (map (lambda (user)
>>   (if (member (user-account-name user)
>>   users*)
>>   (modify-record user
>> (supplementary-groups -> (cons "plugdev" <>)))
>>   user))
>> <>))
>> (services => (append <> (list
>>   (service pcscd-service-type)
>>   (simple-service 'u2f-udev-rules udev-service-type
>>   (list (specification->package "libu2f-host")))
>>   (simple-service 'yubi-udev-rules udev-service-type
>>   (list (specification->package 
>> "yubikey-personalization"
>> ```
>>
>>
>> It's like if `modify-services` was generalized over any kind of record, but
>> instead of using pre-defined verbs like `remove`, the `body` component of 
>> each
>> `(field-name -> body)` clause is wrapped in an implicit SRFI-26 `cut`-like[2]
>> form to create an anonymous function that's applied to the field's value. 
>> It's a
>> super leaky abstraction because I use a different symbol for `->` depending 
>> on
>> whether that record-field is a plain value, a thunk, or a `delay`-ed form 
>> (and
>> it could be implemented more efficiently), but it greatly reduces the length 
>> and
>> indentation level of repeatedly nested, inherited record variations.
>>
>> ```
>> ((compose os-with-yubi
>>   [...])
>>[operative-system])
>> ```
>>
>> I only wrote a handful of top-level `operating-system transformation` 
>> functions,
>> and IIRC I only composed them at that top level; I think the way that you've
>> broken them up into smaller forms and composed them out of each other though
>> deeper, standard-functional composition gives you that same additive benefit
>> over would-be nested forms, with each definition roughly matching up to one 
>> my
>> "anonymous" invocations.
>>
>> What I still dwell on is whether there's a way to further minimize the
>> code-volume of including additional functionality (as was pondered in a prior
>> response, and as `modify-record` does in obsoleting `modify-services`'s 
>> verbs),
>> and how best to avoid order-dependencies and expose inherit
>> inter-service-configuration dependencies and conflicts. Tropin's RDE uses an
>> emacs or systemd-esque `provides`/`requires` system which is satisfying, but
>> introduces implicit standardization on the symbols associated with software
>> roles and builds a significant graph of them, which I feel adds to the 
>> "bulk" of
>> the (still very elegant) solution and embraces the need to wrap every service
>> and transformation into their cohesive system-- that's part of what's kept 
>> me on
>> plain Guix with my bandaid-solutions (in the spirit of learning the standard
>> approach before exploring larger systems built on top of it).
>>
>> Anyway, I like your take, just fount it today and got to thinking-- thanks 
>> for putting it out there~
>>
>> 1: From: 
>> https://github.com/AutumnalAntlers/old-guix-config/blob/main/modules/antlers/systems/transformations/yubi.scm
>> 2: Like `cut`, but deeper: see the `<>` symbol nested deep within in the 
>> `users` clause of the Yubi example.

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Ludovic Courtès]

Hello!

Edouard Klein  skribis:

> Thank you Liliana and Attila for the swift and actionable feedback :)
>
> Below is a revised proposition.
>
> Here is a minimal working example of an os declaration:
> --mwe.scm---
> (use-modules
>  (beaver system)
>  (beaver functional-services)
>  (gnu packages version-control)
>  (gnu services web)
>  (gnu services telephony)
>  (gnu services ssh)
>  (gnu services base)
>  (guix gexp))
>
> (-> (minimal-ovh "osef")
> (instantiate nginx)
> (instantiate mumble-server
>  (welcome-text "coucou")
>  (port 64738))
> (extend openssh `(("alice" ,(local-file 
> "/home/edouard/.ssh/id_rsa.pub"
> (modify openssh
> (password-authentication? #f)
> (allow-empty-passwords? #t))
> (remove guix))
> ---
>
> To see the value of this syntactic sugar, try to replicate this MWE with
> the standard syntax. It's not horrendous, but it *is* off-putting to
> many newcomers to git, whereas this sugary piece is more readable for
> them (sample size of 1, p=0.0005).

Glad you ran a user study.  :-)

This is looking more and more like:

  (define-syntax ->
(syntax-rules ()
  ((_ os rules ...)
   (operating-system
 (inherit os)
 (services (modify-services (operating-system-user-services os)
 rules ...))

One thing that always makes me hesitate due to the longstanding hygienic
macro tradition in Scheme is non-hyienic introduction of identifiers: in
the example you gave above, the identifiers ‘openssh-service-type’ and
‘openssh-configuration’ are automatically derived from ‘openssh’.  This
could lead to surprises and makes grepping harder.  But I don’t know,
maybe that’s the price to pay?

Maybe one conclusion we can draw from this is that configuration records
and service types should be more closely tied to one another.

>>> (service+ OS SERVICE [CONF])
>>> (service- OS SERVICE)

This one looked really revolutionary, but it’s inspiring too!

Thanks,
Ludo’.

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Michal Atlas]

Hello,


just FTR, i don't think that the guix codebase is too bad in this regard.

It's not bad for the slightly initiated, which does still take time, and I 
think that front would perhaps benefit from some attention.


My main take away from this was that what I liked mostly about the BeaverLabs 
syntax was how short and simple minimalist systems became and that larger 
intentions could be represented by short invocations. The thing is services 
already allow that. There's os/git a beautiful function in BeaverLabs, which 
calls a set of other beaverlabs functions but it could equivalently be a 
service which extends other services. Perhaps this implies that what is wished 
for is a set of convenience services that combine sets of higher-level 
features, and making the operating-system record itself friendlier to 
specifying very little.


I did mention observations about the functional approach here, but after a 
while of using it, I myself in the end failed to see its merits apart from the 
slightly shorter service declaration which I personally recovered by a trivial
( #NAME# #BODY# ...) => (service #NAME#-service-type (#NAME#-configuration 
#BODY# ...))
replacement, without the need for all the extra effort.


I agree with the points made by others about grepability and approachability. 
Perhaps though BeaverLabs service syntax could deserve a mention as an 
alternative syntax somewhere in the cookbook or as an appendix in the 
manual/blogpost to show how malleable guix is.

Cheers

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Attila Lendvai]

> > the downside of generating countless macros is that they won't show up
> > in backtraces, and they cannot be found when grep'ping the codebase,
> > and as such make the codebase much less approachable.
> 
> 
> Reading your words really helped me feel that I'm not alone. You more or
> less summarized my feelings about the Guix codebase, which I have been
> reading now for over a year. Guile's syntax features make the code more
> symbolic and less approachable to newcomers.


just FTR, i don't think that the guix codebase is too bad in this regard.

here i just wanted to remind people to the not so obvious cost of syntactic 
abstractions that should be accounted for when making decisions. introducing 
macros that generate macros is rarely justified.

in general, it's *very* valuable when stuff can be grep'ped -- and not only for 
newcomers. after enough time has passed i can feel like a newcomer to my own 
codebase... :) modulo the protocols that i keep while writing code.

e.g. define.*whatever is a grep that i regularly employ. the pattern here is 
that although there are countless ways to define countless different stuff, 
there's a convention to stick to the define.*[name] pattern.

intuitive, unique (i.e. grep'pable) names are also key to facilitate code 
approachability, especially for abstractions that are scattered around in many 
source files. in some situations i even copy-paste names of abstractions into 
comments for any future grep'ping to pick it up.

a negative example for this in the guix codebase is the use of 'service' to 
describe two rather different abstractions: a component of an OS vs. a deamon 
process run by shepherd. for a while it caused me quite some confusion.



-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“The State is that organization in society which attempts to maintain a 
monopoly of the use of force and violence in a given territorial area; in 
particular, it is the only organization in society that obtains its revenue not 
by voluntary contribution or payment for services rendered but by coercion.”
— Murray N. Rothbard (1926–1995), 'Anatomy of the State' (1974)

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Development of GNU Guix and the GNU System distribution.]

Hi Attila,

On Wed, Nov 29 2023, Attila Lendvai wrote:

> i agree. in my experience, it's good practice in general to try to
> make a functional API as convenient as possible, and if that is still
> too verbose or cumbersome, only then add a thin layer of syntactic
> abstractions that expand to code that uses this functional API.
>
> [...]
>
> the downside of generating countless macros is that they won't show up
> in backtraces, and they cannot be found when grep'ping the codebase,
> and as such make the codebase much less approachable.

Reading your words really helped me feel that I'm not alone. You more or
less summarized my feelings about the Guix codebase, which I have been
reading now for over a year. Guile's syntax features make the code more
symbolic and less approachable to newcomers.

Of course, syntax features are helpful and necessary in some places.

Any large project like GNU Guix, however, must continue to engage a
cadre of skilled maintainers. Between Guile records and syntax features,
code can get hairy pretty fast. Some files seem to have received more
hurried attention recently. In the long run, it's not good for Guix to
make code hard to read.

Please forgive me, everyone, for speaking up. I did not follow the
discussion here at all. Attila's commentary merely resonated with me as
I tried to deal with my own shortcomings in contributing to the Guix
codebase.

Eventually, I hope to make meaningful suggestions to the bootloader
interaction with system profiles, and to the automatic "wrapping" of
executables, which is currently done by hand.

The changes Edouard and Michal may well be very valuable. Please do not
allow my comments to reflect on their well-meant ideas and hard work, or
on any other contributions to the software we love.

Thank you all for contributing to GNU Guix!

Kind regards
Felix

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Attila Lendvai]

> lines of code. I think hyper-focusing on syntax to make services
> "nicer" might be the wrong approach here: You could greatly reduce the
> complexity by making them procedures instead of syntax and still keep
> most of the configuration readable to a great extent.


i agree. in my experience, it's good practice in general to try to make a 
functional API as convenient as possible, and if that is still too verbose or 
cumbersome, only then add a thin layer of syntactic abstractions that expand to 
code that uses this functional API.

such code is much easier to work with, especially when debugging stuff 
interactively (i.e. it's possible to recompile a function that will then affect 
every place in the code where the macrology is used).

the downside of generating countless macros is that they won't show up in 
backtraces, and they cannot be found when grep'ping the codebase, and as such 
make the codebase much less approachable. the size of their expansion can also 
become an issue if they are used often enough.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“They muddy the water, to make it seem deep.”
— Friedrich Nietzsche (1844–1900)

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Liliana Marie Prikler]

Am Sonntag, dem 26.11.2023 um 21:46 +0100 schrieb Edouard Klein:
> 
> Liliana Marie Prikler  writes:
> > >     (instantiate nginx)
> > I do wish you spelled out service.  Also, instantiate takes as much
> > characters to type as add-service.
> > 
> 
> Done, see below. I was worried about the paronymy between add-service
> and add-services which already exists. I defer to you and your
> experience on this, naming things is hard.
That's not that much of a deal in scheme, see let and let* for example.

> > > To see the value of this syntactic sugar, try to replicate this
> > > MWE
> > > with the standard syntax. It's not horrendous, but it *is* off-
> > > putting to many newcomers to git, whereas this sugary piece is
> > > more
> > > readable for them (sample size of 1, p=0.0005).
> > Well, that'd be
> > ...
> 
> I tried:
> 
> (let ((base (minimal-ovh "osef")))
>   (operating-system
>     (inherit base)
>     (services
>   (cons*
>    (service nginx-service-type)
>    (service mumble-server-service-type
>     (mumble-server-configuration
>  (welcome-text "couocu")
>  (port 64738)))
>    (service openssh-service-type
>     (openssh-configuration
>  (password-authentication? #f)
>  (allow-empty-passwords? #t)
>  (authorized-keys `(("alice" ,(local-file
> "/home/edouard/.ssh/id_rsa.pub"))
>    (operating-system-user-services base)
> 
> 
> I admit that this is as readable as the sweet version, but:
> 
> - Openssh is already defined in  (minimal-ovh "osef"), so the build
>   fails with: 'guix system: error: service 'ssh-daemon' provided more
>   than once'
> - you forgot the removal of guix-service, which admitedly is not used
> much in practice
>   anyway
Nice catch.

> The problem with those two is that they break the nice structure of
> just adding services, and now one has to first remove an element from
> (operating-system-user-services base), then edit one of the element
> of the resulting list, then add to elements to it. It is probably
> possible to write it in a readable way, maybe less so than the sweet
> version, but not so much as to justify adding the new macros to guix.
Fair enough, but you could still implement that as an (extended)
modify-services.  We haven't reached the level of complexity where we'd
touch multiple fields, which is when operating-system really becomes a
pain to work with.  (The ssh-user from beaverlabs is one such example,
that IIRC is also used extensively on the-dam.)

> However the readability is not the main selling point, the
> writeability is. Please do not discount how hard it is to write that
> kind of stuff when scheme's not your main language. It is possible
> people here forgot how hard this is for beginners, especially those
> like me whose brain is deformed from years using algol-derived
> syntaxes.
> 
> I think we are losing a lot of mindshare because of the hard step of
> learning both guile and guix, and the kind of syntactic sugar I
> suggest may help bridge the gap long enough for newcomers to ease
> into the syntax, after they get a taste of guix' capabilities.
Fair point, and I'm not really opposed to making things more
readable/writable.  The problem comes with groking the additional
syntax.  It is not something the manuals would prepare you for and
probably also encounters weird corner cases like services using
something else than SERVICE-configuration for its configuration data.

> Another experiment: with the sweet syntax, you can easily extend
> services, without having to define a -record-type, etc. Just define a
> function. I can write more at length about that if you want.
That's again just simple-service doing its job tho :)

In essence, what I'm trying to get here is something that'd let us
implement the more complicated part of OS config fiddling with few
lines of code.  I think hyper-focusing on syntax to make services
"nicer" might be the wrong approach here: You could greatly reduce the
complexity by making them procedures instead of syntax and still keep
most of the configuration readable to a great extent.

Maybe we should start with making modify-services more powerful to also
cover the "adding services" case: modify-inputs already does that, so
it's weird that modify-services doesn't.  We should also think on how
to best wrap this in a lambda then; defining a new syntax for most
operations doesn't seem all that useful and scalable to me.

Cheers
> 

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Edouard Klein]

Liliana Marie Prikler  writes:
>>     (instantiate nginx)
> I do wish you spelled out service.  Also, instantiate takes as much
> characters to type as add-service.
>

Done, see below. I was worried about the paronymy between add-service
and add-services which already exists. I defer to you and your
experience on this, naming things is hard.

>> To see the value of this syntactic sugar, try to replicate this MWE
>> with the standard syntax. It's not horrendous, but it *is* off-
>> putting to many newcomers to git, whereas this sugary piece is more
>> readable for them (sample size of 1, p=0.0005).
> Well, that'd be
> ...

I tried:

(let ((base (minimal-ovh "osef")))
  (operating-system
(inherit base)
(services
  (cons*
   (service nginx-service-type)
   (service mumble-server-service-type
(mumble-server-configuration
 (welcome-text "couocu")
 (port 64738)))
   (service openssh-service-type
(openssh-configuration
 (password-authentication? #f)
 (allow-empty-passwords? #t)
 (authorized-keys `(("alice" ,(local-file 
"/home/edouard/.ssh/id_rsa.pub"))
   (operating-system-user-services base)


I admit that this is as readable as the sweet version, but:

- Openssh is already defined in  (minimal-ovh "osef"), so the build
  fails with: 'guix system: error: service 'ssh-daemon' provided more
  than once'
- you forgot  the removal of guix-service, which admitedly is not used much in 
practice
  anyway

The problem with those two is that they break the nice structure of just
adding services, and now one has to first remove an element from
(operating-system-user-services base), then edit one of the element of
the resulting list, then add to elements to it. It is probably possible
to write it in a readable way, maybe less so than the sweet version, but
not so much as to justify adding the new macros to guix.

However the readability is not the main selling point, the writeability
is. Please do not discount how hard it is to write that kind of stuff
when scheme's not your main language. It is possible people here forgot
how hard this is for beginners, especially those like me whose brain is
deformed from years using algol-derived syntaxes.

I think we are losing a lot of mindshare because of the hard step of
learning both guile and guix, and the kind of syntactic sugar I
suggest may help bridge the gap long enough for newcomers to ease into
the syntax, after they get a taste of guix' capabilities.

Another experiment: with the sweet syntax, you can easily extend
services, without having to define a -record-type, etc. Just define a
function. I can write more at length about that if you want.

> On that note, we also have extend-openssh-authorized-keys for the use
> with modify-services.
>
I see it now in the source, but I was unaware of its existence.




Thanks for the swift again review :)

Cheers,

Edouard.


-mwe.scm

(use-modules
 (beaver system)
 (beaver functional-services)
 (gnu packages version-control)
 (gnu services web)
 (gnu services telephony)
 (gnu services ssh)
 (gnu services base)
 (guix gexp))

(-> (minimal-ovh "osef")
(add-service nginx)
(add-service mumble-server
 (welcome-text "coucou")
 (port 64738))
(extend-service openssh `(("alice" ,(local-file 
"/home/edouard/.ssh/id_rsa.pub"
(modify-service openssh
(password-authentication? #f)
(allow-empty-passwords? #t))
(remove-service guix))

--functional-services.scm

(define-module (beaver functional-services)
   #:use-module (gnu system)
   #:use-module (gnu services)
   #:export (add-service extend-service modify-service remove-service))

(define syntax->string (compose symbol->string syntax->datum))

(define (service-configuration stx service)
  "Return the syntax one can use to refer to xxx-configuration for the given
service"
  (datum->syntax stx (string->symbol
  (string-append
   (syntax->string service)
   "-configuration"

(define (service-type stx service)
  "Return the syntax one can use to refer to xxx-service-type for the given
service"
  (datum->syntax stx (string->symbol
  (string-append
   (syntax->string service)
   "-service-type"

(define-syntax add-service
  (lambda (stx)
(syntax-case stx ()
  [(_ os service-name)
   (with-syntax
([service-type (service-type stx #'service-name)])
#'(begin
((lambda (x)  ;; It is wrapped in a lamba to make sure os is
   ;; evaluated once only. It it wasn't in a labmda, whatever
   ;; form os is in the calling code would be repeated
   ;; multiple times, and so if the form was e.g. (some-func
   ;; os), then 

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Michal Atlas]

Hello guix,

I'll chime in because I've been playing around with this kind of thing 
in my channel [1]

for a bit now and perhaps some of the ideas will be deemed useful.


(service+ OS SERVICE [CONF])
(service- OS SERVICE)
(modify-service OS SERVICE UPDATE)



I found that defining a functional API like this is about as convenient 
as the original version, more examples: [2].


Another small change I tried was to not have -> be a macro but rather a 
simple fold.


I took what the dot-macro form expands to and made it a curried function 
called services

(but remove and modify could be equivalently defined),
which takes a service and then returns a lambda that takes an OS and 
modifies it

just as the beaver-labs macros do with that extra service.


(define ((services . new) os)
  (operating-system
    (inherit os)
    (user-services
  (append new (operating-system-user-services os)

(define (-> system . services)
  (fold (cut <> <>) system services))


(-> os (services (service openssh-service-type)) ...) ; yields os with 
added openssh


((services (service openssh-service-type)) os) ; equivalent


The one macro that is indispensable for terse configurations is still 
some sort of service+ (I use  as the name),

which appends -service-type, surrounds the body in a -configuration [3],
and in my case calls `services` on it for good measure to get the os 
modifying lambda straight from the service+ clause.



(-> os
  ( openssh)
  ( guix-publish
    (advertise? #t))
  ...)


This ends up save for 2 extra characters being syntactically almost 
identical to the original version,

just passes around lambdas instead of getting manipulated by nested macros.

Them being functions they're now easier to work with with in more 
traditional Scheme ways,
for example compose works, and we can wrap them in other functions that 
for example apply them only in some circumstances.
Or combine transformations into new ones. The definitions for which end 
up being remarkably short and generalizable.



(-> os

  (compose (os/hostname "the-dam.org") ( gpm)) ; freely composable 
with other functions of the same sort


  (%services-with-arguments ...) ; indifferent to being composed on the 
spot even in the middle of threading


  (if-host "hydra" ( openssh)) ; only adds openssh if the host is 
named hydra, simple function


   ; since the second argument is just a function that may or 
may not be applied by the new lambda


  ((mapped-file-systems  ...)

     ...))) ; not threading by 
macro allows more complex structures without confusion




Then again none of this has had proper field testing, the fact that it 
has less gotchas is just a personal opinion.



Cheers, and all the best


[1]: 
https://git.sr.ht/~michal_atlas/guix-channel/tree/a31b68b46da60002383e2793eba88b99fc5c2382/item/atlas/combinators.scm 
(modified from the original beaver-labs version).


[2]: 
https://git.sr.ht/~michal_atlas/dotfiles/tree/8c78f53139ae176ff0a4cab82ad0fb64bce6989b/item/atlas/config/system/services.scm#L56 



[3]: 
https://git.sr.ht/~michal_atlas/guix-channel/tree/master/item/atlas/utils/services.scm#L53 

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Liliana Marie Prikler]

Am Sonntag, dem 26.11.2023 um 17:49 +0100 schrieb Edouard Klein:
> Thank you Liliana and Attila for the swift and actionable feedback :)
> 
> Below is a revised proposition.
> 
> Here is a minimal working example of an os declaration:
> --mwe.scm---
> (use-modules
>  (beaver system)
>  (beaver functional-services)
>  (gnu packages version-control)
>  (gnu services web)
>  (gnu services telephony)
>  (gnu services ssh)
>  (gnu services base)
>  (guix gexp))
> 
> (-> (minimal-ovh "osef")
>     (instantiate nginx)
I do wish you spelled out service.  Also, instantiate takes as much
characters to type as add-service.
>     (instantiate mumble-server
>  (welcome-text "coucou")
>  (port 64738))
>     (extend openssh `(("alice" ,(local-file
> "/home/edouard/.ssh/id_rsa.pub"
>     (modify openssh
>     (password-authentication? #f)
>     (allow-empty-passwords? #t))
>     (remove guix))
> ---
> 
> To see the value of this syntactic sugar, try to replicate this MWE
> with the standard syntax. It's not horrendous, but it *is* off-
> putting to many newcomers to git, whereas this sugary piece is more
> readable for them (sample size of 1, p=0.0005).
Well, that'd be 
  (let ((base (minimal-ovh "osef")))
(operating-system
  (inherit base)
  (services
(cons*
 (service nginx-service-type)
 (service mumble-service-type
  (mumble-configuration
   (welcome-text "couocu")
   (port 64738)))
 (service openssh-service-type
  (openssh-configuation
   (password-authentication? #f)
   (allow-empty-passwords? #t)
   (authorized-keys  (minimal-ovh "osef")
  (lambda (base) …))

On that note, we also have extend-openssh-authorized-keys for the use
with modify-services.

> Here is the revised functional-services.scm, not yet commited and
> pushed, and only lightly tested in local containers, but not in
> production:
> 
> Advice and comments welcome :)
> 
> 
> functional-services.scm--
> 
> 
> (define-module (beaver functional-services)
>    #:use-module (gnu system)
>    #:use-module (gnu services)
>    #:export (instantiate extend modify remove))
> 
> (define syntax->string (compose symbol->string syntax->datum))
> 
> (define (service-configuration stx service)
>   "Return the syntax one can use to refer to xxx-configuration for
> the given
> service"
>   (datum->syntax stx (string->symbol
>   (string-append
>    (syntax->string service)
>    "-configuration"
> 
> (define (service-type stx service)
>   "Return the syntax one can use to refer to xxx-service-type for the
> given
> service"
>   (datum->syntax stx (string->symbol
>   (string-append
>    (syntax->string service)
>    "-service-type"
> 
> (define-syntax instantiate
>   (lambda (stx)
>     (syntax-case stx ()
>   [(_ os service-name)
>    (with-syntax
>     ([service-type (service-type stx #'service-name)])
>     #'(begin
>     ((lambda (x)  ;; It is wrapped in a lamba to make sure os
> is
>    ;; evaluated once only. It it wasn't in a labmda,
> whatever
>    ;; form os is in the calling code would be repeated
>    ;; multiple times, and so if the form was e.g. (some-
> func
>    ;; os), then some-func would be called multiple times,
>    ;; which may not be desirable.
Isn't it also wrapped in a lambda, because -> is a threading macro that
takes functions rather than syntax?


Cheers

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Edouard Klein]

Thank you Liliana and Attila for the swift and actionable feedback :)

Below is a revised proposition.

Here is a minimal working example of an os declaration:
--mwe.scm---
(use-modules
 (beaver system)
 (beaver functional-services)
 (gnu packages version-control)
 (gnu services web)
 (gnu services telephony)
 (gnu services ssh)
 (gnu services base)
 (guix gexp))

(-> (minimal-ovh "osef")
(instantiate nginx)
(instantiate mumble-server
 (welcome-text "coucou")
 (port 64738))
(extend openssh `(("alice" ,(local-file "/home/edouard/.ssh/id_rsa.pub"
(modify openssh
(password-authentication? #f)
(allow-empty-passwords? #t))
(remove guix))
---

To see the value of this syntactic sugar, try to replicate this MWE with
the standard syntax. It's not horrendous, but it *is* off-putting to
many newcomers to git, whereas this sugary piece is more readable for
them (sample size of 1, p=0.0005).

Here is the revised functional-services.scm, not yet commited and
pushed, and only lightly tested in local containers, but not in
production:

Advice and comments welcome :)


functional-services.scm--


(define-module (beaver functional-services)
   #:use-module (gnu system)
   #:use-module (gnu services)
   #:export (instantiate extend modify remove))

(define syntax->string (compose symbol->string syntax->datum))

(define (service-configuration stx service)
  "Return the syntax one can use to refer to xxx-configuration for the given
service"
  (datum->syntax stx (string->symbol
  (string-append
   (syntax->string service)
   "-configuration"

(define (service-type stx service)
  "Return the syntax one can use to refer to xxx-service-type for the given
service"
  (datum->syntax stx (string->symbol
  (string-append
   (syntax->string service)
   "-service-type"

(define-syntax instantiate
  (lambda (stx)
(syntax-case stx ()
  [(_ os service-name)
   (with-syntax
([service-type (service-type stx #'service-name)])
#'(begin
((lambda (x)  ;; It is wrapped in a lamba to make sure os is
   ;; evaluated once only. It it wasn't in a labmda, whatever
   ;; form os is in the calling code would be repeated
   ;; multiple times, and so if the form was e.g. (some-func
   ;; os), then some-func would be called multiple times,
   ;; which may not be desirable.
   (operating-system
 (inherit x)
 (services
  (cons
   (service service-type)
   (operating-system-user-services x)
 os)))]
  [(_ os service-name forms ...)
   (with-syntax
([service-type (service-type stx #'service-name)]
 [service-configuration (service-configuration stx #'service-name)])
#'(begin
((lambda (x)  ;; Wrapping in a lambda for the same reasons as above
   (operating-system
 (inherit x)
 (services
  (cons
   (service service-type
 (service-configuration forms ...))
   (operating-system-user-services x)
 os)))])))

(define-syntax extend
  (lambda (stx)
(syntax-case stx ()
  [(_ os service-name forms ...)
   (with-syntax
([service-type (service-type stx #'service-name)])
#'(begin
((lambda (x)
   (operating-system
 (inherit x)
 (services
  (cons
   (simple-service (format  #f "A ~a extension" (syntax->string 
#'service-name))
   service-type
   forms ...)
   (operating-system-user-services x)
 os)))])))

(define-syntax modify
  (lambda (stx)
(syntax-case stx ()
  [(_ os service-name forms ...)
   (with-syntax
([service-type (service-type stx #'service-name)]
 [service-configuration (service-configuration stx #'service-name)])
#'(begin
((lambda (x)
   (operating-system
 (inherit x)
 (services
  (modify-services (operating-system-user-services x)
(service-type
 config =>
 (service-configuration
  (inherit config)
  forms ...))
 os)))])))

(define-syntax remove
  (lambda (stx)
(syntax-case stx ()
  [(_ os service-name forms ...)
   (with-syntax
([service-type (service-type stx #'service-name)])
#'(begin
((lambda (x)
  

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Attila Lendvai]

> (service+ OS SERVICE [CONF])
> (service- OS SERVICE)
> (modify-service OS SERVICE UPDATE)


what would the benefit of generating multiple macros for each service compared 
to the above functional API (with 3-4 elements altogether)?

i could be missing something here, but it seems to be precious little to me 
while it costs some extra complexity.

if i were to add a syntactic abstraction for this, i'd generate a full DSL in 
the form of a (modify-operating-system OS [fictional DSL to describe desired 
changes]).

but i don't think the extra complexity justifies any macrology here.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
By the time a man realises that his father was right, he has a son who thinks 
he’s wrong.

Re: Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Liliana Marie Prikler]

Am Freitag, dem 24.11.2023 um 22:43 +0100 schrieb Edouard Klein:
> Dear Guixers,
> 
> Here is a quick status update on my proposition to expose composable
> functions to change operating-system declarations.
> 
> Thank you all for the feedback you gave me :) It's very nice to not
> be talking in the void.
> 
> After Liliana opined that these functions should not put too much
> burden on the maintainers, a position with which I wholeheartedly
> agree, here is what I came up with:
> 
> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/functional-services.scm
> 
> (file copied at the end of this email to make the list self-content
> instead of relying on external services)
> 
> That's a macro-writing macro called "define-os-macros-for" that, for
> any service (for example nginx), defines five forms. They all
> evaluate to a modified os:
> 
> (.nginx os)  ;; Add nginx in its default configuration
> (.nginx os toto titi tutu... )  ;; Add nginx, with toto... given as
> arguments to nginx-configuration
> (+nginx os toto titi tutu...)  ;; Extend an existing nginx service,
> giving
> toto... as arguments to simple-service
> (~nginx os toto...)  ;; Edit an existing nginx service, passing
> toto... as
> arguments to modify-service
> (-nginx os)  ;; Removes nginx
> 
> You can see all of them used in:
> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm?ref_type=heads
> 
> e.g. the mkdir-p function is a (+activation ...),
> all web server functions use (+nginx...),
> mumble is (.mumble... ),
> os/git uses (~openssh... ).
> 
> I've use (-foo) forms while testing, but not in production. The
> others are currently running in prod on guix-hosting.com and the-
> dam.org.
The naming is a little confusing and imho not clearly helpful.  For
example, why (.nginx os) instead of the triple

(service+ OS SERVICE [CONF])
(service- OS SERVICE)
(modify-service OS SERVICE UPDATE)

Of course, you could also define triples (add-SERVICE OS [CONF]), 
(remove-SERVICE OS) and (modify-SERVICE OS UPDATE), which would fall in
line with the macro approach, but keep names a little more readable. 
(If you want to use your names, you can rename them in #:use-modules).

> -functional-services.scm
> (define-module (beaver functional-services)
>    #:use-module (gnu system)
>    #:export (*-append))
> 
> (define (syntax->string s)
>   "Shorthand to convert a piece of syntax to a string"
>   (symbol->string (syntax->datum s)))
Or (compose symbol->string syntax->datum).

> (define (*->string s)
>   "Convert into a string any symbol-like type we may encounter in a
> macro"
>   (cond
>    [(string? s) s]
>    [(symbol? s) (symbol->string s)]
>    [else (syntax->string s)]))  ;; assume syntax
There should not be a need to define this.  Try to use exact types.

> (define* (*-append #:rest args)
>   "Return a symbol which is a concatenation of the given symbol-like
> args"
>   (string->symbol (apply string-append (map *->string args
As above.

> (define-syntax define-os-macros-for
>   ;; This macro, called like e.g. (define-os-macros-for foo) will
> define four
>   ;; forms, .foo, +foo, ~foo and -foo
>   ;;
>   ;; These forms take an operating-system as their first argument,
> and
>   ;; evaluate to an operating-system, which allows one to compose
> them
>   ;; (e.g. (foo (bar os))).
>   (lambda (x)
>     (syntax-case x ()
>   [(_ foo)
>    (with-syntax
>     ([dot-foo   (datum->syntax x (*-append "." #'foo))]
>  [plus-foo  (datum->syntax x (*-append "+" #'foo))]
>  [tilde-foo (datum->syntax x (*-append "~" #'foo))]
>  [minus-foo (datum->syntax x (*-append "-" #'foo))]
>  [foo-service-type  (datum->syntax x (*-append #'foo "-
> service-type"))]
>  [foo-configuration (datum->syntax x (*-append #'foo "-
> configuration"))])
You're repeating datum->syntax here a bit too much for my liking.

This patch is also missing the most important part of beaverlabs to
make this work: the -> threading macro.

Cheers

Syntactic Diabetes (was Re: A friendlier API for operating-system declarations)

[Edouard Klein]

Dear Guixers,

Here is a quick status update on my proposition to expose composable
functions to change operating-system declarations.

Thank you all for the feedback you gave me :) It's very nice to not be
talking in the void.

After Liliana opined that these functions should not put too much burden
on the maintainers, a position with which I wholeheartedly agree, here
is what I came up with:

https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/functional-services.scm

(file copied at the end of this email to make the list self-content
instead of relying on external services)

That's a macro-writing macro called "define-os-macros-for" that, for any
service (for example nginx), defines five forms. They all evaluate to a
modified os:

(.nginx os)  ;; Add nginx in its default configuration
(.nginx os toto titi tutu... )  ;; Add nginx, with toto... given as
arguments to nginx-configuration
(+nginx os toto titi tutu...)  ;; Extend an existing nginx service, giving
toto... as arguments to simple-service
(~nginx os toto...)  ;; Edit an existing nginx service, passing toto... as
arguments to modify-service
(-nginx os)  ;; Removes nginx

You can see all of them used in:
https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm?ref_type=heads

e.g. the mkdir-p function is a (+activation ...),
all web server functions use (+nginx...),
mumble is (.mumble... ),
os/git uses (~openssh... ).

I've use (-foo) forms while testing, but not in production. The others
are currently running in prod on guix-hosting.com and the-dam.org.

That'll be the meat of the first half of my presentation during the
informal Paris meetup in december.

Do you folks think something like that could one day make it in the
mainline ?

Cheers,

Edouard.


-functional-services.scm
(define-module (beaver functional-services)
   #:use-module (gnu system)
   #:export (*-append))

(define (syntax->string s)
  "Shorthand to convert a piece of syntax to a string"
  (symbol->string (syntax->datum s)))

(define (*->string s)
  "Convert into a string any symbol-like type we may encounter in a macro"
  (cond
   [(string? s) s]
   [(symbol? s) (symbol->string s)]
   [else (syntax->string s)]))  ;; assume syntax

(define* (*-append #:rest args)
  "Return a symbol which is a concatenation of the given symbol-like args"
  (string->symbol (apply string-append (map *->string args


(define-syntax define-os-macros-for
  ;; This macro, called like e.g. (define-os-macros-for foo) will define four
  ;; forms, .foo, +foo, ~foo and -foo
  ;;
  ;; These forms take an operating-system as their first argument, and
  ;; evaluate to an operating-system, which allows one to compose them
  ;; (e.g. (foo (bar os))).
  (lambda (x)
(syntax-case x ()
  [(_ foo)
   (with-syntax
([dot-foo   (datum->syntax x (*-append "." #'foo))]
 [plus-foo  (datum->syntax x (*-append "+" #'foo))]
 [tilde-foo (datum->syntax x (*-append "~" #'foo))]
 [minus-foo (datum->syntax x (*-append "-" #'foo))]
 [foo-service-type  (datum->syntax x (*-append #'foo "-service-type"))]
 [foo-configuration (datum->syntax x (*-append #'foo 
"-configuration"))])
#'(begin
;; The fist macro is .foo
(define-syntax dot-foo
  (syntax-rules ()
;; Its simplest form is (.foo os), which will add an instance
;; of service foo to os.
[(dot-foo os)
 ((lambda (x)  ;; It is wrapped in a lamba to make sure os is
;; evaluated once only. It it wasn't in a labmda, whatever
;; form os is in the calling code would be repeated
;; multiple times, and so if the form was e.g. (some-func
;; os), then some-func would be called multiple times,
;; which may not be desirable.
(operating-system
  (inherit x)
  (services
   (cons
(service foo-service-type)
(operating-system-user-services x)
  os)]
;; This other form, (.foo os something bar baz...) takes
;; multiple forms as arguments, which are reproduced as is to
;; construct the foo-configuration
[(dot-foo os forms (... ...))
 ((lambda (x)  ;; Wrapping in a lambda for the same reasons as 
above
(operating-system
  (inherit x)
  (services
   (cons
(service foo-service-type
 (foo-configuration forms (... ...)))
(operating-system-user-services x)
  os)]))
;; The second one is +foo. It extends a service, reproducing the

Re: A friendlier API for operating-system declarations

[antlers]

Anecdotally, my personal configurations are built out of composable
operating-system transformers that add packages, services, etc. through
trees of inherited records.

That's fairly straightforward for eg. adding packages, but becomes more
complicated when you consider modifying service configurations (which you
might want to inherit from when present and create when absent), nested
service configurations (inheriting multiple times, the whole away down), or
modifying kernel-arguments (where you might want to strip values for the
option that you're setting from existing strings).

I feel that operating-system record fields should be `extendable` in the
same way as services, and have always assumed that that's basically what
RDE's `features` are.

It would be great to have structure that eliminates some of the nesting and
boilerplate of an expression like this:
```
;; imagine if this was (imperative-ness aside) as simple as:
;; lambda os: os.services(foo).z.foo += bar

(lambda (os)
  (operating-system
(inherit os)
(sevices
  (modify-services
(operating-system-services os)
(y-service-type
  config => (y-configuration
  (inherit config)
  (z-configuration
(z-configuration
 (inherit (y-configuration-z config)
 (foo (cons bar
(z-configuration-foo
  (y-configuration-z config)

;; And if we want to create the service when it doesn't exist?

(lambda (os)
  (operating-system
(inherit os)
(sevices
  (if (find (lambda (s)
  (eq (service-kind s) y))
(operating-system-services os))
  (modify-services
(operating-system-services os)
(y-service-type
  config => (y-configuration
  (inherit config)
  (z-configuration
(z-configuration
 (inherit (y-configuration-z config)
 (foo (cons bar
(z-configuration-foo
  (y-configuration-z
config))
  (cons (service y-service-type
  (y-configuration
(z-configuration
  (z-configuration
(foo (list bar))
(operating-system-services os))
```

Re: A friendlier API for operating-system declarations

[Edouard Klein]

Hi !
Ludovic Courtès  writes:

> Hi Edouard,
>
> Edouard Klein  skribis:
>
>> For my clients and my own use, I use a layer on top of operating-system
>> declarations in which I create functions that take an os as a first
>> argument, and return an os.
>>
>> With the help of the handy -> macro, I can chain them, thus allowing an
>> easy stacking of multiple "roles" or "functionalities" on the same
>> machine, and an easy transfer of said functionality to another machine:
>> just cut and paste one line from one declaration to the other.
>>
>> I have written a tutorial for my clients here:
>>
>> https://guix-hosting.com/docs/tuto/getting-started.html
>
> Very nice!
>

:)

>> After a few months of experience, and positive feedback from my clients,
>> my question to you guys is: would you be interested in mainlining this,
>> or should I keep my development efforts separate in my channel ?
>
> I think it’s worth considering, taking into account maintenance
> considerations as Liliana wrote.
>

Indeed. This was started as a quick and dirty overlay over guix to make
it easier for people used to Ansible to deal with declarative OS
configuration with guix, but it now need to be easy to maintain.

>> I do think this API is easier than manipulating services, and although
>> extendable services are awesome and a very nifty piece of engineering,
>> they require quite a good knowledge of scheme and take a while to be
>> used to, while this new API, while way less powerful, lowers the barrier
>> to entry for newcomers.
>
> Normally, adding a service amounts to adding a line in the ‘services’
> field:
>
>   (operating-system
> ;; …
> (services (append (list (service nginx-service-type) …)
>   %base-services)))
>
> It is more verbose that what you have, but it’s still roughly one line
> (or one entry) per service.
>

You wouldn't believe the irrational fear of nested parentheses some
people have ;-)

I do agree a purely functional interface to OS declaration wouldn't
bring much over the existing one. My goal is to then compose those
primitive functions into usually-used-together bundles, such as e.g.
nginx+gunicorn+flask in a function called, maybe, 'python-webapp'.

This is kind of subjective, though. I have my own usage and some
customer feedback to guide me, but we can keep it on my channel until we
feel confident that the additional value is worth the maintenance cost.


> Another interesting bit in your channel are templates though: an OS
> template for hosting at OVH, etc.  These are very valuable because they
> take care of all the boilerplate.
>

I'd be happy to mainline it. I'll submit a patch when I get back home.


> Thanks for sharing!
>

Well thanks to you for starting the whole schmilblick.

> Ludo’.

Edouard.

Re: A friendlier API for operating-system declarations

[Ludovic Courtès]

Hi Edouard,

Edouard Klein  skribis:

> For my clients and my own use, I use a layer on top of operating-system
> declarations in which I create functions that take an os as a first
> argument, and return an os.
>
> With the help of the handy -> macro, I can chain them, thus allowing an
> easy stacking of multiple "roles" or "functionalities" on the same
> machine, and an easy transfer of said functionality to another machine:
> just cut and paste one line from one declaration to the other.
>
> I have written a tutorial for my clients here:
>
> https://guix-hosting.com/docs/tuto/getting-started.html

Very nice!

> After a few months of experience, and positive feedback from my clients,
> my question to you guys is: would you be interested in mainlining this,
> or should I keep my development efforts separate in my channel ?

I think it’s worth considering, taking into account maintenance
considerations as Liliana wrote.

> I do think this API is easier than manipulating services, and although
> extendable services are awesome and a very nifty piece of engineering,
> they require quite a good knowledge of scheme and take a while to be
> used to, while this new API, while way less powerful, lowers the barrier
> to entry for newcomers.

Normally, adding a service amounts to adding a line in the ‘services’
field:

  (operating-system
;; …
(services (append (list (service nginx-service-type) …)
  %base-services)))

It is more verbose that what you have, but it’s still roughly one line
(or one entry) per service.

Another interesting bit in your channel are templates though: an OS
template for hosting at OVH, etc.  These are very valuable because they
take care of all the boilerplate.

Thanks for sharing!

Ludo’.

Re: A friendlier API for operating-system declarations

[Liliana Marie Prikler]

Am Donnerstag, dem 23.03.2023 um 09:06 +0100 schrieb Edouard Klein:
> After a few months of experience, and positive feedback from my
> clients, my question to you guys is: would you be interested in
> mainlining this, or should I keep my development efforts separate in
> my channel ?
Having just dropped a series of "old-style" service procedures, I don't
think that maintaining a separate API on the mainline is a good idea.

> I do think this API is easier than manipulating services, and
> although extendable services are awesome and a very nifty piece of
> engineering, they require quite a good knowledge of scheme and take a
> while to be used to, while this new API, while way less powerful,
> lowers the barrier to entry for newcomers.
I see how this style (and a similar one that has already been shown for
guix home) can improve configuration file clarity and that's a good
thing.  On the other hand, it looks like you currently have to manually
craft a lot of the procedures that go into the -> syntax and that's...
well, not so good.

If we could enable this style easier – say if we made it so that
define-configuration and service-type also emitted ready-to-use
constructors to use with a keyword style at little extra cost other
than compilation time, I think that'd be a net positive for everyone
involved.

Cheers

Re: A friendlier API for operating-system declarations

[Edouard Klein]

Hi Josselin,

Josselin Poiret  writes:

> [[PGP Signed Part:Undecided]]
> Hi Edouard,
>
> Edouard Klein  writes:
>
>> #+begin_src scheme
>> (->
>>  (minimal-ovh "ssh-rsa AAASomethingSomething== root@minimal-ovh")
>>  (http-static-content "sub2.example.com" #:to-dir "/srv/sub2")
>>  (http-static-content "sub1.example.com" #:to-dir "/srv/sub1/")
>>  (add-services my-db))
>> #+end_src
>>
>> The code of the function is on my channel:
>> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm
>>
>> After a few months of experience, and positive feedback from my clients,
>> my question to you guys is: would you be interested in mainlining this,
>> or should I keep my development efforts separate in my channel ?
>
> I am quite in favor of using operating-system transformations rather
> than inheritance, because they're composable!  This could be leveraged
> to get a nice CLI API as well.
>

I hadn't thought of that but now that you mention it, starting a basic
web server or database, isolated in a container, directly from the
command line may come in handy sometimes. This is a good idea ! I'll
think about how to achieve this :)

In the meantime something like:

#+begin_src bash
guix system container -e "(begin (use-modules (beaver system))
(http-static-content minimal-container))" --share=somedir=/var/www
#+end_src

Will spin up such a server. Kind of like python3 -m http.server, but
without the security flaws (you get a full blown nginx), and isolated in
a way that even if your process gets owned, it's only an unpriviledged
user in a container.


>> I do think this API is easier than manipulating services, and although
>> extendable services are awesome and a very nifty piece of engineering,
>> they require quite a good knowledge of scheme and take a while to be
>> used to, while this new API, while way less powerful, lowers the barrier
>> to entry for newcomers.
>
> By this, do you mean that there's no way with your syntax to modify a
> given service?  Is there a reason for this?  It does seem to me that it
> could probably be done.
>

I'm not sure I understand your question.

What I meant is that my proposal is only syntactic sugar over the
existing system, and not meant to replace it.

The existing system is actually very good and the way you
can define how to collate the instances of a given service type, even if
they are instanciated far away from one another, likely by different
authors is really clever and well made. I'm talking here about things
like how you can easily request that a new user be made for your
service, and account-service-type will collate all the users that need
to exist, and actually get the job done. Basically what fold-services does.

I just want to hide all of this complexity for people who just want to
activate one or more instances of a service coded up by somebody else.

If by modifying a service you mean the parameters like the port ones
listen to, or the directory the data should be put in etc. the functions
just take keyword parameters and pass them to the underlying
*-configuration scheme record. They are quite easy to edit and compose.
Maybe we could even devise a semi-automated way of generating one function
for every *-configuration structure that exists.

See
https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm#L236

how http-static-content is basically just a wrapper over 
nginx-server-configuration.

If you meant editing the service after the function has run, you then
need to inspect the operating-system record and fall back to the
original API, by finding the service in the record and editing its
members, but that seems complex to me. I'm probably misunderstanding
the use case.



>> They are an easy way to maintain a declarative whole operating system
>> configuration, with a syntax similar enough to docker and ansible that
>> sysadmins familiar with it can quickly get up and running, thus exposing
>> more people to Guix.
>>
>> What do you think ?
>
> You've got me interested :) especially since you already have customer
> feedback!
>

Thanks :)



Cheers,

Edouard.
> Best,

Re: A friendlier API for operating-system declarations

[Josselin Poiret]

Hi Edouard,

Edouard Klein  writes:

> #+begin_src scheme
> (->
>  (minimal-ovh "ssh-rsa AAASomethingSomething== root@minimal-ovh")
>  (http-static-content "sub2.example.com" #:to-dir "/srv/sub2")
>  (http-static-content "sub1.example.com" #:to-dir "/srv/sub1/")
>  (add-services my-db))
> #+end_src
>
> The code of the function is on my channel:
> https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm
>
> After a few months of experience, and positive feedback from my clients,
> my question to you guys is: would you be interested in mainlining this,
> or should I keep my development efforts separate in my channel ?

I am quite in favor of using operating-system transformations rather
than inheritance, because they're composable!  This could be leveraged
to get a nice CLI API as well.

> I do think this API is easier than manipulating services, and although
> extendable services are awesome and a very nifty piece of engineering,
> they require quite a good knowledge of scheme and take a while to be
> used to, while this new API, while way less powerful, lowers the barrier
> to entry for newcomers.

By this, do you mean that there's no way with your syntax to modify a
given service?  Is there a reason for this?  It does seem to me that it
could probably be done.

> They are an easy way to maintain a declarative whole operating system
> configuration, with a syntax similar enough to docker and ansible that
> sysadmins familiar with it can quickly get up and running, thus exposing
> more people to Guix.
>
> What do you think ?

You've got me interested :) especially since you already have customer
feedback!

Best,
-- 
Josselin Poiret


signature.asc
Description: PGP signature

A friendlier API for operating-system declarations

[Edouard Klein]

Dear Guixers,

For my clients and my own use, I use a layer on top of operating-system
declarations in which I create functions that take an os as a first
argument, and return an os.

With the help of the handy -> macro, I can chain them, thus allowing an
easy stacking of multiple "roles" or "functionalities" on the same
machine, and an easy transfer of said functionality to another machine:
just cut and paste one line from one declaration to the other.

I have written a tutorial for my clients here:

https://guix-hosting.com/docs/tuto/getting-started.html

that gives an example of what an os configuration may look like:
#+begin_src scheme
(->
 (minimal-ovh "ssh-rsa AAASomethingSomething== root@minimal-ovh")
 (http-static-content "sub2.example.com" #:to-dir "/srv/sub2")
 (http-static-content "sub1.example.com" #:to-dir "/srv/sub1/")
 (add-services my-db))
#+end_src

The code of the function is on my channel:
https://gitlab.com/edouardklein/guix/-/blob/beaverlabs/beaver/system.scm

After a few months of experience, and positive feedback from my clients,
my question to you guys is: would you be interested in mainlining this,
or should I keep my development efforts separate in my channel ?

I do think this API is easier than manipulating services, and although
extendable services are awesome and a very nifty piece of engineering,
they require quite a good knowledge of scheme and take a while to be
used to, while this new API, while way less powerful, lowers the barrier
to entry for newcomers.

They are an easy way to maintain a declarative whole operating system
configuration, with a syntax similar enough to docker and ansible that
sysadmins familiar with it can quickly get up and running, thus exposing
more people to Guix.

What do you think ?

Cheers,

Edouard.