Re: GSoC update
> Could you expound a bit? That’s a very short summary for all the sweat
> you’ve put in it. :-)
My apologies, at the time I sent the mail in a hurry.
Basically now instead of converting unit files to services individually it
happens in bulk so that it can check if there is a corresponding .socket file
per service file (and the reverse). If there is a corresponding .socket file
then the input and output of the .service will be redirected to the result of
(accept) on the socket corresponding to the .socket file.
It also makes the select that waits for the commands from herd to also wait for
the sockets.
> Also, what is the patch against? It’s not against ‘master’; I suppose
> it’s against the previous state of your own branch, do you have a copy
> of your repo on-line?
It's against the previous patch that I sent, but I can put the branch online.
> It’s OK that the thing doesn’t quite work—we knew it was not an easy
> task. What’s disappointing though is that you didn’t come to us to
> discuss the issues until now. GSoC is not about working in all
> loneliness separately from the rest of the group; it’s about becoming
> part of the group.
>
> On IRC Jelle and I (and possibly others) offered help on the ‘signalfd’
> issue; I also outlined reasonable milestones (first, only use
> signalfd(2) instead of SIGCHLD, then discuss together what it would take
> to Fiberize the whole thing.) It’s sad that you remained stuck instead
> of taking this opportunity to discuss it with us.
Until now (in general, not only during the gsoc) I tried to solve any issues
that had arisen when I was programming by myself, so it was a bit difficult to
change that mindset - I will try to be more communicative after this however.
> The patch changes lots of things and unfortunately, without
> explanations, I do not understand what to do with it. Like what’s the
> new feature? How is it used? What implementation choices were made?
> What’s left to be done?…
The new feature is initial support of .socket unit files. It is used like:
(let* ((port1 (open-input-file "/systemd/test.service"))
(port2 (open-input-file "/systemd/test.socket")))
(apply register-services (unit-files->services `(("/systemd/test.service"
"test" service ,(read-unit-file port1))
("/systemd/test.socket"
"test" socket ,(read-unit-file port2)
(close-port port1)
(close-port port2))
The things that are left are supporting more systemd options, and making it work
properly.
GSoC update
Hi Guix! This patch adds initial support for .socket unit files. It does not currently work but is near completion. During the past month I also worked on a patch that adds signalfd and fiber support but these are currently way too unstable and for that reason I have not included them in this patch. From cd260ae65056b53749e7c03f2498a28af2525934 Mon Sep 17 00:00:00 2001 From: Ioannis Panagiotis Koutsidis Date: Tue, 10 Jul 2018 20:03:21 +0300 Subject: [PATCH] .socket units --- modules/shepherd.scm | 44 +++-- modules/shepherd/service.scm | 170 ++--- modules/shepherd/systemd.scm | 354 --- 3 files changed, 368 insertions(+), 200 deletions(-) diff --git a/modules/shepherd.scm b/modules/shepherd.scm index 5d97598..45fcb23 100644 --- a/modules/shepherd.scm +++ b/modules/shepherd.scm @@ -31,6 +31,7 @@ #:use-module (shepherd config) #:use-module (shepherd support) #:use-module (shepherd service) + #:use-module (shepherd systemd) #:use-module (shepherd system) #:use-module (shepherd runlevel) #:use-module (shepherd args) @@ -259,9 +260,18 @@ (setvbuf command-source _IOFBF 1024) (process-connection command-source)) (_ #f))) - (match (select (list sock) (list) (list) (if poll-services? 0.5 #f)) -(((sock) _ _) - (read-from sock)) + + (match (select (cons* sock unit-sockets-list) (list) (list) + (if poll-services? 0.5 #f)) +(((rsock _ ...) _ _) + (let* ((sockserv1 (find (lambda (x) + (eq? (slot-ref x 'running) rsock)) + (service-list))) +(sockserv (if sockserv1 (before sockserv1) #f)) +(res (if sockserv (car (lookup-services (string->symbol (car sockserv #f))) + (if res + (start res (accept rsock)) + (read-from sock (_ #f)) (when poll-services? diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index 5b0d72d..ceba004 100644 --- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -74,6 +74,7 @@ make-forkexec-constructor make-kill-destructor exec-command +exec-command2 fork+exec-command read-pid-file make-system-constructor @@ -102,7 +103,10 @@ action-runtime-error-key action-runtime-error-arguments -condition->sexp)) +condition->sexp + +before +service-list)) ;; Keep track of lazy initialization of SIGCHLD handler (define %sigchld-handler-installed? #f) @@ -161,16 +165,12 @@ respawned, shows that it has been respawned more than TIMES in SECONDS." (requires #:init-keyword #:requires #:init-value '() #:getter required-by) - ;; If `#t', then assume the `running' slot specifies a PID and - ;; respawn it if that process terminates. Otherwise `#f'. + ;; If not #f, then assume the `running' slot specifies a PID and + ;; respawn it if that process terminates. Otherwise it can be + ;; 'on-success, 'on-failure, 'on-abnormal, 'on-watchdog, 'on-abort, or 'always (respawn? #:init-keyword #:respawn? - #:init-value #f + #:init-value 'no #:getter respawn?) - ;; For the systemd restart values. Can be 'no (when respawn? is #f), - ;; 'on-success, 'on-failure, 'on-abnormal, 'on-watchdog, 'on-abort, or 'always - (respawn-systemd #:init-keyword #:respawn-systemd - #:init-value 'always - #:getter respawn-systemd) ;; The action to perform to start the service. This must be a ;; procedure and may take an arbitrary amount of arguments, but it ;; must be possible to call it without any argument. If the @@ -211,7 +211,11 @@ respawned, shows that it has been respawned more than TIMES in SECONDS." (stop-delay? #:init-keyword #:stop-delay? #:init-value #f) ;; The times of the last respawns, most recent first. - (last-respawns #:init-form '())) + (last-respawns #:init-form '()) + ;; if it is a socket unit file, it contains all services that depend on it + (before #:init-keyword #:before + #:init-value '() + #:getter before)) (define (service? obj) "Return true if OBJ is a service." @@ -331,7 +335,7 @@ wire." ;; Start the service itself. Asyncs are blocked so that if ;; the newly-started process dies immediately, the SIGCHLD ;; handler is invoked later, once we have set the 'running' - ;; field. + ;; field .
Re: GSoC 2018 Syntax and semantics of systemd units in the Shepherd - 1st update
I am currently working on the implementation of .socket unit files, signalfd for signal handling, and fibers. It is mostly done, I just have to fix some issues that are left. On 06/25/18 13:47, boskov...@gmail.com wrote: Hello, could you please send us an update on your project?
Re: GSoC 2018 Syntax and semantics of systemd units in the Shepherd - 1st update
Thank you a lot for your comments! I will make sure to make the changes that you suggested. As for match and things like car/cdr, I had issues with match and signal handling in the service file, which was why I changed it with a cond. As for the unit parser I also take the rest of the list via cdar because match in something like (x y rest ...) does not bind rest - I will probably have to use (x . (y . rest)) in the replacement. On 06/11/18 14:47, Ludovic Courtès wrote: Hello Ioannis! Thanks for the update! Ioannis Panagiotis Koutsidis skribis: As the 1st phase is coming to an end I decided to post my progress. I have implemented the unit file parsing as well as some of the basic entries supported by it, such as ExecStart, User, Group, Restart, etc. In addition, support for the systemd Restart values (on-success, on-failure, on-abnormal, and on-abort) was added to the Shepherd via the restart-systemd field in the class, letting services written in guile to also use that feature. Very nice! During the next phases I will focus on other common .service entries, .socket support, as well as thoroughly testing the code. Cool. Adding unit tests like those currently under tests/ is definitely something you should do—you probably already run tests manually anyway, so it’s mostly a matter of putting them in a file. For things like the unit file parser, you may find it more convenient to write the test in Scheme (currently all the tests are shell scripts.) That can easily be done by using the .scm file name extension for your test and then defining ‘SCM_LOG_COMPILER’ in Makefile.am. If unsure, you can look at how Guix itself does it, or just stop by on #guix or ask on the list for details. Some comments about the code: From a0a46ead5e43cd2672a08adb4c16919c377514c2 Mon Sep 17 00:00:00 2001 From: Ioannis Panagiotis Koutsidis Date: Sat, 9 Jun 2018 16:17:27 +0300 Subject: [PATCH] Initial systemd unit support Could you try to split it in several patches, where each patch represents a single “logical” change? By that I mean that you could have a first patch that modifies ‘restart’ and all in (shepherd service), possibly with extended tests to exercise the new functionality if appropriate. A second patch would add the unit file parser in (shepherd systemd) along with its unit test. For commit logs, please try to follow the ChangeLog convention: <https://www.gnu.org/prep/standards/html_node/Change-Logs.html>. You can look at ‘git log’ and basically try to mimic what’s been done before. Don’t lose your hair over commit logs though; it’s good to try to follow the conventions, but if you’re unsure or if you make mistakes, it’s not the end of the world. @@ -165,6 +166,11 @@ respawned, shows that it has been respawned more than TIMES in SECONDS." (respawn? #:init-keyword #:respawn? #:init-value #f #:getter respawn?) + ;; For the systemd restart values. Can be 'no (when respawn? is #f), + ;; 'on-success, 'on-failure, 'on-abnormal, 'on-watchdog, 'on-abort, or 'always + (respawn-systemd #:init-keyword #:respawn-systemd + #:init-value 'always + #:getter respawn-systemd) As briefly discussed on IRC, I think we should keep a single field for this. So perhaps ‘respawn?’ must simply be renamed to ‘respawn’ (no question mark), with a comment like above explaining what the possible values are. + (let* ([e (status:exit-val status)] + [t (status:term-sig status)] + [r (respawn-systemd serv)] Please avoid square brackets to remain consistent with the rest of the code. :-) + [clean (or (zero? e) +(equal? t SIGHUP) +(equal? t SIGINT) +(equal? t SIGTERM) +(equal? t SIGPIPE))]) Use ‘=’ rather than ‘equal?’ when we know we’re dealing with numbers. +(if (or (equal? r 'always) +(equal? r 'on-watchdog) ;; not implemented yet +(and (equal? r 'on-success) clean) +(and (equal? r 'on-abnormal) (not clean) (equal? e #f)) +(and (equal? r 'on-failure) (not clean)) +(and (equal? r 'on-abort)(equal? t SIGABRT))) Likewise, use ‘eq?’ for symbols. +++ b/modules/shepherd/systemd.scm @@ -0,0 +1,143 @@ +;; systemd.scm -- Systemd support +;; Copyright (C) 2018 Ioannis Panagiotis Koutsidis +;; +;; This file is part of the GNU Shepherd. +;; +;; The GNU Shepherd is free software; you can redistribute it and/or modify it +;; under the terms of the GNU General Public License as published by +;; the Free Software Foundation; either version 3 of the License, or (at +;; your option) any later version. +;; +;; The GNU Shepherd is distributed in the hope that it will be useful, but +;; WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;
GSoC 2018 Syntax and semantics of systemd units in the Shepherd - 1st update
Hi Guix! As the 1st phase is coming to an end I decided to post my progress. I have implemented the unit file parsing as well as some of the basic entries supported by it, such as ExecStart, User, Group, Restart, etc. In addition, support for the systemd Restart values (on-success, on-failure, on-abnormal, and on-abort) was added to the Shepherd via the restart-systemd field in the class, letting services written in guile to also use that feature. During the next phases I will focus on other common .service entries, .socket support, as well as thoroughly testing the code. From a0a46ead5e43cd2672a08adb4c16919c377514c2 Mon Sep 17 00:00:00 2001 From: Ioannis Panagiotis Koutsidis Date: Sat, 9 Jun 2018 16:17:27 +0300 Subject: [PATCH] Initial systemd unit support --- modules/shepherd/service.scm | 78 --- modules/shepherd/systemd.scm | 143 +++ 2 files changed, 194 insertions(+), 27 deletions(-) create mode 100644 modules/shepherd/systemd.scm diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm index 93d3779..5b0d72d 100644 --- a/modules/shepherd/service.scm +++ b/modules/shepherd/service.scm @@ -4,6 +4,7 @@ ;; Copyright (C) 2014 Alex Sassmannshausen ;; Copyright (C) 2016 Alex Kost ;; Copyright (C) 2018 Carlo Zancanaro +;; Copyright (C) 2018 Ioannis Panagiotis Koutsidis ;; ;; This file is part of the GNU Shepherd. ;; @@ -165,6 +166,11 @@ respawned, shows that it has been respawned more than TIMES in SECONDS." (respawn? #:init-keyword #:respawn? #:init-value #f #:getter respawn?) + ;; For the systemd restart values. Can be 'no (when respawn? is #f), + ;; 'on-success, 'on-failure, 'on-abnormal, 'on-watchdog, 'on-abort, or 'always + (respawn-systemd #:init-keyword #:respawn-systemd + #:init-value 'always + #:getter respawn-systemd) ;; The action to perform to start the service. This must be a ;; procedure and may take an arbitrary amount of arguments, but it ;; must be possible to call it without any argument. If the @@ -270,7 +276,7 @@ wire." (define-method (running? (obj )) (and (slot-ref obj 'running) #t)) -;; Return a list of all actions implemented by OBJ. +;; Return a list of all actions implemented by OBJ. (define-method (action-list (obj )) (map action-name (slot-ref obj 'actions))) @@ -886,9 +892,12 @@ start." ;; Produce a destructor that sends SIGNAL to the process with the pid ;; given as argument, where SIGNAL defaults to `SIGTERM'. (define make-kill-destructor - (lambda* (#:optional (signal SIGTERM)) + (lambda* (#:optional (signal SIGTERM) + (timeout #f)) (lambda (pid . args) (kill pid signal) + ;; TODO: Make sure that the process has actually stopped by timeout. + ;; If it has not, send a SIGKILL #f))) ;; Produce a constructor that executes a command. @@ -996,7 +1005,7 @@ otherwise by updating its state." ((0 . _) ;; Nothing left to wait for. #t) - ((pid . _) + ((pid . status) (let ((serv (find-service (lambda (serv) (and (enabled? serv) (match (slot-ref serv 'running) @@ -1007,13 +1016,13 @@ otherwise by updating its state." ;; SERV can be #f for instance when this code runs just after a ;; service's 'stop' method killed its process and completed. (when serv - (respawn-service serv)) + (respawn-service serv status)) ;; As noted in libc's manual (info "(libc) Process Completion"), ;; loop so we don't miss any terminated child process. (loop)) -(define (respawn-service serv) +(define (respawn-service serv status) "Respawn a service that has stopped running unexpectedly. If we have attempted to respawn the service a number of times already and it keeps dying, then disable it." @@ -1022,22 +1031,37 @@ then disable it." (not (respawn-limit-hit? (slot-ref serv 'last-respawns) (car respawn-limit) (cdr respawn-limit - (if (not (slot-ref serv 'waiting-for-termination?)) - (begin -;; Everything is okay, start it. -(local-output "Respawning ~a." - (canonical-name serv)) -(slot-set! serv 'last-respawns - (cons (current-time) - (slot-ref serv 'last-respawns))) -(start serv)) - ;; We have just been waiting for the - ;; termination. The `running' slot has already - ;; been set to `#f' by `stop'. - (begin -(local-output "Service ~a terminated." - (canonical-name serv)) -(slot-set! serv 'w
Re: GSoC project ``Syntax and semantics of systemd units in the Shepherd''
I installed the iso on my qemu without any issues. I was not aware about the script, thanks for letting me know. Thankfully the installation was easy enough not to require it. Thank you for your help! (resending the mail because I pressed "reply" instead of "reply all", sorry) On 09/05/18 10:50, rek...@elephly.net wrote: > > Hi Ioannis, > > Ludovic Courtès <l...@gnu.org> writes: > >> Hello Ioannis, >> >> Ioannis Panagiotis Koutsidis <ixk...@student.bham.ac.uk> skribis: >> >>> I am Ioannis Panagiotis Koutsidis (Rukako on #guix) and I was selected >>> for the ``Syntax and semantics of systemd units in the Shepherd'' GSoC >>> 2018 project. >> >> Welcome on board! >> >> Now’s a good time to familiarize yourself with the Shepherd, presumably >> by installing GuixSD. Then please let us know about your first >> impressions and how you feel about this GSoC project. > > Did you get around to installing GuixSD yet? Playing with the Shepherd > really makes most sense on GuixSD. You can also install Guix (note that > we have an installer script, which makes this very easy) and then use > Guix to build virtual machines with GuixSD. > > -- > Ricardo >
GSoC project ``Syntax and semantics of systemd units in the Shepherd''
Dear all, I am Ioannis Panagiotis Koutsidis (Rukako on #guix) and I was selected for the ``Syntax and semantics of systemd units in the Shepherd'' GSoC 2018 project. I am looking forward to working with you, please feel free to email me or beep me at IRC at any time. 0x0C8877F4.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
