juli pushed a commit to branch wip-goblinsify in repository shepherd. commit 10ff21fa171b11a39f8bf6bebe6019cb2b4294c5 Author: Juliana Sims <j...@incana.org> AuthorDate: Wed Nov 27 17:30:45 2024 -0500
WIP: doc: Document new Goblins interface. * doc/shepherd.texi: Document new Goblins interface, move old interface to legacy interface docs. --- doc/shepherd.texi | 501 ++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 396 insertions(+), 105 deletions(-) diff --git a/doc/shepherd.texi b/doc/shepherd.texi index 1aa5088..dbd24ba 100644 --- a/doc/shepherd.texi +++ b/doc/shepherd.texi @@ -15,6 +15,7 @@ Copyright @copyright{} 2020 Brice Waegeneire@* Copyright @copyright{} 2020 Oleg Pykhalov@* Copyright @copyright{} 2020, 2023 Jan (janneke) Nieuwenhuizen@* Copyright @copyright{} 2024 Jakob Kirsch@* +Copyright @copyright{} 2024 Juliana Sims@* Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or @@ -596,13 +597,14 @@ not specified, @file{@var{localstatedir}/run/shepherd/socket} is taken. @cindex service @tindex <service> -The @dfn{service} is obviously a very important concept of the Shepherd. -On the Guile level, a service is represented as a record of type -@code{<service>}. Each service has a number of properties that you -specify when you create it: how to start it, how to stop it, whether to -respawn it when it stops prematurely, and so on. At run time, each -service has associated @dfn{state}: whether it is running or stopped, -what PID or other value is associated with it, and so on. +The @dfn{service} is a very important concept of the Shepherd. On the +Guile level, a service is represented as a +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Objects.html, +Goblins object}. When creating a service, one specifies a number of +properties: how to start it, how to stop it, whether to respawn it when +it stops prematurely, and so on. At run time, each service has +associated @dfn{state}: whether it is running or stopped, what PID or +other value is associated with it, and so on. This section explains how to define services and how to query their configuration and state using the Shepherd's programming interfaces. @@ -623,10 +625,11 @@ at the REPL (@pxref{REPL Service}). @end itemize These procedures may not be used in Guile processes other than -@command{shepherd} itself. +@command{shepherd} itself (@pxref{A Note on Goblins} for more information). @end quotation @menu +* A Note on Goblins:: Using the Goblins interface. * Defining Services:: Defining services. * Service Registry:: Mapping service names to records. * Interacting with Services:: Accessing service state. @@ -634,13 +637,152 @@ These procedures may not be used in Guile processes other than stopping services. * Timers:: Timed services. * The root Service:: The service that comes first. -* Legacy GOOPS Interface:: Former interface and how to migrate. +* Legacy Interfaces:: Former interfaces and how to migrate. * Service Examples:: Examples that show how services are used. * Managing User Services:: Running the Shepherd as a user. @end menu @c @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@node A Note on Goblins +@section A Note on Goblins +@cindex Goblins + +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/index.html, +Goblins} is a library implementing the object-capability security (ocaps) +paradigm which enables the Shepherd to operate as a distributed service +manager. While the Goblins manual or the +@url{https://files.spritely.institute/papers/spritely-core.html, Heart +of Spritely} paper are the best places to learn more about Goblins and +ocaps, there are a few things Shepherd users need to know to best take +advantage of its powerful features. + +A Goblins object is a Scheme procedure constructed with some special +state. The definition of a Goblins object usually looks something like + +@lisp +(define (^obj bcom . args) + (lambda (arg) + ;; some logic...)) +@end lisp + +You don't need to understand everything going on here; you just need to +know that when you invoke a Goblins object, it is the inner lambda which +is actually receiving arguments, not the outer, constructor lambda; and +similarly, code like + +@lisp +(define some-obj + (with-vat some-vat + (spawn ^obj))) +@end lisp + +binds the inner lambda to @code{some-obj}. The Shepherd handles all +this defining and constructing for you, but it's helpful to understand +what happens internally. + +@quotation Note +Goblins objects can generally be thought of as analogous to actors from +the @url{https://en.wikipedia.org/wiki/Actor_model, actor model}, and +indeed, they are sometimes discussed as actors which accept messages to +better facilitate the conceptual model of distributed computation. The +object-capability paradigm in general can be seen as a restricted form +of the actor model. Rather than discussing invoking objects, we may +just as well discuss messaging actors. Here, we generally hew to object +terminology, but the relationship between ocaps and actors is discussed +more fully in the +@url{https://files.spritely.institute/papers/spritely-core.html, Heart +of Spritely} paper. +@end quotation + +This last example introduces a vat, which warrants explanation. Goblins +objects may only be created and manipulated in the context of +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Vats.html, +vats}, which are essentially event loops and execution contexts for +objects. Vats are what enable transactionality and manage remote +messages on behalf of the objects it holds. The Shepherd handles +creating and managing vats on behalf of users, as well as spawning +objects properly in the correct vat context. From a Shepherd user +perspective, the most significant impact of vats is that objects may +only be invoked with special operators: @code{$}, @code{<-}, +@code{<-np}, and @code{<-np-extern}. This latter operator is a special +case because it can be used outside of a vat; we'll return to that +later. + +@code{$} is the ``synchronous'' or ``near'' invoker (see +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Synchronous-calls.html, +``Synchronous calls''} in the Goblins manual). It can only +operate on objects in the vat where it is used. Because it can never be +guaranteed that a given service is ``near'' unless one is in the context +of the service itself, users should generally avoid @code{$} unless they +are sure of what they are doing. + +@c TODO: When there is a Goblins manual page on promises, we should link +@c to it instead of discussing JS promises. +Generally, users will want to use @code{<-} to operate on services when +they expect a return value. @code{<-} can operate on any object as long +as it's used inside a vat. Unlike the other operators discussed here, +@code{<-} returns a promise. Goblins promises are analogous to +@url{https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise, +JavaScript promises}, but more powerful because they support +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Promise-pipelining.html, +promise pipelining}. Promises represent an asynchronous computation +which has not yet been completed (``resolved'') and which may fail +(``break''). + +In order to operate on the values to which promises resolve, one must +use @code{on}, which takes the promise and a callback procedure it will +apply to the promise's resolved value. @code{on} can also take plain +values which it will simply pass to the callback. The @code{#:promise?} +keyword argument must be passed a @code{#t} if the result of the +callback's computation needs to be returned, in which case it will be +returned in a promise. To avoid so-called ``callback hell'', Goblins +provides a set of +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Let_002dOn.html, +let-on} and +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Joiners.html, +joiner} macros. + +@code{<-np} is like @code{<-} except that it doesn't return a value at +all. @code{<-np-extern} is like @code{<-np} except that it may be used +outside of a vat. @code{<-np-extern} should never be necessary in +Shepherd code. The Goblins manual section +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Asynchronous-calls.html, +``Asynchronous calls''} has more information on all these operators as +well as @code{on}. + +It's easiest to understand what this means with an example. We will +ignore vat creation and manipulation since it isn't relevant to Shepherd +user code. Let's pretend we are operating in a context where we need to +inform a remote service that our service is terminating and write its +response to a log. We might write the following termination handler: + +@c TODO: This example should use whatever remote interface we introduce. +@lisp +;; In this example, "self" is the actor terminating. +(define (termination-handler self process status) + ;; Something went wrong. + (unless (zero? status) + (on (<- ($ self 'remote-friend) 'inform-error status) + (lambda (response) + (let ((logger ($ self 'logger))) + (put-message (format #f "Remote object said ~a" response)) + (put-message ($ self 'logger) 'terminate))))) + ;; Try to respawn. + ($ self 'respawn)) +@end lisp + +This example introduces another concept which is not inherently related +to but greatly increases the power of objects: +@url{https://files.spritely.institute/docs/guile-goblins/0.14.0/Methods.html, +the @code{methods} macro}. @code{methods} allows a single object to be +invoked for a variety of different behaviors; or, as a more useful +model, allows an actor to respond to multiple messages. Objects in the +Shepherd generally use @code{methods} unless noted others, and the +methods themselves are documented as appropriate. As the example above +shows, a specific method is in invoke by passing a symbol of its name as +the first argument to the object. + @node Defining Services @section Defining Services @@ -677,7 +819,7 @@ procedure and its optional keyword arguments. @cindex canonical name of services Return a new service with the given @var{provision}, a list of symbols denoting what the service provides. The first symbol in the list is the -@dfn{canonical name} of the service, thus it must be unique. +@dfn{canonical name} of the service and must be unique. The meaning of keyword arguments is as follows: @@ -797,7 +939,7 @@ a minute. Failing to do that, the service would remain in ``stopping'' state and users would be unable to stop it. @end quotation -@xref{Service De- and Constructors}, for info on common service +@xref{Service De- and Constructors}, for information on common service destructors. @item #:termination-handler @@ -819,9 +961,9 @@ running. A typical example for this is the @code{restart} action. The @end table @end deffn -A special service that every other service implicitly depends on is the -@code{root} (also known as @code{shepherd}) service. @xref{The root -Service}, for more information. +Every other service implicitly depends on the special @code{root} (also +known as @code{shepherd}) service. @xref{The root Service}, for more +information. @cindex graph of services Services and their dependencies form a @dfn{graph}. At the @@ -849,45 +991,47 @@ procedure that will be called to perform the action. A @var{proc} has one argument, which will be the running value of the service. @end defmac -Naturally, the @code{(shepherd service)} provides procedures to access -this information for a given service object: +@c TODO: use extend-methods for actions? +Actions are not methods, but their interface is similar. -@deffn {Procedure} service-provision @var{service} -Return the symbols provided by @var{service}. +Naturally, service objects have methods to access this information: + +@deffn {Method} provision +Return the symbols provided by this service. @end deffn -@deffn {Procedure} service-canonical-name @var{service} -Return the @dfn{canonical name} of @var{service}, which is the first -element of the list returned by @code{service-provision}. +@deffn {Method} canonical-name +Return the @dfn{canonical name} of this service, which is the first +element of the list returned by @code{provision}. @end deffn -@deffn {Procedure} service-requirement @var{service} -Return the list of services required by @var{service} as a list of -symbols. +@deffn {Method} requirement +Return a list of symbols identifying the services required by this +service. @end deffn -@deffn {Procedure} one-shot-service? @var{service} -@deffnx {Procedure} transient-service? @var{service} -Return true if @var{service} is a one-shot/transient service. +@deffn {Method} one-shot? +@deffnx {Method} transient? +Return @code{#t} if this service is a one-shot/transient service. @end deffn -@deffn {Procedure} respawn-service? @var{service} -Return true if @var{service} is meant to be respawned if its associated -process terminates prematurely. +@deffn {Method} respawn? +Return @code{#t} if this service's process is meant to be respawned if +it terminates prematurely. @end deffn -@deffn {Procedure} service-respawn-delay @var{service} -Return the respawn delay of @var{service}, in seconds (an integer or a +@deffn {Method} respawn-delay +Return the respawn delay of this service in seconds (an integer or a fraction or inexact number). See @code{#:respawn-delay} above. @end deffn -@deffn {Procedure} service-respawn-limit @var{service} -Return the respawn limit of @var{service}, expressed as a pair---see +@deffn {Method} respawn-limit +Return the respawn limit of this service, expressed as a pair---see @code{#:respawn-limit} above. @end deffn -@deffn {Procedure} service-documentation @var{service} -Return the documentation (a string) of @var{service}. +@deffn {Method} documentation +Return this service's documentation, a string. @end deffn @@ -944,111 +1088,100 @@ Return the running service that provides @var{name}, or false if none. @section Interacting with Services What we have seen so far is the interface to @emph{define} a service and -to access it (@pxref{Defining Services}). The procedures below, also -exported by the @code{(shepherd service)} module, let you modify and -access the state of a service. You may use them in your configuration -file, for instance to start some or all of the services you defined -(@pxref{Service Examples}). - -Under the hood, each service record has an associated @dfn{fiber} -(really: an actor) that encapsulates its state and serves user -requests---a fiber is a lightweight execution thread (@pxref{Service -Internals}). - -The procedures below let you change the state of a service. - -@deffn {Procedure} start-service @var{service} . @var{args} -Start @var{service} and its dependencies, passing @var{args} to its -@code{start} method. Return its running value, @code{#f} on failure. +to access it (@pxref{Defining Services}). The methods below let you +modify and access the state of a service. You may wish to use them in +your configuration file, for instance to start some or all of the +services you defined (@pxref{Service Examples}). + +@c TODO: update Service Internals +@c Under the hood, each service record has an associated @dfn{fiber} +@c (really: an actor) that encapsulates its state and serves user +@c requests---a fiber is a lightweight execution thread (@pxref{Service +@c Internals}). + +The methods below let you change the state of a service. + +@deffn {Method} start . @var{args} +Start this services and its dependencies, passing @var{args} to its +@code{start} procedure. Return a promise resolving to its running +value, @code{#f} on failure. @end deffn -@deffn {Procedure} stop-service @var{service} . @var{args} -Stop @var{service} and any service that depends on it. Return the list of -services that have been stopped (including transitive dependent services). - -If @var{service} is not running, print a warning and return its canonical name -in a list. -@end deffn +@deffn {Method} stop . @var{args} +Stop this service and any service that depends on it. Return a promise +resolving to the list of services that have been stopped (including +transitive dependent services). -@deffn {Procedure} perform-service-action @var{service} @var{the-action} . @var{args} -Perform @var{the-action} (a symbol such as @code{'restart} or @code{'status}) -on @var{service}, passing it @var{args}. The meaning of @var{args} depends on -the action. +If this service is not running, print a warning and return its canonical +name in a list. @end deffn -The @code{start-in-the-background} procedure, described below, is -provided for your convenience: it makes it easy to start a set of -services right from your configuration file, while letting -@command{shepherd} run in the background. - -@deffn {Procedure} start-in-the-background @var{services} -Start the services named by @var{services}, a list of symbols, in the -background. In other words, this procedure returns immediately without -waiting until all of @var{services} have been started. - -This procedure can be useful in a configuration file because it lets you -interact right away with shepherd using the @command{herd} command. +@deffn {Method} perform-action @var{the-action} . @var{args} +Perform @var{the-action} (a symbol such as @code{'restart} or +@code{'status}) on this service, passing it @var{args}. The meaning of +@var{args} depends on the action. @end deffn -The following procedures let you query the current state of a service. +The following methods let you query the current state of a service. -@deffn {Procedure} service-running? @var{service} -@deffnx {Procedure} service-stopped? @var{service} -@deffnx {Procedure} service-enabled? @var{service} -Return true if @var{service} is currently running/stopped/enabled, false -otherwise. +@deffn {Method} running? +@deffnx {Method} stopped? +@deffnx {Method} enabled? +Return @code{#t} if this service is currently running/stopped/enabled, +else @code{#f}. @end deffn -@deffn {Procedure} service-status @var{service} -Return the status of @var{service} as a symbol, one of: @code{'stopped}, +@deffn {Method} status +Return the status of this service as a symbol, one of: @code{'stopped}, @code{'starting}, @code{'running}, or @code{'stopping}. @end deffn -@deffn {Procedure} service-running-value @var{service} -Return the current ``running value'' of @var{service}---a Scheme value +@deffn {Method} running-value +Return the current ``running value'' of this service---a Scheme value associated with it. It is @code{#f} when the service is stopped; otherwise, it is a truth value, such as an integer denoting a PID (@pxref{Service De- and Constructors}). @end deffn -@deffn {Procedure} service-status-changes @var{service} +@deffn {Method} status-changes Return the list of symbol/timestamp pairs representing recent state -changes for @var{service}. +changes for this service. @end deffn -@deffn {Procedure} service-startup-failures @var{service} -@deffnx {Procedure} service-respawn-times @var{service} -Return the list of startup failure times or respawn times of -@var{service}. +@deffn {Method} startup-failures +@deffnx {Method} respawn-times +Return the list of startup failure times or respawn times of this +service. @end deffn -@deffn {Procedure} service-process-exit-statuses @var{services} -Return the list of last exit statuses of @var{service}'s main process +@deffn {Method} exit-statuses +Return the list of last exit statuses of this service's main process (most recent first). @end deffn @cindex replacement, or a service -@deffn {Procedure} service-replacement @var{service} -Return the @dfn{replacement} of @var{service}, or @code{#f} if there is +@deffn {Method} replacement +Return the @dfn{replacement} of this service, or @code{#f} if there is none. -The replacement is the service that will replace @var{service} when it -is eventually stopped. +The replacement is the service that will replace this one when it is +eventually stopped. @end deffn -@deffn {Procedure} service-recent-messages @var{service} -Return a list of messages recently logged by @var{service}---typically +@deffn {Method} recent-messages +Return a list of messages recently logged by this service---typically lines written by a daemon on standard output. Each element of the list is a timestamp/string pair where the timestamp is the number of seconds since January 1st, 1970 (an integer). @end deffn -@deffn {Procedure} service-log-file @var{service} -Return the file where messages by @var{service} are logged, or @code{#f} -if there is no such file, for instance because @var{service}'s output is -logged by some mechanism not under shepherd's control. +@deffn {Method} log-file +Return the file where messages by this service are logged, or @code{#f} +if there is no such file, for instance because this service's output is +logged by some mechanism not under the Shepherd's control. @end deffn +@c TODO: update this @xref{Service Internals}, if you're curious about the nitty-gritty details! @@ -1627,10 +1760,168 @@ invoked by the @command{reboot -k} command. @xref{Invoking reboot}. @c @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +@node Legacy Interfaces +@section Legacy Interfaces + +@cindex former interface +The Shepherd's long history means that it has gone through multiple +iterations. Over the course of its existence, two distinct service +interfaces have come and gone. Those interfaces are preserved for +backwards compatibility, and they are documented here. Users are +advised to update code to directly use the Goblins interface documented +in the rest of this manual. + +@menu +* Legacy Procedural Interface:: Procedure-based service interface. +* Legacy GOOPS Interface:: GOOPS-based service interface. +@end menu + +@node Legacy Procedural Interface +@subsection Legacy Procedural Interface + +Before the port to Goblins, the Shepherd exposed standard procedures to +access and manipulate services. These procedures are preserved for +backwards compatibility and wrap their method equivalents. They are +exported by @code{(shepherd service)}. + +First, these procedures allow access to a service's internal state. + +@deffn {Procedure} service-provision @var{service} +Return the symbols provided by @var{service}. +@end deffn + +@deffn {Procedure} service-canonical-name @var{service} +Return the @dfn{canonical name} of @var{service}, which is the first +element of the list returned by @code{service-provision}. +@end deffn + +@deffn {Procedure} service-requirement @var{service} +Return the list of services required by @var{service} as a list of +symbols. +@end deffn + +@deffn {Procedure} one-shot-service? @var{service} +@deffnx {Procedure} transient-service? @var{service} +Return true if @var{service} is a one-shot/transient service. +@end deffn + +@deffn {Procedure} respawn-service? @var{service} +Return true if @var{service} is meant to be respawned if its associated +process terminates prematurely. +@end deffn + +@deffn {Procedure} service-respawn-delay @var{service} +Return the respawn delay of @var{service}, in seconds (an integer or a +fraction or inexact number). See @code{#:respawn-delay} above. +@end deffn + +@deffn {Procedure} service-respawn-limit @var{service} +Return the respawn limit of @var{service}, expressed as a pair---see +@code{#:respawn-limit} above. +@end deffn + +@deffn {Procedure} service-documentation @var{service} +Return the documentation (a string) of @var{service}. +@end deffn + +The following procedures update state. Like the procedures above, they +simply wrap method invocations. + +@deffn {Procedure} start-service @var{service} . @var{args} +Start @var{service} and its dependencies, passing @var{args} to its +@code{start} method. Return its running value, @code{#f} on failure. +@end deffn + +@deffn {Procedure} stop-service @var{service} . @var{args} +Stop @var{service} and any service that depends on it. Return the list of +services that have been stopped (including transitive dependent services). + +If @var{service} is not running, print a warning and return its canonical name +in a list. +@end deffn + +@deffn {Procedure} perform-service-action @var{service} @var{the-action} . @var{args} +Perform @var{the-action} (a symbol such as @code{'restart} or @code{'status}) +on @var{service}, passing it @var{args}. The meaning of @var{args} depends on +the action. +@end deffn + +The @code{start-in-the-background} procedure, described below, doesn't +actually do much anymore because Goblins starts all services +asynchronously by default. + +@deffn {Procedure} start-in-the-background @var{services} +Start the services named by @var{services}, a list of symbols, in the +background. In other words, this procedure returns immediately without +waiting until all of @var{services} have been started. + +This procedure can be useful in a configuration file because it lets you +interact right away with shepherd using the @command{herd} command. +@end deffn + +The following procedures query the current state of a service. + +@deffn {Procedure} service-running? @var{service} +@deffnx {Procedure} service-stopped? @var{service} +@deffnx {Procedure} service-enabled? @var{service} +Return true if @var{service} is currently running/stopped/enabled, false +otherwise. +@end deffn + +@deffn {Procedure} service-status @var{service} +Return the status of @var{service} as a symbol, one of: @code{'stopped}, +@code{'starting}, @code{'running}, or @code{'stopping}. +@end deffn + +@deffn {Procedure} service-running-value @var{service} +Return the current ``running value'' of @var{service}---a Scheme value +associated with it. It is @code{#f} when the service is stopped; +otherwise, it is a truth value, such as an integer denoting a PID +(@pxref{Service De- and Constructors}). +@end deffn + +@deffn {Procedure} service-status-changes @var{service} +Return the list of symbol/timestamp pairs representing recent state +changes for @var{service}. +@end deffn + +@deffn {Procedure} service-startup-failures @var{service} +@deffnx {Procedure} service-respawn-times @var{service} +Return the list of startup failure times or respawn times of +@var{service}. +@end deffn + +@deffn {Procedure} service-process-exit-statuses @var{services} +Return the list of last exit statuses of @var{service}'s main process +(most recent first). +@end deffn + +@cindex replacement, or a service +@deffn {Procedure} service-replacement @var{service} +Return the @dfn{replacement} of @var{service}, or @code{#f} if there is +none. + +The replacement is the service that will replace @var{service} when it +is eventually stopped. +@end deffn + +@deffn {Procedure} service-recent-messages @var{service} +Return a list of messages recently logged by @var{service}---typically +lines written by a daemon on standard output. Each element of the list +is a timestamp/string pair where the timestamp is the number of seconds +since January 1st, 1970 (an integer). +@end deffn + +@deffn {Procedure} service-log-file @var{service} +Return the file where messages by @var{service} are logged, or @code{#f} +if there is no such file, for instance because @var{service}'s output is +logged by some mechanism not under shepherd's control. +@end deffn + @node Legacy GOOPS Interface -@section Legacy GOOPS Interface +@subsection Legacy GOOPS Interface -@cindex GOOPS, former interface +@cindex GOOPS From its inception in 2002 with negative version numbers (really!)@: up to version 0.9.x included, the Shepherd's service interface used GOOPS, Guile's object-oriented programming system (@pxref{GOOPS,,, guile, GNU