Re: [systemd-devel] Creating units using D-Bus

[Jóhann B. Guðmundsson]

On 06/10/2015 03:02 PM, Stephen Gallagher wrote:

You make some good points Jóhann. It probably does make sense to focus
(at least at first) on managing timer units shipped alongside services
as opposed to trying to develop a UI for managing arbitrary timer
units. I'll discuss this with some of my colleagues.

That said, there still may be some value in enabling the creation of
new timer units for these purposes. (For example, there may be timers
that are only useful if two normally-unrelated services are installed,
so neither one would be likely to ship such a timer in their package,
even disabled). But it's hard to say whether that's worth designing for
or solving by shipping additional packages with helper units.


Now that I see that you have started to understand what I tried to 
convey when running that cron to timer migration but failed miserably in 
doings so, going to say that value is an illusion Ithrow in another ( 
real life ) example to complex matters further.


Let's use this drupal related cron job as an example "|0 * * * * wget -O 
- -q -t 1 http://www.example.com/cron.php";| that many will be familiar 
with which you would migrate to native systemd timers.


Shipping this with drupal cannot be done since you cannot bind that to a 
specific web server ( the installed server might be apache, might be 
lighthttp or might be nginx etc ) not binding it to the web server will 
keep this timer unit running with all it downside once the administrator 
shuts down the webserver ( as happens currently with cron ).


What I'm trying to emphasize here there exist only one to one mapping 
with each service ( or target )  and timer unit and those two things 
need to be bound together.


With my former cron to timer feature and serverWG hat on, what I would 
propose and do which is quite the opposite from what you propose here  
is have all components ship their cron job in a separated sub-package ( 
those are mostly crap anyway ) then design the cockpit UI in such manner 
that dissociates cron job and uses generic term like for example 
"scheduled task" which covers both solution and whatever the future 
might bring and offers the user to create an arbitrary task ( which 
would use cron and have screen(s) tailored to that and create and 
associate task to an service ( with screen(s) tailored to that which 
would use timers )


JBG
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Stephen Gallagher]

On Wed, 2015-06-10 at 13:21 +, 
> 
> On 06/10/2015 12:30 PM, Stephen Gallagher wrote:
> > 
> > A good overview of what we're aiming to accomplish can be found 
> > here: h
> > ttps://github.com/cockpit-project/cockpit/wiki/Feature:-Systemd
> > -timers#stories
> > 
> > Though at the end of the day, it might be fair to say that systemd
> > timers are cool and very capable and we really just want to have a
> > decent UI for people to manipulate them. Cockpit seems like a good
> > place since it already does management of a lot of other systemd
> > functionality for Fedora Server.
> > 
> > If you have specific questions or concerns with that design, we'd 
> > love
> > to hear them.
> 
> Timers are not suited to all administrated tasks ( only the ones that 
> 
> relate to bootup,hardware and daemons ) and you seem to have fallen 
> into 
> the trap of taking the cron concept and start applying it to timers ( 
> as 
> opposed to simply continue to use cron )  when it's not the same 
> thing, 
> ( to me it is the same as you take the run level concept and apply it 
> to 
> systemd boot targets and think that it's the same thing) and what's 
> describe there on that story page is something that cron is better 
> suited to handle ( and probably much easier to implement as well ).
> 
> Now as I mentioned before you need to bind every timer unit that 
> get's 
> created with the relevant service running on the host. so when a 
> daemon 
> is started or stopped or even uninstalled ( think for example 
> postgresql 
> + postgresql backup timer service ) , or device appears then 
> disappears 
> the relevant timer unit is stopped in the process however timer units 
> 
> are not suitable for end user tasks ( which are the only thing that 
> is 
> listed there on that story page ).
> 
> As I mentioned on numerous occasion when I was driving the cron to 
> timer 
> migration effort in Fedora that timers and cron are solution that 
> complement each other not replace each other, which why the cron jobs 
> to 
> be migrated to native times units was limited to roughly half of the 
> total of shipped cron jobs in the distribution.
> 
> I guess the best way for you to realize this is to set up a batch 
> server 
> with 20, 50 or hundreds of timer units and maintain it and you will 
> quickly understand timers shortcomings and why they should not be 
> used 
> for end users usecase like the ones on that story page.
> 
> You most certainly can make administrators life more hell by using 
> timers for all usecases, just like you can drive on the wrong side of 
> 
> the road but you shouldn't.
> 

You make some good points Jóhann. It probably does make sense to focus
(at least at first) on managing timer units shipped alongside services
as opposed to trying to develop a UI for managing arbitrary timer
units. I'll discuss this with some of my colleagues.

That said, there still may be some value in enabling the creation of
new timer units for these purposes. (For example, there may be timers
that are only useful if two normally-unrelated services are installed,
so neither one would be likely to ship such a timer in their package,
even disabled). But it's hard to say whether that's worth designing for
or solving by shipping additional packages with helper units.

signature.asc
Description: This is a digitally signed message part
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Jóhann B. Guðmundsson]

On 06/10/2015 12:30 PM, Stephen Gallagher wrote:


A good overview of what we're aiming to accomplish can be found here: h
ttps://github.com/cockpit-project/cockpit/wiki/Feature:-Systemd
-timers#stories

Though at the end of the day, it might be fair to say that systemd
timers are cool and very capable and we really just want to have a
decent UI for people to manipulate them. Cockpit seems like a good
place since it already does management of a lot of other systemd
functionality for Fedora Server.

If you have specific questions or concerns with that design, we'd love
to hear them.


Timers are not suited to all administrated tasks ( only the ones that 
relate to bootup,hardware and daemons ) and you seem to have fallen into 
the trap of taking the cron concept and start applying it to timers ( as 
opposed to simply continue to use cron )  when it's not the same thing, 
( to me it is the same as you take the run level concept and apply it to 
systemd boot targets and think that it's the same thing) and what's 
describe there on that story page is something that cron is better 
suited to handle ( and probably much easier to implement as well ).


Now as I mentioned before you need to bind every timer unit that get's 
created with the relevant service running on the host. so when a daemon 
is started or stopped or even uninstalled ( think for example postgresql 
+ postgresql backup timer service ) , or device appears then disappears 
the relevant timer unit is stopped in the process however timer units 
are not suitable for end user tasks ( which are the only thing that is 
listed there on that story page ).


As I mentioned on numerous occasion when I was driving the cron to timer 
migration effort in Fedora that timers and cron are solution that 
complement each other not replace each other, which why the cron jobs to 
be migrated to native times units was limited to roughly half of the 
total of shipped cron jobs in the distribution.


I guess the best way for you to realize this is to set up a batch server 
with 20, 50 or hundreds of timer units and maintain it and you will 
quickly understand timers shortcomings and why they should not be used 
for end users usecase like the ones on that story page.


You most certainly can make administrators life more hell by using 
timers for all usecases, just like you can drive on the wrong side of 
the road but you shouldn't.


JBG
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Lennart Poettering]

B1;4002;0cOn Wed, 10.06.15 13:25, Simon McVittie 
(simon.mcvit...@collabora.co.uk) wrote:

> On 10/06/15 11:46, "Jóhann B. Guðmundsson" wrote:
> > I hate to say this since I'm against litter unit files across the entire
> > filesystem like infective disease and administrator then have to run
> > around trying to chase them down but is it not better to store this
> > under /srv somewhere [1], not etc, so it wont conflicts with "Stateless
> > Systems, Factory Reset, Golden Master" Systems?
> 
> To me this seems like (programmatically-assisted) configuration by a
> sysadmin, just like when NetworkManager writes configured networks into
> /etc/NetworkManager, or when `systemctl (en|dis)able` manipulates
> symlinks in /etc? "You reset to factory configuration by deleting /etc"
> seems entirely consistent with "tool-generated configuration goes in
> /etc" to me.
> 
> /srv is certainly not the place: if "no program should rely on a
> specific subdirectory structure of /srv" (that's a direct quote from the
> FHS) then systemd can't rely on a particular directory structure being a
> valid place to put its own files. You might say that systemd units could
> go in /srv/systemd, but the FHS' point is that it would be, for
> instance, entirely valid for the freedesktop.org administrators to use
> /srv/systemd as the directory for systemd.freedesktop.org's web content.
> 
> This makes /srv pretty useless from a distribution point of view, but
> useful for sysadmins - Debian's take on that directory is essentially
> "don't put anything in /srv out-of-the-box, so that the sysadmin can use
> it however they want to" (a lot like /opt).

Yes, I agree.

Also, again, as mentioned: we need to place this in a location that is
both writable (at least sometimes) and available from the moment we
transition from the initrd into the host. Otherwise we'd have to
reload configuration half-way after the other file systems become
available, and requeue a transaction which I'd really like to avoid to
have as a default. 

/var, /srv, /home do not qualify hence. /etc does. Hence it needs to
be /etc.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Stephen Gallagher]

On Wed, 2015-06-10 at 10:46 +, 
> 
> On 06/09/2015 10:21 PM, Lennart Poettering wrote:
> > On Tue, 09.06.15 19:33, Jakub Skořepa (ja...@skorepa.info) wrote:
> > 
> > > Hello,
> > > My name is Jakub Skořepa and I'm working on Cockpit UI for 
> > > Systemd
> > > Timers.
> > > 
> > > For that I need to create and modify systemd unit files. Cockpit 
> > > uses D
> > > -Bus for everything so I need D-Bus API for that. I think that it 
> > > would
> > > be beneficial if systemd was able to create unit files on-the-fly 
> > > using
> > > through D-Bus method call.
> > You can do that already, in the concept of "transient" units. 
> > However,
> > these units are not persistent, they are stored in /run and go away 
> > on
> > reboots. Google for the StartTransientUnit() bus call for details.
> > 
> > I think it would be a good idea to also allow to create persisent
> > units in a similar way eventually. However, that's not as obvious 
> > and
> > easy as it sounds, it starts from the question where to store those
> > units. Cron-like semantics would suggest somewhere below /var, but
> > that means they aren't available at early boot, and we'd have to
> > reload the configuration and requeue all jobs when /var appears. 
> > Which
> > means we'd have to put them in /etc instead, which however is
> > suboptimal for many usecases
> > 
> > I think I'd be willing to merge a patch that adds adds a new bus 
> > call
> > CreatePersistentUnit() that works like StartTransientUnit() except
> > that it only creates but not starts a unit, and that it stores the
> > unit in /etc. (Note: the only reason StartTransientUnit() not only
> > creates but also starts a unit is because the unit would otherwise 
> > be
> > GC'ed away immediately and not be reconstructable after that...)
> > 
> > 
> 
> I hate to say this since I'm against litter unit files across the 
> entire 
> filesystem like infective disease and administrator then have to run 
> around trying to chase them down but is it not better to store this 
> under /srv somewhere [1], not etc, so it wont conflicts with 
> "Stateless 
> Systems, Factory Reset, Golden Master" Systems?
> 
> On top of that if this gets created under /etc it's a free for all 
> for 
> administers to tinker with.
> 
> Arguably supporting this et all or supporting this without forcefully 
> 
> having to bind those timer units with an existing type service unit 
> is a 
> mistake so it might just be better to direct them to install and use 
> cron instead and have cockpit drop a snippet into /etc/cron.d and the 
> 
> likes instead
> 
> Jakub,Stephen what's the end game with this as in what kind of timer 
> definitions do you expect administrator be doing and support from the 
> 
> cockpit UI?



A good overview of what we're aiming to accomplish can be found here: h
ttps://github.com/cockpit-project/cockpit/wiki/Feature:-Systemd
-timers#stories

Though at the end of the day, it might be fair to say that systemd
timers are cool and very capable and we really just want to have a
decent UI for people to manipulate them. Cockpit seems like a good
place since it already does management of a lot of other systemd
functionality for Fedora Server.

If you have specific questions or concerns with that design, we'd love
to hear them.


signature.asc
Description: This is a digitally signed message part
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Simon McVittie]

On 10/06/15 11:46, "Jóhann B. Guðmundsson" wrote:
> I hate to say this since I'm against litter unit files across the entire
> filesystem like infective disease and administrator then have to run
> around trying to chase them down but is it not better to store this
> under /srv somewhere [1], not etc, so it wont conflicts with "Stateless
> Systems, Factory Reset, Golden Master" Systems?

To me this seems like (programmatically-assisted) configuration by a
sysadmin, just like when NetworkManager writes configured networks into
/etc/NetworkManager, or when `systemctl (en|dis)able` manipulates
symlinks in /etc? "You reset to factory configuration by deleting /etc"
seems entirely consistent with "tool-generated configuration goes in
/etc" to me.

/srv is certainly not the place: if "no program should rely on a
specific subdirectory structure of /srv" (that's a direct quote from the
FHS) then systemd can't rely on a particular directory structure being a
valid place to put its own files. You might say that systemd units could
go in /srv/systemd, but the FHS' point is that it would be, for
instance, entirely valid for the freedesktop.org administrators to use
/srv/systemd as the directory for systemd.freedesktop.org's web content.

This makes /srv pretty useless from a distribution point of view, but
useful for sysadmins - Debian's take on that directory is essentially
"don't put anything in /srv out-of-the-box, so that the sysadmin can use
it however they want to" (a lot like /opt).

-- 
Simon McVittie
Collabora Ltd. 

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Jóhann B. Guðmundsson]

On 06/09/2015 10:21 PM, Lennart Poettering wrote:

On Tue, 09.06.15 19:33, Jakub Skořepa (ja...@skorepa.info) wrote:


Hello,
My name is Jakub Skořepa and I'm working on Cockpit UI for Systemd
Timers.

For that I need to create and modify systemd unit files. Cockpit uses D
-Bus for everything so I need D-Bus API for that. I think that it would
be beneficial if systemd was able to create unit files on-the-fly using
through D-Bus method call.

You can do that already, in the concept of "transient" units. However,
these units are not persistent, they are stored in /run and go away on
reboots. Google for the StartTransientUnit() bus call for details.

I think it would be a good idea to also allow to create persisent
units in a similar way eventually. However, that's not as obvious and
easy as it sounds, it starts from the question where to store those
units. Cron-like semantics would suggest somewhere below /var, but
that means they aren't available at early boot, and we'd have to
reload the configuration and requeue all jobs when /var appears. Which
means we'd have to put them in /etc instead, which however is
suboptimal for many usecases

I think I'd be willing to merge a patch that adds adds a new bus call
CreatePersistentUnit() that works like StartTransientUnit() except
that it only creates but not starts a unit, and that it stores the
unit in /etc. (Note: the only reason StartTransientUnit() not only
creates but also starts a unit is because the unit would otherwise be
GC'ed away immediately and not be reconstructable after that...)




I hate to say this since I'm against litter unit files across the entire 
filesystem like infective disease and administrator then have to run 
around trying to chase them down but is it not better to store this 
under /srv somewhere [1], not etc, so it wont conflicts with "Stateless 
Systems, Factory Reset, Golden Master" Systems?


On top of that if this gets created under /etc it's a free for all for 
administers to tinker with.


Arguably supporting this et all or supporting this without forcefully 
having to bind those timer units with an existing type service unit is a 
mistake so it might just be better to direct them to install and use 
cron instead and have cockpit drop a snippet into /etc/cron.d and the 
likes instead


Jakub,Stephen what's the end game with this as in what kind of timer 
definitions do you expect administrator be doing and support from the 
cockpit UI?


1. http://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s17.html
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Lennart Poettering]

On Tue, 09.06.15 14:47, Stephen Gallagher (sgall...@redhat.com) wrote:

> In order for ModifyUnit() to work, there also needs to be a way to
> retrieve the current definition of the unit on the system. (And,
> ideally, a way to retrieve this for any unit currently loaded into
> systemd).

"systemctl cat" gives you the current definition of a unit. "systemctl
show" will give you the full list of settings in effect for a unit
(which is considerably more than the former). We have a nice API for
the latter (since all those setting are exposed as simply bus props),
but the former involves opening the text files on the client side.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Lennart Poettering]

On Tue, 09.06.15 19:33, Jakub Skořepa (ja...@skorepa.info) wrote:

> Hello,
> My name is Jakub Skořepa and I'm working on Cockpit UI for Systemd
> Timers.
> 
> For that I need to create and modify systemd unit files. Cockpit uses D
> -Bus for everything so I need D-Bus API for that. I think that it would
> be beneficial if systemd was able to create unit files on-the-fly using
> through D-Bus method call.

You can do that already, in the concept of "transient" units. However,
these units are not persistent, they are stored in /run and go away on
reboots. Google for the StartTransientUnit() bus call for details.

I think it would be a good idea to also allow to create persisent
units in a similar way eventually. However, that's not as obvious and
easy as it sounds, it starts from the question where to store those
units. Cron-like semantics would suggest somewhere below /var, but
that means they aren't available at early boot, and we'd have to
reload the configuration and requeue all jobs when /var appears. Which
means we'd have to put them in /etc instead, which however is
suboptimal for many usecases 

I think I'd be willing to merge a patch that adds adds a new bus call
CreatePersistentUnit() that works like StartTransientUnit() except
that it only creates but not starts a unit, and that it stores the
unit in /etc. (Note: the only reason StartTransientUnit() not only
creates but also starts a unit is because the unit would otherwise be
GC'ed away immediately and not be reconstructable after that...)

> I plan creating simple server for doing this task but long-term it woul
> d be better if systemd was capable of doing this.
> 
> I propose methods with following signatures:
> 
> # CreateUnit:
> # ModifyUnit: (same signature)
> # CreateUnits:
> # ModifyUnits: (same signature as CreateUnits)

Note that StartTransientUnits() already allows creating multiple
units, with appropriate properties, and any such new call should be
modelled after that, should it cover persistent units.

Also, we have the SetProperties() call already for on the Unit object
which allows altering any property during runtime, either persisently
in /etc or transiently in /run. Hence ModifyUnits is unnecessary.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Creating units using D-Bus

[Stephen Gallagher]

On Tue, 2015-06-09 at 19:33 +0200, Jakub Skořepa wrote:
> Hello,
> My name is Jakub Skořepa and I'm working on Cockpit UI for Systemd
> Timers.
> 
> For that I need to create and modify systemd unit files. Cockpit uses 
> D
> -Bus for everything so I need D-Bus API for that. I think that it 
> would
> be beneficial if systemd was able to create unit files on-the-fly 
> using
> through D-Bus method call.
> 
> I plan creating simple server for doing this task but long-term it 
> woul
> d be better if systemd was capable of doing this.
> 
> I propose methods with following signatures:
> 
> # CreateUnit:
> 
>  Creates specified unit
> 
> sa{sa{ss}}
> |  |  ||
>   Name |  |Value
>  Section name |
>  Key
> 
> # ModifyUnit: (same signature)
> 
>  If section/key is not then section/key is left intact.
>  If Value="" then key is erased.
> 
> # CreateUnits:
> 
>  Creates multiple units. Same as CreateUnit called for each top-level 
> 
>  array element.
> 
> a{sa{sa{ss}}}
>   |  |  ||
> Name |  |Value
>Section name |
>Key
> 
> # ModifyUnits: (same signature as CreateUnits)
> 
>  Modifies multiple units. Same as ModifyUnit called for each top
> -level
>  array element.
> 


In order for ModifyUnit() to work, there also needs to be a way to
retrieve the current definition of the unit on the system. (And,
ideally, a way to retrieve this for any unit currently loaded into
systemd).

signature.asc
Description: This is a digitally signed message part
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] Creating units using D-Bus

[Jakub Skořepa]

Hello,
My name is Jakub Skořepa and I'm working on Cockpit UI for Systemd
Timers.

For that I need to create and modify systemd unit files. Cockpit uses D
-Bus for everything so I need D-Bus API for that. I think that it would
be beneficial if systemd was able to create unit files on-the-fly using
through D-Bus method call.

I plan creating simple server for doing this task but long-term it woul
d be better if systemd was capable of doing this.

I propose methods with following signatures:

# CreateUnit:

 Creates specified unit

sa{sa{ss}}
|  |  ||
  Name |  |Value
 Section name |
 Key

# ModifyUnit: (same signature)

 If section/key is not then section/key is left intact.
 If Value="" then key is erased.

# CreateUnits:

 Creates multiple units. Same as CreateUnit called for each top-level 
 array element.

a{sa{sa{ss}}}
  |  |  ||
Name |  |Value
   Section name |
   Key

# ModifyUnits: (same signature as CreateUnits)

 Modifies multiple units. Same as ModifyUnit called for each top-level
 array element.

Regards,
Jakub

signature.asc
Description: This is a digitally signed message part


smime.p7s
Description: S/MIME cryptographic signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel