Re: libinit idea

2014-02-24 Thread Don Lewis
On 23 Feb, Lucius Rizzo wrote:
 * Andreas Nilsson andrn...@gmail.com [2014-02-23 09:33]:
 Imho, the replacement to init and rc-scripts I sometimes think about
 would be to import SMF from opensolaris/illumos. There one can at
 least get the commands run and config used without looking at the
 source code.
 
 I like SMF from Solaris 11 onwards and even SmartOS. However, I have
 found to like systemd and use via systemctl on Arch far nicer than any
 other rc scripts to date. 
 
 Anyone care to share their thoughts on the pros and cons of something
 like systemd the way Arch does?

I've got a Fedora server here that has systemd and I've come to dislike
it.  It seems to be one of those Do not open.  No user serviceable
parts inside. sorts of things.

I was never able to get it to start NUT properly.

More often than not, it fails to come up multi-user.  The machine has a
large number of disks (mostly JFS and XFS) attached to it, and even
after what I think should be a clean shutdown, it seems to want to fsck
a bunch of them. Unfortunately, there seems to be some sort of timeout
on that, so a bunch get skipped and then don't get mounted.  I have to
manually fsck everything in single user mode.  Then if I reboot, it
*might* come up properly.  I haven't been able to find any knobs to
adjust the timeout.  Sometimes, there is just a message that says
something like an error occurred at the top of the screen, just before
the prompt for the single-user password, with no clue as to what it is
unhappy about.

Emergency shutdown can also be a problem.  If I'm around when the power
fails, I manually try to shut down the machine before the UPS battery
runs down.  I don't have the screen on the UPS, so I hit the power
button and cross my fingers that the machine will make it through the
clean shutdown sequence in time.  It seems to take forever (many
minutes) and I have no idea what the heck it is spending all of its time
on.

The documentation seems to be very sparse.

My plan is to migrate this function to a FreeBSD server.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Thomas Mueller
from Don Lewis:

 I've got a Fedora server here that has systemd and I've come to dislike
 it.  It seems to be one of those Do not open.  No user serviceable
 parts inside. sorts of things.
  
 I was never able to get it to start NUT properly.
  
 More often than not, it fails to come up multi-user.  The machine has a
 large number of disks (mostly JFS and XFS) attached to it, and even
 after what I think should be a clean shutdown, it seems to want to fsck
 a bunch of them. Unfortunately, there seems to be some sort of timeout
 on that, so a bunch get skipped and then don't get mounted.  I have to
 manually fsck everything in single user mode.  Then if I reboot, it
 *might* come up properly.  I haven't been able to find any knobs to
 adjust the timeout.  Sometimes, there is just a message that says
 something like an error occurred at the top of the screen, just before
 the prompt for the single-user password, with no clue as to what it is
 unhappy about.

 Emergency shutdown can also be a problem.  If I'm around when the power
 fails, I manually try to shut down the machine before the UPS battery
 runs down.  I don't have the screen on the UPS, so I hit the power
 button and cross my fingers that the machine will make it through the
 clean shutdown sequence in time.  It seems to take forever (many
 minutes) and I have no idea what the heck it is spending all of its time
 on.

 The documentation seems to be very sparse.

 My plan is to migrate this function to a FreeBSD server.

This looks scandalously slow.  It reminds me of the time with OS/2 Warp 4 in 
the late 1990s when I had to close Netscape web browser in preparation for 
shutdown, and it took 15 minutes because it was a hog for memory, by late 1990s 
standards.  I had 20 MB RAM, not bad for those days.

What would happen if you typed at the command prompt
shutdown -r now
or
shutdown -p now
?

Would it take seemingly forever?

I would like to try systemd in Linux, can't say at this stage whether I'll like 
it, hate it, or somewhere in between.

Tom
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


RE: libinit idea

2014-02-24 Thread Bruno Lauzé
It's really good to read the comments and see people point of view on this 
topic. 
I totally agree RC system is really good and stable, easy and extensible.
The main concern I have here is performance. Starting a system is really slow.
Boot to X (with simple xdm) is minimum 25 secs and I just feel there's 
something to do to cut this delay.




 Date: Sun, 23 Feb 2014 11:33:30 -0800
 From: list_free...@bluerosetech.com
 To: fjwc...@gmail.com
 Subject: Re: libinit idea
 CC: freebsd-current@freebsd.org

 On 2/23/2014 10:31 AM, Freddie Cash wrote:
 The main developer for systemd is very anti-portability and anti-!Linux. He
 had actively rejected patches that made his projects work on non-Linux
 systems. In order to port systemd to a non-Linux system, he wants you to
 first implement every Linux feature that systemd uses.

 systemd is a non-starter, and not with considering.

 It's pretty clear Lennart Poettering failed history.

 Perhaps the best examples of why systemd is an anti-pattern is this:
 systemd stores logs in a binary journal. After the Linux zealots have
 spent about two decades using binary system data storage as one of the
 reasons Windows sucks, that strikes me as particularly ironic.

 The RC system we have is proof you do not need anything more than bourne
 and a respectable amount of intelligence to design a good init system.
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
   
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Matthias Gamsjager
On Mon, Feb 24, 2014 at 2:15 PM, Bruno Lauzé brunola...@msn.com wrote:

 It's really good to read the comments and see people point of view on this
 topic.
 I totally agree RC system is really good and stable, easy and extensible.
 The main concern I have here is performance. Starting a system is really
 slow.
 Boot to X (with simple xdm) is minimum 25 secs and I just feel there's
 something to do to cut this delay.




How about delaying the startup of services that are not necessary right at
the start. For example sshd, samba etc could be loaded after xdm ( or even
after the DE has loaded).
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread David Chisnall
On 24 Feb 2014, at 13:25, Matthias Gamsjager mgamsja...@gmail.com wrote:

 How about delaying the startup of services that are not necessary right at
 the start. For example sshd, samba etc could be loaded after xdm ( or even
 after the DE has loaded).

It's a good idea, but it depends on a far more complex system than our current 
rc.  

For example, what happens if you're mounting home directories over SMB?  You 
need to make sure that XDM doesn't complete the login until samba has started, 
which requires automounter / autofs integration with the init system.  You want 
to speculatively start samba (because you know a user is going to need it), but 
in such a way that you can delay directory lookup resolution until it's 
actually started.

There is a reason most systems are moving away from services started by a pile 
of shell scripts: the complex dependencies and event models that they must 
handle require some complex generic logic.  You can probably implement this in 
shell scripts, but it probably won't be fast and it definitely won't be 
maintainable.  

I would be fully in favour of importing a modern, flexible, service management 
framework into FreeBSD, but it needs some very careful design.  Launchd, 
systemd, SMF, and so on all have some good ideas, but none is quite the right 
fit for FreeBSD.

David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Andreas Nilsson
On Mon, Feb 24, 2014 at 2:25 PM, Matthias Gamsjager mgamsja...@gmail.comwrote:

 On Mon, Feb 24, 2014 at 2:15 PM, Bruno Lauzé brunola...@msn.com wrote:

  It's really good to read the comments and see people point of view on
 this
  topic.
  I totally agree RC system is really good and stable, easy and extensible.
  The main concern I have here is performance. Starting a system is really
  slow.
  Boot to X (with simple xdm) is minimum 25 secs and I just feel there's
  something to do to cut this delay.
 



 How about delaying the startup of services that are not necessary right at
 the start. For example sshd, samba etc could be loaded after xdm ( or even
 after the DE has loaded).


On my core i5 with regular hdd I get:
23sec to boot kernel
13sec from mounted root to slim login promp.

Fixing the kernel boot time seems like a bigger gain, imho.

Best regards
Andreas
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Christian Laursen

On 02/24/14 14:34, Andreas Nilsson wrote:

On Mon, Feb 24, 2014 at 2:25 PM, Matthias Gamsjager mgamsja...@gmail.comwrote:


How about delaying the startup of services that are not necessary right at
the start. For example sshd, samba etc could be loaded after xdm ( or even
after the DE has loaded).



On my core i5 with regular hdd I get:
23sec to boot kernel
13sec from mounted root to slim login promp.


I have not done any measurements here but this is what I experience as 
well. Once / has been mounted the rest of the startup doesn't take long.



Fixing the kernel boot time seems like a bigger gain, imho.


+1

--
Christian Laursen
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Lucius Rizzo
* Don Lewis truck...@freebsd.org [2014-02-24 00:56]:
 On 23 Feb, Lucius Rizzo wrote:
  * Andreas Nilsson andrn...@gmail.com [2014-02-23 09:33]:
  Imho, the replacement to init and rc-scripts I sometimes think about
  would be to import SMF from opensolaris/illumos. There one can at
  least get the commands run and config used without looking at the
  source code.
  
  I like SMF from Solaris 11 onwards and even SmartOS. However, I have
  found to like systemd and use via systemctl on Arch far nicer than any
  other rc scripts to date. 
  
  Anyone care to share their thoughts on the pros and cons of something
  like systemd the way Arch does?
 
 I've got a Fedora server here that has systemd and I've come to dislike
 it.  It seems to be one of those Do not open.  No user serviceable
 parts inside. sorts of things.

I had an entirely different experience - albeit on Arch. I had to use
systemd via systemctl for pretty much everything to build on the
machine. I use btrfs + linux-ck on Arch with systemd. In order to get
stuff to work, I had a pretty detailed  learning  experience with
systemd when I helped on the sendmail AUR port for Arch including
writing/enabling scripts to work with journald and systemd. 

Naturally, I'm somewhat surprised to see the the author's attitude
towards licensing and non-Linux support. Its rare to come across devs
who are zealous to such a degree. 

However, there seems to be an agreement at least in part to modernize or
optimize the boot-up/rc env in FreeBSD.

I don't know for most of you but for me, while this is interesting...I
almost never really reboot that often for me to actually see a real
advantage of a 3 sec boot-up difference by throwing away what we have and
migrating to something else. 

Which is why we should also consider more than just boot-up times -- for
me having a newer more intelligent way to manage rc environment might
bear more interest than a few second boot difference that I only see
once in a blue moon.

Though, now I am also at a point where I know commands in -
Debian/CentOS/Arch, BSD (FreeBSD/NetBSD) and Solaris 10/11/SmartOS (all
with different ways of doing thing). What's annoying is that each of
these OS/flavors have nuances in use of their rc scripts and while
^now^ it doesn't bother me at all, I imagine it may for a newb. 

In most cases, everything works. But when it doesn't, its  annoying to
remember svcadmin vs systemctl vs /etc/init.d vs service vs /etc/rc.d/
along with all the other commands just to check the logs...

-- 

| _o_ |_)o_ _  _  
|_|_|(_||_|_ | \|/_/_(_) - Lucius.Tel
--
++ New members are urgently needed in the Society for Prevention of ++
++  Cruelty to Yourself.  Apply within. ++
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Don Lewis
On 24 Feb, Thomas Mueller wrote:
 from Don Lewis:
 
 I've got a Fedora server here that has systemd and I've come to
 dislike it.  It seems to be one of those Do not open.  No user
 serviceable parts inside. sorts of things.
   
 I was never able to get it to start NUT properly.
   
 More often than not, it fails to come up multi-user.  The machine has
 a large number of disks (mostly JFS and XFS) attached to it, and even
 after what I think should be a clean shutdown, it seems to want to
 fsck a bunch of them. Unfortunately, there seems to be some sort of
 timeout on that, so a bunch get skipped and then don't get mounted. 
 I have to manually fsck everything in single user mode.  Then if I
 reboot, it
 *might* come up properly.  I haven't been able to find any knobs to
 adjust the timeout.  Sometimes, there is just a message that says
 something like an error occurred at the top of the screen, just
 before the prompt for the single-user password, with no clue as to
 what it is unhappy about.
 
 Emergency shutdown can also be a problem.  If I'm around when the
 power fails, I manually try to shut down the machine before the UPS
 battery runs down.  I don't have the screen on the UPS, so I hit the
 power button and cross my fingers that the machine will make it
 through the clean shutdown sequence in time.  It seems to take
 forever (many minutes) and I have no idea what the heck it is
 spending all of its time on.
 
 The documentation seems to be very sparse.
 
 My plan is to migrate this function to a FreeBSD server.
 
 This looks scandalously slow.  It reminds me of the time with OS/2
 Warp 4 in the late 1990s when I had to close Netscape web browser in
 preparation for shutdown, and it took 15 minutes because it was a hog
 for memory, by late 1990s standards.  I had 20 MB RAM, not bad for
 those days.
 
 What would happen if you typed at the command prompt
 shutdown -r now
 or
 shutdown -p now
 ?
 Would it take seemingly forever?

In Linux-land shutdown -h now does what our shutdown -p now does.
For whatever reason, doing shutdown that way seems faster.  That's not
so handy for me in the power loss case because the machine is running X
and is most likely sitting in the screensaver.  Switching to another
vty, doing a root login, and typing in the shutdown command is a lot of
typing to get right while flying blind without a monitor.

There might also be a slowdown due to the network being down, though
it's hard to tell in my case.  I'm also not using NFS, which would be
the obvious culprit.

I forgot to mention that the command line tools are feel cumbersome.  To
restart a service:
FreeBSD:   /etc/rc.d/foo restart
Old Linux: /etc/init.d/foo restart
Systemd:   systemctl restart foo.service
seems worse that that when I'm actually typing it ...

 Would it take seemingly forever?
 
 I would like to try systemd in Linux, can't say at this stage whether
 I'll like it, hate it, or somewhere in between.

There's no substitute for firsthand experience.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Don Lewis
On 23 Feb, David Chisnall wrote:

 5) In a world where we're moving towards sandboxing services via
 Capsicum, the service launcher needs to be able to create services
 with a potentially large set of initial file descriptors (including a
 socket to Casper), based on the configuration policy.

I don't think we want to lose the option of running the daemon in debug
mode in the foreground with extra verbosity sent to stdout/stderr and
possibly launched from gdb.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Nikolai Lifanov
On 02/24/14 13:00, Don Lewis wrote:
 On 24 Feb, Thomas Mueller wrote:
 from Don Lewis:

 I've got a Fedora server here that has systemd and I've come to
 dislike it.  It seems to be one of those Do not open.  No user
 serviceable parts inside. sorts of things.
   
 I was never able to get it to start NUT properly.
   
 More often than not, it fails to come up multi-user.  The machine has
 a large number of disks (mostly JFS and XFS) attached to it, and even
 after what I think should be a clean shutdown, it seems to want to
 fsck a bunch of them. Unfortunately, there seems to be some sort of
 timeout on that, so a bunch get skipped and then don't get mounted. 
 I have to manually fsck everything in single user mode.  Then if I
 reboot, it
 *might* come up properly.  I haven't been able to find any knobs to
 adjust the timeout.  Sometimes, there is just a message that says
 something like an error occurred at the top of the screen, just
 before the prompt for the single-user password, with no clue as to
 what it is unhappy about.

 Emergency shutdown can also be a problem.  If I'm around when the
 power fails, I manually try to shut down the machine before the UPS
 battery runs down.  I don't have the screen on the UPS, so I hit the
 power button and cross my fingers that the machine will make it
 through the clean shutdown sequence in time.  It seems to take
 forever (many minutes) and I have no idea what the heck it is
 spending all of its time on.

 The documentation seems to be very sparse.

 My plan is to migrate this function to a FreeBSD server.

 This looks scandalously slow.  It reminds me of the time with OS/2
 Warp 4 in the late 1990s when I had to close Netscape web browser in
 preparation for shutdown, and it took 15 minutes because it was a hog
 for memory, by late 1990s standards.  I had 20 MB RAM, not bad for
 those days.

 What would happen if you typed at the command prompt
 shutdown -r now
 or
 shutdown -p now
 ?
 Would it take seemingly forever?
 
 In Linux-land shutdown -h now does what our shutdown -p now does.
 For whatever reason, doing shutdown that way seems faster.  That's not
 so handy for me in the power loss case because the machine is running X
 and is most likely sitting in the screensaver.  Switching to another
 vty, doing a root login, and typing in the shutdown command is a lot of
 typing to get right while flying blind without a monitor.
 
 There might also be a slowdown due to the network being down, though
 it's hard to tell in my case.  I'm also not using NFS, which would be
 the obvious culprit.
 
 I forgot to mention that the command line tools are feel cumbersome.  To
 restart a service:
   FreeBSD:   /etc/rc.d/foo restart
   Old Linux: /etc/init.d/foo restart
   Systemd:   systemctl restart foo.service
 seems worse that that when I'm actually typing it ...
 

The Handbook-recommended invocation, which also works on linux, is
service foo restart.

 Would it take seemingly forever?

 I would like to try systemd in Linux, can't say at this stage whether
 I'll like it, hate it, or somewhere in between.
 
 There's no substitute for firsthand experience.
 

- Nikolai Lifanov
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-24 Thread Slawa Olhovchenkov
On Mon, Feb 24, 2014 at 10:00:57AM -0800, Don Lewis wrote:

 I forgot to mention that the command line tools are feel cumbersome.  To
 restart a service:
   FreeBSD:   /etc/rc.d/foo restart

service foo restart

   Old Linux: /etc/init.d/foo restart

service foo restart

   Systemd:   systemctl restart foo.service
 seems worse that that when I'm actually typing it ...
 
  Would it take seemingly forever?
  
  I would like to try systemd in Linux, can't say at this stage whether
  I'll like it, hate it, or somewhere in between.
 
 There's no substitute for firsthand experience.
 
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread Andreas Nilsson
On Sun, Feb 23, 2014 at 8:46 AM, Scot Hetzel swhet...@gmail.com wrote:

 On Sat, Feb 22, 2014 at 5:54 PM, Bruno Lauzé brunola...@msn.com wrote:
  https://github.com/brunolauze/libnit
 
  I know there's really big debate about init system but here's my
 tentative to propose a new model to replace rc.
 
  Let's call it libinit but the name as no significance for now.
 
  I started coding a library with the following architecture.
 
  the main idea is to rewrite rc in C language.


To me this seems like much work for no real gain. And it would make it that
much harder to debug a misbehaving script, in more than one way:
* c is a lot longer than sh, ie sh expresses the logic that much more
clearly.
* to actually see the new script would require to have the source code
installed, something far from everybody has.

 
  a utility called system would act a little bit like service command does.
 
  a folder would contains libraries instead of scripts inside
 [target]/etc/rc.d
  so we can add as many librairies a user desire and interlink the order
 of each piece among all like in rc.
 
 libraries don't belong in [target]/etc/rc.d, they would have to be in
 {/usr,}/lib{exec,}/rc.d or ${PREFIX}/lib{exec,}/rc.d

 Imho, the replacement to init and rc-scripts I sometimes think about would
be to import SMF from opensolaris/illumos. There one can at least get the
commands run and config used without looking at the source code.

Best regards
Andreas
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread David Chisnall
Hi Bruno,

To preface this, I'd like to say that I do believe that FreeBSD does need a 
more modern init system.  SMF on Solaris and Launchd on Darwin both have some 
advantages.  These are what I see as the limitations in our current design (not 
in priority order):

1) Options are implicit.  Because init scripts are written in a Turing-complete 
language, including routines scattered across a large number of files, it's 
very difficult for a tool (e.g. a GUI system administration tool like the SMF 
GUI on Solaris) to enumerate all of the options.  Additionally, these options 
are untyped so a GUI that does manage to find them can't tell whether it should 
be displaying a checkbox (for a boolean value) or a text field.  Additionally, 
it can't tell what the dependencies are between them.

2) Dependency information between services is poor.  If service A depends on B, 
and B is not running, then starting A should start B.  If nothing else depends 
on B and B was not explicitly started, then B should stop.  The last part is 
the hard one with shell scripts, because it requires maintaining the dependency 
graph and effectively garbage collecting it (explicitly started services are 
roots, dependencies are references).

3) It's easy for tools to add lines to rc.conf, it's hard to remove them.  If 
you're administering a large number of nodes, you want to be able to push out 
updates to all, in a way that doesn't clobber local changes.  Text file 
processing here is always going to be a fragile hack.

4) Shell scripts are a good tool for invoking complex sequences of command-line 
programs, but have a measurable overhead relative to fork() / exec() for 
running a single command.  Most rc actions just run a single program, we should 
only be doing a shell invocation for ones that are more complex.

5) In a world where we're moving towards sandboxing services via Capsicum, the 
service launcher needs to be able to create services with a potentially large 
set of initial file descriptors (including a socket to Casper), based on the 
configuration policy.

6) As with launchd, it would make sense for a service management framework to 
subsume inetd and cron (and devd!) functionality, because timer events, network 
events, system startup, system shutdown, device attachment, and explicit 
service starting from the command line are even sources that trigger changes to 
the service dependency graph.

7) Inspecting the system status is very hard with the current system.  I want 
to be able to see a list of all running services, along with the reason why 
they were started.

8) Again, in capsicum world we'd like inetd-like functionality for UNIX domain 
sockets, so that it's possible to lazily start things like powerboxes.

In terms of design, I don't think that turning rc scripts into libraries is a 
win.  I would like to see them replaced by a declarative config file in a 
structured format that provides dependency information, configuration options, 
and the commands required to start / stop the service (which can be shell 
scripts, but don't have to invoke a shell).  The configuration options should 
be separate from the configuration variables, with the former provided by the 
package and the latter by the system administrator.

We already have, in the base system, a library for parsing a structured 
configuration file format that is used for the pkg tools and a library with an 
efficient binary serialisation (libnv) that is used for various 
Capsicum-related functions.  I'd like to see these combined so that in embedded 
devices all of the configuration files could be stored in the binary 
serialisation (which is denser and faster to parse) and turned back into the 
human-readable one only for editing.  

I do like the idea of an init library, so that the functionality is easily 
reusable and can be run in nested scopes.  I don't think we gain anything by 
rewriting shell scripts in C, other than a maintenance headache and a 
requirement that all port maintainers be fluent C programmers...

David


On 22 Feb 2014, at 23:54, Bruno Lauzé brunola...@msn.com wrote:

 https://github.com/brunolauze/libnit
 
 I know there's really big debate about init system but here's my tentative to 
 propose a new model to replace rc.
 
 Let's call it libinit but the name as no significance for now.
 
 I started coding a library with the following architecture.
 
 the main idea is to rewrite rc in C language.
 
 a utility called system would act a little bit like service command does.
 
 a folder would contains libraries instead of scripts inside [target]/etc/rc.d
 so we can add as many librairies a user desire and interlink the order of 
 each piece among all like in rc.
 
 each library would follow and expose the following pattern:
 
 char **provide(); /* returns all the PROVIDE a library contains */
 
 then for each provide() value the library would expose :
 
 XXX_provide()
 XXX_require()
 XXX_before()
 XXX_keywords()
 
 and 

Re: libinit idea

2014-02-23 Thread Allan Jude
On 2014-02-23 04:12, David Chisnall wrote:
 Hi Bruno,
 
 3) It's easy for tools to add lines to rc.conf, it's hard to remove them.  If 
 you're administering a large number of nodes, you want to be able to push out 
 updates to all, in a way that doesn't clobber local changes.  Text file 
 processing here is always going to be a fragile hack.
 

sysrc solves this nicely, it is in base now, and is great for
programmatically adding, removing and changing lines in rc.conf style
files. It is also in ports for older versions of FreeBSD where it is not
in base.

At ScaleEngine, we make extensive use of a global rc.conf pushed by
puppet, with local modifications in rc.conf.local (and we actually
extend this with an rc.conf.role file, added to the rc_conf_files.


-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature


Re: libinit idea

2014-02-23 Thread David Chisnall
On 23 Feb 2014, at 18:11, Allan Jude free...@allanjude.com wrote:

 sysrc solves this nicely, it is in base now, and is great for
 programmatically adding, removing and changing lines in rc.conf style
 files. It is also in ports for older versions of FreeBSD where it is not
 in base.

The problem is, there is no such thing as an rc.conf style file.  rc.conf is 
just a shell script.  If you only edit it with sysrc, or you are careful to 
preserve the structure, then it's fine.  There is absolutely nothing stopping 
you, however, from writing arbitrarily complex shell scripts inside rc.conf.  
Sure, it's a terrible idea to do so, but when has that ever stopped anyone?

An rc-replacement could enforce this by only accepting purely declarative files 
for configuration, guaranteeing that if they were syntactically valid they 
would also be machine editable, no matter what the user does to them.

David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread Lucius Rizzo
* Andreas Nilsson andrn...@gmail.com [2014-02-23 09:33]:
 Imho, the replacement to init and rc-scripts I sometimes think about
 would be to import SMF from opensolaris/illumos. There one can at
 least get the commands run and config used without looking at the
 source code.

I like SMF from Solaris 11 onwards and even SmartOS. However, I have
found to like systemd and use via systemctl on Arch far nicer than any
other rc scripts to date. 

Anyone care to share their thoughts on the pros and cons of something
like systemd the way Arch does?

-- 

| _o_ |_)o_ _  _  
|_|_|(_||_|_ | \|/_/_(_) - Lucius.Tel
--
++ Q:   How many Bell Labs Vice Presidents does it take to change a light bulb? 
++
++ A:   That's proprietary information.  Answer available from ATT on payment 
++
++  of license fee (binary only). ++
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread Freddie Cash
Typos and terseness brought to you by the LG G2 running SlimKat.
On Feb 23, 2014 10:22 AM, Lucius Rizzo lucius.ri...@lucius.xxx wrote:

 * Andreas Nilsson andrn...@gmail.com [2014-02-23 09:33]:
  Imho, the replacement to init and rc-scripts I sometimes think about
  would be to import SMF from opensolaris/illumos. There one can at
  least get the commands run and config used without looking at the
  source code.

 I like SMF from Solaris 11 onwards and even SmartOS. However, I have
 found to like systemd and use via systemctl on Arch far nicer than any
 other rc scripts to date.

 Anyone care to share their thoughts on the pros and cons of something
 like systemd the way Arch does?

The main developer for systemd is very anti-portability and anti-!Linux. He
had actively rejected patches that made his projects work on non-Linux
systems. In order to port systemd to a non-Linux system, he wants you to
first implement every Linux feature that systemd uses.

systemd is a non-starter, and not with considering.

In theory, devd could be extended to support socket-activation, and the
support for that added to RCng, providing the bulk of the systemd features,
without having to deal with the systemd devs.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread Nikolai Lifanov

On 2014-02-23 13:17, David Chisnall wrote:

On 23 Feb 2014, at 18:11, Allan Jude free...@allanjude.com wrote:


sysrc solves this nicely, it is in base now, and is great for
programmatically adding, removing and changing lines in rc.conf style
files. It is also in ports for older versions of FreeBSD where it is 
not

in base.


The problem is, there is no such thing as an rc.conf style file.
rc.conf is just a shell script.  If you only edit it with sysrc, or
you are careful to preserve the structure, then it's fine.  There is
absolutely nothing stopping you, however, from writing arbitrarily
complex shell scripts inside rc.conf.  Sure, it's a terrible idea to
do so, but when has that ever stopped anyone?

An rc-replacement could enforce this by only accepting purely
declarative files for configuration, guaranteeing that if they were
syntactically valid they would also be machine editable, no matter
what the user does to them.

David



Just my $0.02:

I don't believe our current RC is broken. It's faster than most, it 
supports an early-late divider, virtual targets (NETWORK, etc.), 
dependencies, etc.
Rewriting scripts (units) in C has a non-trivial cost to customization 
for end users. I have custom, packaged, RC scripts in /usr/local that 
are pretty easy to add and maintain. Shell script logic can go there and 
not in rc.conf proper. The rcorder program provides a pretty good idea 
for when these can be run, and, as already pointed out, sysrc can be 
used to add/remove/configure these on clusters of (automatically) 
managed machines. If all scripts a properly written, service foo 
status can also provide something that can be acted upon by 
configuration management systems.
Serialization is great (libnv), but it's just gravy. The only feature I 
see that's missing is (SMF-style) service management, but for hardware 
events, like losing a network link or a disk, can be plugged in to devd 
configuration, also easily. Also, in real-life deployments, shutting 
down service dependencies is not practical either. For example, if 
postfix crashes, I don't want to stop postgey or dovecot. I just want to 
nanny postfix back up rather than trying to bring up the whole stack. 
Also, forking a shell does not have any significant cost to it when 
bringing up something like MySQL, since it's a small fraction of where 
the system spends its time to bring up a useful service.


Rewriting scripts in C doesn't provide service management or give any 
on-demand (inetd, read: launchd) functionality.


- Nikolai Lifanov

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread David Chisnall
On 23 Feb 2014, at 18:31, Freddie Cash fjwc...@gmail.com wrote:

 The main developer for systemd is very anti-portability and anti-!Linux. He
 had actively rejected patches that made his projects work on non-Linux
 systems. In order to port systemd to a non-Linux system, he wants you to
 first implement every Linux feature that systemd uses.
 
 systemd is a non-starter, and not with considering.

I don't think that's a relevant discussion.  The license would likely preclude 
systemd from making it into the base system anyway.  Please let's not be too 
negative about the author of systemd: he's responsible for more people 
switching from Linux to FreeBSD than any other single individual I can think of 
and I would strongly encourage him to continue.

The relevant question is whether it does anything in a way that is sufficiently 
sensible to merit a FreeBSD service management infrastructure doing it in the 
same (or a similar) way.

Oh, two things missing from my original list:

- Service jails should be able to run without an init process, with just the 
required libraries installed and the host machine's init system starting the 
jail and the service process(es) inside it.

- The init system should use process descriptors, not pids, for tracking 
processes, preventing issues with pid reuse and so on (and removing the need to 
write pid files).  If process descriptors do not provide the required 
functionality (e.g. the ability to trace forked children) then this should be 
added.

David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread Nikolai Lifanov

On 2014-02-23 13:47, David Chisnall wrote:

On 23 Feb 2014, at 18:31, Freddie Cash fjwc...@gmail.com wrote:

The main developer for systemd is very anti-portability and 
anti-!Linux. He

had actively rejected patches that made his projects work on non-Linux
systems. In order to port systemd to a non-Linux system, he wants you 
to

first implement every Linux feature that systemd uses.

systemd is a non-starter, and not with considering.


I don't think that's a relevant discussion.  The license would likely
preclude systemd from making it into the base system anyway.  Please
let's not be too negative about the author of systemd: he's
responsible for more people switching from Linux to FreeBSD than any
other single individual I can think of and I would strongly encourage
him to continue.



I also noticed this.


The relevant question is whether it does anything in a way that is
sufficiently sensible to merit a FreeBSD service management
infrastructure doing it in the same (or a similar) way.

Oh, two things missing from my original list:

- Service jails should be able to run without an init process, with
just the required libraries installed and the host machine's init
system starting the jail and the service process(es) inside it.



Isn't this a bit too complicated? If there is an init script under 
$jail/usr/local/etc/rc.d, then the host init will need to find it, which 
can be even more complicated if rc search path in the jail is customized 
(prefixed if it's managed by a different department, for example). Host 
init will have to read the jail configuration, parse it too, and then 
manage children and pids of the jailed services, including reparenting, 
all within a jail context. Then the admin in that jail would need to be 
able to restart services, affecting host init, which opens a whole new 
can of worms. If init program is skinny and not too complicated, which 
it is, there is no tangible overhead. And if a jail really needs a 
single simple service, init process in the jail can *be* that, like 
jexec myjail /bin/sh -c somestuff (or even /usr/local/bin/myservice -c 
myservice conf).



- The init system should use process descriptors, not pids, for
tracking processes, preventing issues with pid reuse and so on (and
removing the need to write pid files).  If process descriptors do not
provide the required functionality (e.g. the ability to trace forked
children) then this should be added.



This is a good idea.


David

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to 
freebsd-current-unsubscr...@freebsd.org

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread OutBackDingo
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



On 02/23/2014 02:09 PM, Nikolai Lifanov wrote:
 On 2014-02-23 13:47, David Chisnall wrote:
 On 23 Feb 2014, at 18:31, Freddie Cash fjwc...@gmail.com
 wrote:
 
 The main developer for systemd is very anti-portability and 
 anti-!Linux. He had actively rejected patches that made his
 projects work on non-Linux systems. In order to port systemd to
 a non-Linux system, he wants you to first implement every Linux
 feature that systemd uses.
 
 systemd is a non-starter, and not with considering.
 
 I don't think that's a relevant discussion.  The license would
 likely preclude systemd from making it into the base system
 anyway.  Please let's not be too negative about the author of
 systemd: he's responsible for more people switching from Linux to
 FreeBSD than any other single individual I can think of and I
 would strongly encourage him to continue.
 
 
 I also noticed this.
 
 The relevant question is whether it does anything in a way that
 is sufficiently sensible to merit a FreeBSD service management 
 infrastructure doing it in the same (or a similar) way.
 
 Oh, two things missing from my original list:
 
 - Service jails should be able to run without an init process,
 with just the required libraries installed and the host machine's
 init system starting the jail and the service process(es) inside
 it.
 
 
 Isn't this a bit too complicated? If there is an init script under 
 $jail/usr/local/etc/rc.d, then the host init will need to find it,
 which can be even more complicated if rc search path in the jail is
 customized (prefixed if it's managed by a different department, for
 example). Host init will have to read the jail configuration, parse
 it too, and then manage children and pids of the jailed services,
 including reparenting, all within a jail context. Then the admin in
 that jail would need to be able to restart services, affecting host
 init, which opens a whole new can of worms. If init program is
 skinny and not too complicated, which it is, there is no tangible
 overhead. And if a jail really needs a single simple service, init
 process in the jail can *be* that, like jexec myjail /bin/sh -c
 somestuff (or even /usr/local/bin/myservice -c myservice conf).
 
 - The init system should use process descriptors, not pids, for 
 tracking processes, preventing issues with pid reuse and so on
 (and removing the need to write pid files).  If process
 descriptors do not provide the required functionality (e.g. the
 ability to trace forked children) then this should be added.
 
 
 This is a good idea.
 
 David

openrc already does this and runs on FreeBSD, ive been using it on 5
boxes in testing so far, clealy replaces init without breaking rc.conf
configurations.

 
 ___ 
 freebsd-current@freebsd.org mailing list 
 http://lists.freebsd.org/mailman/listinfo/freebsd-current To
 unsubscribe, send any mail to 
 freebsd-current-unsubscr...@freebsd.org
 ___ 
 freebsd-current@freebsd.org mailing list 
 http://lists.freebsd.org/mailman/listinfo/freebsd-current To
 unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJTCkyWAAoJEP7cFACJNhNalmgH/1hquNUT75g4xOGc8CX+RyVD
kVYKW2SUgMOCCQwgXMydsZUafQ8kLSU24H8T9W2J/q2WDm1lWedqeO9/yk7t3pNS
8+c8YsTIuAYtPgiBES65YW667Cj9+FwN268fEqF58/WNS6kNx8snCfoRkvy6l7jN
M62ajdNJtgSxF/Iwyfuq4W6REBW3CF4rL+g64OGrK8PQ6yPteJfM3CUGQoUlHXZC
KHHY+qMYBSb/H0j9z7PzcpJsC/bb4fJRzccfVc34IV0ovdevLtQeJojTy4E55NhB
kWTX1EDRYVxv9Zvy1uPut5qAtc0b+UB5sAPON0RGxBB0rzjzIYZXlieKo4s752w=
=BXYO
-END PGP SIGNATURE-
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread Darren Pilgrim

On 2/23/2014 10:31 AM, Freddie Cash wrote:

The main developer for systemd is very anti-portability and anti-!Linux. He
had actively rejected patches that made his projects work on non-Linux
systems. In order to port systemd to a non-Linux system, he wants you to
first implement every Linux feature that systemd uses.

systemd is a non-starter, and not with considering.


It's pretty clear Lennart Poettering failed history.

Perhaps the best examples of why systemd is an anti-pattern is this: 
systemd stores logs in a binary journal.  After the Linux zealots have 
spent about two decades using binary system data storage as one of the 
reasons Windows sucks, that strikes me as particularly ironic.


The RC system we have is proof you do not need anything more than bourne 
and a respectable amount of intelligence to design a good init system.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


RE: libinit idea

2014-02-23 Thread dteske


 -Original Message-
 From: David Chisnall [mailto:thera...@freebsd.org]
 Sent: Sunday, February 23, 2014 10:18 AM
 To: Allan Jude
 Cc: FreeBSD Current
 Subject: Re: libinit idea
 
 On 23 Feb 2014, at 18:11, Allan Jude free...@allanjude.com wrote:
 
  sysrc solves this nicely, it is in base now, and is great for
  programmatically adding, removing and changing lines in rc.conf style
  files. It is also in ports for older versions of FreeBSD where it is
  not in base.
 
 The problem is, there is no such thing as an rc.conf style file.  rc.conf
is just a
 shell script.  If you only edit it with sysrc, or you are careful to
preserve the
 structure, then it's fine.

Actually, sysrc is well aware that rc.conf is a shell script and sysrc will
actually
make certain that structure is preserved (regardless of whether you've
placed
shell code in rc.conf or not).

 There is absolutely nothing stopping you, however,
 from writing arbitrarily complex shell scripts inside rc.conf.  Sure, it's
a terrible
 idea to do so, but when has that ever stopped anyone?
 

When sysrc is told to make a change, it first finds where it needs to make
that
change (/etc/rc.conf or /etc/rc.conf.local or one of other paths mentioned
in
rc_conf_files setting), it will refuse to edit the file *IF* the file
doesn't first
(before editing) pass a shell syntax check (using sh -n FILE). After
making the
changes in a temporarily file (produced with mktemp(1)), it again performs
yet
another syntax check (also using sh -n FILE) to make sure that it
continues to
yield clean syntax.

On top of that, it makes certain not to muck with dynamic assignments. So it
will leave-alone an assignment such as foo=$( bar ) if told to edit the
variable
foo -- instead opting to add a foo=newvalue after the dynamic assignment
(logic
is simple... sysrc wasn't responsible for putting that dynamic assignment
in, so
it is going to leave it as-is and put the new [static] assignment further
below.

So Allan was absolutely correct in stating that sysrc is for editing
rc.conf style
files -- shell scripts really. In essence, sysrc is actually a shell script
modifying
utility that takes extreme special care to rely on the fact that rc.conf
*is* a
shell script.

 An rc-replacement could enforce this by only accepting purely declarative
files
 for configuration, guaranteeing that if they were syntactically valid they
would
 also be machine editable, no matter what the user does to them.
 

sysrc does this for you.
-- 
Devin

_
The information contained in this message is proprietary and/or confidential. 
If you are not the intended recipient, please: (i) delete the message and all 
copies; (ii) do not disclose, distribute or use the message in any manner; and 
(iii) notify the sender immediately. In addition, please be aware that any 
message addressed to our domain is subject to archiving and review by persons 
other than the intended recipient. Thank you.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-23 Thread Allan Jude
On 2014-02-23 17:04, Warner Losh wrote:
 
 On Feb 23, 2014, at 11:17 AM, David Chisnall thera...@freebsd.org wrote:
 
 On 23 Feb 2014, at 18:11, Allan Jude free...@allanjude.com wrote:

 sysrc solves this nicely, it is in base now, and is great for
 programmatically adding, removing and changing lines in rc.conf style
 files. It is also in ports for older versions of FreeBSD where it is not
 in base.

 The problem is, there is no such thing as an rc.conf style file.  rc.conf is 
 just a shell script.  If you only edit it with sysrc, or you are careful to 
 preserve the structure, then it's fine.  There is absolutely nothing 
 stopping you, however, from writing arbitrarily complex shell scripts inside 
 rc.conf.  Sure, it's a terrible idea to do so, but when has that ever 
 stopped anyone?

 An rc-replacement could enforce this by only accepting purely declarative 
 files for configuration, guaranteeing that if they were syntactically valid 
 they would also be machine editable, no matter what the user does to them.
 
 We already have a rc.conf.default. Why not a rc.conf.automation that does 
 that and is added to the list of things to source? Then things like sysrc 
 could operation on that secure in the knowledge that no shell commands could 
 be there, and all bets are off if someone edits it by hand?
 
 Warner
 

This is basically what we do, we have puppet add:

rc_conf_files=/etc/rc.conf /etc/rc.conf.local /etc/rc.conf.scaleengine

to rc.conf, and then we push our global config to the .scaleengine file

-- 
Allan Jude



signature.asc
Description: OpenPGP digital signature


Re: libinit idea

2014-02-23 Thread Warner Losh

On Feb 23, 2014, at 11:17 AM, David Chisnall thera...@freebsd.org wrote:

 On 23 Feb 2014, at 18:11, Allan Jude free...@allanjude.com wrote:
 
 sysrc solves this nicely, it is in base now, and is great for
 programmatically adding, removing and changing lines in rc.conf style
 files. It is also in ports for older versions of FreeBSD where it is not
 in base.
 
 The problem is, there is no such thing as an rc.conf style file.  rc.conf is 
 just a shell script.  If you only edit it with sysrc, or you are careful to 
 preserve the structure, then it's fine.  There is absolutely nothing stopping 
 you, however, from writing arbitrarily complex shell scripts inside rc.conf.  
 Sure, it's a terrible idea to do so, but when has that ever stopped anyone?
 
 An rc-replacement could enforce this by only accepting purely declarative 
 files for configuration, guaranteeing that if they were syntactically valid 
 they would also be machine editable, no matter what the user does to them.

We already have a rc.conf.default. Why not a rc.conf.automation that does that 
and is added to the list of things to source? Then things like sysrc could 
operation on that secure in the knowledge that no shell commands could be 
there, and all bets are off if someone edits it by hand?

Warner

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: libinit idea

2014-02-22 Thread Scot Hetzel
On Sat, Feb 22, 2014 at 5:54 PM, Bruno Lauzé brunola...@msn.com wrote:
 https://github.com/brunolauze/libnit

 I know there's really big debate about init system but here's my tentative to 
 propose a new model to replace rc.

 Let's call it libinit but the name as no significance for now.

 I started coding a library with the following architecture.

 the main idea is to rewrite rc in C language.

 a utility called system would act a little bit like service command does.

 a folder would contains libraries instead of scripts inside [target]/etc/rc.d
 so we can add as many librairies a user desire and interlink the order of 
 each piece among all like in rc.

libraries don't belong in [target]/etc/rc.d, they would have to be in
{/usr,}/lib{exec,}/rc.d or ${PREFIX}/lib{exec,}/rc.d


-- 
DISCLAIMER:

No electrons were maimed while sending this message. Only slightly bruised.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org