Re: execline-in-execline

2018-06-07 Thread Guillaume Perréal
As a matter of example : this scripts "rewrites" its final command 
depending on optional arguments. Its purpose is to restore select parts 
of the environment variables, uid/gid and working directory, that were 
saved in a directory at some point in the past.


https://github.com/Adirelle/s6rc-overlay/blob/master/src/sbin/with-contenv

Maybe it is pushing execline a bit too far but I didn't fell like coding 
it in C (which I do no master).


Le 07/06/2018 à 17:59, Profpatsch a écrit :

Laurent Bercot writes:


  Remember that once an execlineb script has been parsed, it's just a
command line, no more, no less. So your example script can just be
written as:

#!execline
define url example.com
s6-tcpclient $url 80
foreground { fdmove 1 7 echo -en "..." }
fdmove 0 6 cat

  No second execlineb invocation necessary at all. No quoting
  nightmares.

Ooh, you are right!
It’s even more elegant than I thought!
Nice.





Re: Debianization

2018-04-21 Thread Guillaume Perréal

Le 21/04/2018 à 12:28, Didier Kryn a écrit :
    May I humbly ask why you don't start with Devuan (which is Debian 
stripped out of systemd)? Of course it is less challenging than doing 
the job on Debian, but it would be wholehartedly applauded and tested 
by the Devuan community, while you shouldn't expect much recognition 
from Debian.


            Didier
I have tried to boot an Alpine Linux with s6-init and the set of of the 
services I wanted and it is quite a lot a work. I think writing a full 
replacement with shims and "translators" is an order of magnitude higher.


Guillaume.


Re: tests

2018-01-11 Thread Guillaume Perréal

Le 11/01/2018 à 16:44, Laurent Bercot a écrit :

 Oh, definitely, and s6 is already used in numerous Docker containers.
Most people who want to do that use the s6-overlay project:
https://github.com/just-containers/s6-overlay

 Some big companies (Badoo, for instance) rely on s6 to power their
containers, so I think it's a good testimony to the fact that it works :)

I also use it to run my X sessions: it monitors background services and 
applets, as well as some "front-end" applications (browser, mailer, ...) 
that tends to die often.

--
Guillaume.


Re: http://skarnet.org/toolchains/cross/ not found

2017-11-02 Thread Guillaume Perréal

Le 02/11/2017 à 07:52, Laurent Bercot a écrit :

 I don't have MIPS toolchains :/
 If you know the exact gcc building flags needed to make one, I can
easily build one. And if you know the correct set of sysdeps to
cross-build skalibs on MIPS (ramips or other), I'll add support to
lh-bootstrap. But it's not ready out-of-the-box yet, because I haven't
had need or opportunity to use this arch, and neither have the people
who use lh-bootstrap.
Ok, nevermind. I am not sure about the flags nor the sysdeps. There are 
working, openwrt-based images for those platforms (see 
https://github.com/gnubee-git/GnuBee-lede/tree/lede-17.01.4) and AFAIK 
openwrt builds its cross-compiled toolchain as part as the whole 
building process. However, I do not know if and how I could use it to 
get the flags or the sysdeps, and it might be very specific to that 
platform. So do not waste your time with that.


Thanks,
--
Guillaume.


http://skarnet.org/toolchains/cross/ not found

2017-11-01 Thread Guillaume Perréal

Hello Laurent,

I came across this http://skarnet.org/software/lh-bootstrap/. As I am 
interested, I have looked for a x86_64-to-ramips toolchain, but 
http://skarnet.org/toolchains/cross/ gives me a 404. The same goes for 
the native toolchains.


Regards,

--
Guillaume Perréal.



Re: .env file handling

2017-10-24 Thread Guillaume Perréal

Le 24/10/2017 à 08:28, Colin Booth a écrit :

I would say that is one of the two difficulties. The other one being
that execline also tries hard not to carry any overhead, which means
that often times you can end up in situations where the aggressive
scoping that it does makes things challenging at best (assuming you're
trying to stay with the spirit of the language).
The scoping might be a bit challenging, but the variable substitution 
system is royal. I do not constantly ask myself "what will happen if 
this variable contains a space, or a quote ?".


--
Guillaume.


Re: execline misc

2017-04-17 Thread Guillaume Perréal
They could go in a page titled "how to use execline as a startup shell" but 
honestly I do not think many sysadmins which would use them, given few 
desktop/server distros ship execline in the first place.

- Mail original -
> De: "Laurent Bercot" 
> À: skaware@list.skarnet.org
> Envoyé: Dimanche 16 Avril 2017 17:57:03
> Objet : Re: execline misc
> 
> Ah, yes. I removed the execline-shell and execline-startup scripts
> because they were basically out of scope and confusing to the user.
> I forgot to remove the documentation pages. I'm inclined to remove
> the
> documentation pages too, but if you prefer, I guess I could put the
> scripts back in instead, in an examples/ subdirectory.
> 
>   What do you think?
> 


Re: Man pages

2017-03-31 Thread Guillaume Perréal
Pandoc (http://pandoc.org/) might be useful. The out-of-the-box template 
is ugly, but nonetheless usable.


Regards,

Guillaume.

Le 26/03/2017 à 08:18, Colin Booth a écrit :

On Mar 25, 2017 10:01 PM, "S. Gilles"  wrote:

This is a rather silly question, but the other day I wanted to look
up the syntax of some command or other, but had no internet connection.
I had always assumed that the online documentation was generated
from manpages, but I don't see any in the source.

Am I overlooking a repository, or are the docs HTML only?  (And if
the latter, would hypothetical patches to add manpages be accepted?)

Docs are HTML only but are shipped with the source
(component_name/docs/*.html) and should be present on all systems that
build skaware stuff. This has been mentioned before in the #s6 IRC channel,
and yes, patches to add manpages would be accepted. I've threatened to do
the same, that or to rewrite the docs into something that trivially
compiles into both html and man-style troff, but a lack of time (and the
presence of a copy of lynx on all of my systems that run s6) has kept me
from really digging in to the project.

Cheers!





Configuration and scripts location

2017-02-22 Thread Guillaume Perréal

Hello,

Tweaking my services, I have wondered where to put their configuration.

For longruns, one could either put the in the data/ and env/ 
subdirectories, or in a /etc subdirectory. The former allows to easily 
have different instances of the same services but the files might be 
harder to locate, and requires to update the s6-rc database to change 
the configuration. The latter allows to use a well-known location and to 
update the configuration without updating the s6-rc database.


For oneshots, as one cannot use neither env/nor database. There is no 
much solutions.


The same question goes for helper scripts (for example, the "event 
handler" of udhcpc, responsible for applying the configuration). I'm 
torn between putting them in s6-rc/scripts/the-service-that-uses-them, a 
"scripts" subdirectory in their configuration, or something like 
{/lib,/usr/share}/the-service-that-uses-them. Moreover, considering that 
the service scripts are moved around, use of absolute paths is required.


What are you doing, and why ?

Regards,

--

Guillaume Perréal.





Re: execline and $0-based stuff

2017-02-22 Thread Guillaume Perréal
I also discovered that the "expr" command from busybox have some regexp 
support, e.g :


expr 'match' '/some/path/getty-tty1/run' '.*/getty-\([^/]*\)/run'

Exits with 0 and prints "tty1".

While this doesn't print anything and exits with 1 :

expr 'match' '/some/path/not-a-tty/run' '.*/getty-\([^/]*\)/run'

I have two other questions but I'll start proper mail threads.

--

Guillaume.


Le 22/02/2017 à 03:03, Casper Ti. Vector a écrit :

An example of my attempt, already working on at least 2 Alpine machines:


$ ls -d /etc/s6-rc/main/*getty*
/etc/s6-rc/main/getty.tty1
$ cat /etc/s6-rc/base/getty/run
#!/bin/rc -e
exec >[2=1]
. /etc/s6-rc/bin/fn.rc
exec `{cdr $self} 38400 $name linux
$ cat /etc/s6-rc/bin/fn.rc
#!/bin/rc
fn car { echo $1 | sed 's/^.*\.//' }
fn cdr { echo $1 | sed 's/\.[^.]*$//' }
# ... [code used in other templates] ...
self=`{basename `{pwd}}
if (~ $self *.log) {
 logd=/var/log/`{cdr $self}
} else name=`{car $self}
# ... [code used in other templates] ...

On Sat, Feb 18, 2017 at 01:46:20PM +0100, gperr...@free.fr wrote:

extract a part of the path of the executed script ($0) in a variable




Re: s6-linux-init: /etc/rc.tini not executed

2017-02-02 Thread Guillaume Perréal

Le 02/02/2017 à 14:46, Colin Booth a écrit :

Have I overlooked/misunderstood something ?

Make sure you are running s6-svscan with the -s option.
Already checked: it is. The /init script was created by 
s6-linux-init-maker and I barely touched it.


I do not know why s6-svscan does not exec its signal handlers.

--
Guillaume.


s6-linux-init: /etc/rc.tini not executed

2017-02-02 Thread Guillaume Perréal

Hello,

I finally have a working initramfs and know the system is happily 
booting. I am slowly adding one-shots and services to build a functional 
server


However, it seems there is an issue with poweroff :

If I manually launch /run/s6/services/.s6-svcan/SIGUSR1, everything is 
fine. /etc/rc.tini is executed first, shutting down s6-rc, then 
/etc/rc.shutdown.


But when I use s6-poweroff, it seems s6-svscan skips directly to 
/etc/rc.shutdown, omitting /etc/rc.tini.


Have I overlooked/misunderstood something ?

The source of my scripts are there, if one would like to check them: 
https://github.com/Adirelle/s6-alpy/blob/master/rootfs/etc


--

Guillaume.



Re: s6-linux-init, alpine linux, and initramfs

2017-01-31 Thread Guillaume Perréal

Thank you very much !

This have helped me a lot.

I had to mount /dev and to resort to busybox's switch_root because the 
one built using execline tools had trouble executing. Despite using 
executables from the actual root filesystem, it had issue spawing tools 
at some point in the loop. I guess this is because they are not 
statically compiled.


One improvement would be to use the kernel commandline to select the 
modules to load and detect the root filesystem. Right now I just use 
hard-coded values.


The "modules=" option is added as an environment variable by the kernel 
so this should do the trick:


import -d, -s modules modprobe -a ${modules}

The "root=" seems a bit tricker: it is only available through parsing 
/proc/cmdline. I think something like this will do it:


backtick -ni ROOTFS {
pipeline {
forbacktickx OPT { redirfr -r 0 /proc/cmdline s6-cat }
import -u OPT
s6-echo $OPT
}
pipeline { s6-grep ^root= }
s6-cut -d= -f2-
}
import -u ROOTFS

backtick -ni ROOTFSTYPE { blkid -s TYPE -o value $ROOTFS }
import -u ROOTFSTYPE

I have seen alpine's intit script use KOPT_* variables (like KOPT_root, 
KOPT_quiet and so on) but I have found no reference to them nor have I 
found them in the environment of process 1.


Now i will take a look at s6-rc examples scripts.

--

Guillaume.


Le 30/01/2017 à 15:19, Laurent Bercot a écrit :


2) have a very small init script to load the modules, mount the 
filesystems (/dev, /proc, /sys, /), and finally pivot-chroot into 
s6-linux-init phase 1. This would be less elegent but it might be 
easier to set up.


 This. If you need or want an initramfs, you need to comply with the
implicit initramfs contract: when you exec into /sbin/init, it must be
the only process running on your machine, just as if the kernel started
it; and any init system should be able to work, the initramfs should not
tie you to a specific init. So, spawning a supervision tree in the
initramfs is a no-no, because it breaks both aspects.

 You can see an initramfs as a mini-system that you set up to do what
needs to be done *and then tear down* before exec'ing into the real 
system.

So, do that: load your modules, find your rootfs, pivot-chroot into it,
and start your real system with your init of choosing.

 Ideally, you'd even unmount /proc and /sys (which you likely need during
your initramfs execution) before entering /sbin/init. But obviously
that's not practical since your boot sequence will mount then again
very soon, so the separation between "pre-init" and "post-init" can
be a bit less strict. You can document that the state of your system
at init time is "pristine as if the kernel had directly started init,
except that /proc and /sys are already mounted", for instance, and
that's acceptable. (/dev is not even a question - you should have a
devtmpfs mounted at boot anyway, and mount --move it after your
pivot_root.)

 But "after initramfs, I already have a supervision system and just
need to run the rest of the boot sequence" is not acceptable - if only
because you then need to keep supervision executables in RAM, and a
component of your pivoted initramfs in your PATH.

 Just make initramfs as transparent as possible, it's a lot cleaner.

 Oh, by the way, pivot_root works with initrd, but not initramfs: see
https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt 


so you can use busybox/toybox's switch_root instead, or you can do the
switch_root by hand.

 I have a skeleton /init here that only needs in your initramfs:
 - empty /dev, /proc, /rootfs and /sys directories. OK if the kernel
mounts devtmpfs at boot (which it should).
 - a /sbin directory with a "mdev" static binary inside. (busybox with
only mdev selected will still be 100ish kB, that's unfortunately normal.)
 - a /command directory with static cd, execlineb, export, foreground,
if, redirfd, s6-echo and s6-mount binaries inside. Also "define" for the
skeleton but you'd replace it with something else for your rootfs 
detection.

 - whatever else you need to do your job - you could add modutils to your
busybox build, for instance, if you want to load modules. You may want
a /etc/mdev.conf depending on the devices you're expecting to detect.
 - also execline, s6-portable-utils and s6-linux-utils binaries 
accessible

in the /command directory of your real root filesystem.

 You can get it at http://pastebin.com/KZfdETy5
 My gzipped initramfs image made with that is about 104kB, for x86_64.

 HTH,

--
 Laurent





s6-linux-init, alpine linux, and initramfs

2017-01-30 Thread Guillaume Perréal

Hello there,

Me again. After tweaking my Xsession for a stater, I am trying to build 
a VM with s6-linux-init.


I am starting from Alpine Linux, because I am not into recompiling Linux 
and all tools from scratch (well, not yet) and this distro already 
provides binaries for all skarnet tools. Like many distro, they use an 
initramfs, because most of the drivers (including sd-mod, scsi and ext*) 
are built as modules.


I have found how to customize the initramfs. Now I am facing a choice: 
what should I put in the initramfs ?


1) put s6-linux-init phase 1 into the initramfs, use it at /init, then 
use an embedded /etc/rc.init to load modules, mount / and exec into the 
root's /etc/rc.init. The advantage would be a full s6 boot process. One 
drawback is that I have to put all execline and s6 tools (but s6-rc) in 
the initramfs. Another one is that the phase 1 of s6-linux-init is not 
very verbose and does not have any emergency fallback.


2) have a very small init script to load the modules, mount the 
filesystems (/dev, /proc, /sys, /), and finally pivot-chroot into 
s6-linux-init phase 1. This would be less elegent but it might be easier 
to set up.


Any idea on this ?

I know the right way would be to recompile linux with the right modules 
to boot directly into s6-linux-init phase 1 from the root partition.


++

--

Guillaume.



Re: s6-rc:

2017-01-27 Thread Guillaume Perréal

Thank you. I used a simplfied version, since I have no logging concerns:

  # Precreate the control FIFO
  s6-mkfifo -m 0600 $S6_SCANDIR/.s6-svcan/control

  background {
if {
# Block until s6-svscan starts reading its control FIFO
redir -w 8 $S6_SCANDIR/.s6-svcan/control
# Do not leak the fd
fdclose 8
s6-rc-init -c $S6_RC_COMPILED -l $S6_RC_LIVE $S6_SCANDIR
}
s6-rc $VERBOSE -l $S6_RC_LIVE -u change session
}

s6-svscan -st 0 $S6_SCANDIR

This is a bit "dirty" because it depends on s6-svscan internal, i.e. it 
creates a FIFO named "control" in its .s6-svscan directory.


--
Guillaume.

Le 27/01/2017 à 11:41, Casper Ti. Vector a écrit :

Using the fifo trick [1]?  As a side note, though the s6 documentation
considers it "dirty", similar methods [2] are widely used for process
synchronisation in CSP-style concurrency.

[1] .




s6-rc:

2017-01-27 Thread Guillaume Perréal

Hello,

Trying to use s6-rc, I have run into something that looks like a race 
condition between s6-rc-init and s6-svscan.


The context: as a concrete project to test s6-rc, I am using it to 
manage my X session, mainly for setup and supervising background tasks. 
So I want s6-svscan to be the "root" process of my X session.


My first try what this (roughly, variables are self-explanatory and 
"session" is my "root" bundle):


background { s6-svscan -st 0 $S6_SCANDIR }
if { s6-rc-init -c $S6_RC_COMPILED -l $S6_RC_LIVE $S6_SCANDIR }
s6-rc $VERBOSE -l $S6_RC_LIVE -u change session

The issue is that once s6-rc ends, s6-svscan is "reparented" to init and 
I want to keep it attached the the login manager, so I refactored it to 
this:


background {
if { s6-rc-init -c $S6_RC_COMPILED -l $S6_RC_LIVE $S6_SCANDIR }
s6-rc $VERBOSE -l $S6_RC_LIVE -u change session
}
s6-svscan -st 0 $S6_SCANDIR

Now it works as intended but s6-rc-init sometimes complains that 
$S6_SCANDIR is not supervised (i.e. s6-svscan is not ready). I think 
this could probably happen with the first solution too.


I have added "s6-sleep 1" before s6-rc-init to fix it, but I am 
wondering if there is an more elegant yet simple way to handle this ?


Regards,

--

Guillaume.



s6-rc compilation issue

2017-01-24 Thread Guillaume Perréal

Hello there,

I have a small issue to compile s6-rc.

I am trying some skarnet tools (execline, s6 and s6-rc). As I want to 
test them and to install and uninstall them easily, I use GNU Stow 
(https://www.gnu.org/software/stow/). I have chosen to install them into 
/usr/local/stow/skarnet.


I have successfully built and installed skalibs, execline and s6 using 
the following commands:


./configure --prefix=/usr/local/stow/skarnet
make
sudo make install

But when trying to build s6-rc, it gives me this error :

> make
make: *** No rule to make target '-ls6', needed by 
's6-rc-fdholder-filler'.  Stop.


This can be fixed be creating a symlink :

ln -s /usr/local/stow/skarnet/lib/s6/libs6.a 
/usr/local/stow/skarnet/lib/libs6.a

stow -d /usr/local/stow -R skarnet

So I guess there is a lib path missing somewhere, but as I have never 
written any configure script, I am not sure how to fix it. How should it 
be fixed ?


Best regards,
Guillaume Perréal.