Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Steve Litt
On Sat, 28 Dec 2019 04:24:40 +0200
Alex Suykov  wrote:

> Fri, Dec 27, 2019 at 07:23:09PM +0800, Casper Ti. Vector wrote:
> 
> > I also wonder if someone on this mailing list is interested in
> > actually implementing a cgroup-based babysitter as is outlined in
> > the post, perhaps packaged together with standalone workalikes of
> > the cgroup chainloaders (`create-control-group' etc) from nosh?  I
> > am still quite a newbie to actual system programming in C...  
> 
> https://github.com/arsv/minibase/blob/master/src/misc/runcg.c
> 
> It's really simple.
> 
> I don't think a tool like this has any actual uses, other than in
> arguments with systemd fans, but I guess that alone could justify
> its existance.

Yes and no. IMHO...

* Yes if somebody makes a separate executable whose inputs are through
  stdin, command line args, environment variables, and intermediate
  files, and whose actions consist of doing the right things with
  /sys/fs/cgroup/... or wherever, and which is run ONLY by a process'
  run script, and requires absolutely no change to existing s6 stack
  code.

* No if any code within the s6 stack must be changed. So much good
  software has gone bad trying to incorporate features for only the
  purpose of getting new users.

SteveT

Steve Litt 
December 2019 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread eric vidal
> https://github.com/arsv/minibase/blob/master/src/misc/runcg.c
> 
> It's really simple.
> 
> I don't think a tool like this has any actual uses, other than in
> arguments with systemd fans, but I guess that alone could justify
> its existance.

Thanks for this link this is really interesting.

Well, for me, cgroups is clearly a lack to s6. Using it for every process like 
systemd do have no sense, but in some case it can be useful to protect e.g an 
"over-eat" allocation memory by a program (in particular over a server).

As cgroups is a linux specific feature, s6-supervise should not take care about 
it even if the code is modified at compile time. This will increase the code of 
s6-supervise only for an "optional" features. But laurent you do what you need 
to do.

I would prefer an extra layer of supervision and independent from s6 uniquely 
used when it necessary.

I think about it from a while for 66. Adding a section like [cgroups] 
containing all @key field needed to configure the future cgroups environment is 
not a big deal and allow user to quickly define what it need. In such case the 
"extra supervision layer" is added to the run scripts  which execs the e.g 
runcg program. 
But maybe i'm totally wrong with my thought...
-- 
eric vidal 


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Alex Suykov
Fri, Dec 27, 2019 at 07:23:09PM +0800, Casper Ti. Vector wrote:

> I also wonder if someone on this mailing list is interested in actually
> implementing a cgroup-based babysitter as is outlined in the post,
> perhaps packaged together with standalone workalikes of the cgroup
> chainloaders (`create-control-group' etc) from nosh?  I am still quite
> a newbie to actual system programming in C...

https://github.com/arsv/minibase/blob/master/src/misc/runcg.c

It's really simple.

I don't think a tool like this has any actual uses, other than in
arguments with systemd fans, but I guess that alone could justify
its existance.


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Oliver Schad
On Sat, 28 Dec 2019 00:54:07 +0100
Oliver Schad  wrote:

> Disclaimer: this has race-conditions by design. systemd has them as
> well. No, they don't say that of course. You can't read the tasks
> atomically and change their state to stopped, freeze or whatever. So
> they always could fork away.

Sorry, I have to correct that a bit: you can use the freezer cgroup as
I did, but there is no guarantee, that you can successfully kill a
freezed process.

So you can decide:

1) race-condition
2) unkillallable processes without race-condition

-- 
Automatic-Server AG •
Oliver Schad
Geschäftsführer
Turnerstrasse 2
9000 St. Gallen | Schweiz

www.automatic-server.com | oliver.sc...@automatic-server.com
Tel: +41 71 511 31 11 | Mobile: +41 76 330 03 47


pgpxtDHqyWIuU.pgp
Description: OpenPGP digital signature


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Oliver Schad
On Fri, 27 Dec 2019 12:32:27 +
"Laurent Bercot"  wrote:

> As for cgroups-related chainloaders, I could probably write some in
> s6-linux-utils, but wasn't the cgroups interface designed to make
> sure those operations are trivial to implement as small scripts?

I changed in the past sysv init scripts in exactly that way, that they
created a cgroup first at start and killed all processes within that
cgroup at the end.

That where 5? lines of shell. You could provide that system wide, if you
would offer some include mechanism (pre-run, post-stop) or within the
run/finish scripts just include a shell library.

In the first place it would be ok, to have the name of the service
available as an identifier for the cgroup. A random name would only
work if you persist it somewhere and have to manage the clean-up - I
would prefer to use the directory's name of the run script. Is it
available through a environment?

Something like:


##
run
##
CGROUP_BASE_DIR=/sys/fs/cgroup/freezer
cgroup=$run_dir

mkdir $CGROUP_BASE_DIR/$cgroup
echo $$ > $CGROUP_BASE_DIR/$cgroup/tasks

# exec or include
exec do_the_real_stuff



##
finish
##
CGROUP_BASE_DIR=/sys/fs/cgroup/freezer
cgroup=$run_dir
state_file=$CGROUP_BASE_DIR/$cgroup/freezer.state

echo FROZEN > $state_file
i=0
while ! grep FROZEN $state_file > /dev/null; do
  let i=$i+1
  sleep 0.1
  if [ $i -gt 100 ]; then
break
  fi
done
kill -9 $(cat $CGROUP_BASE_DIR/$cgroup/tasks)



Disclaimer: this has race-conditions by design. systemd has them as
well. No, they don't say that of course. You can't read the tasks
atomically and change their state to stopped, freeze or whatever. So
they always could fork away.

What you can do is repeat the killing/freezing/stopping until it
succeeds (mabye never).

Best Regards
Oli

-- 
Automatic-Server AG •
Oliver Schad
Geschäftsführer
Turnerstrasse 2
9000 St. Gallen | Schweiz

www.automatic-server.com | oliver.sc...@automatic-server.com
Tel: +41 71 511 31 11 | Mobile: +41 76 330 03 47


pgpnNc8oL23OG.pgp
Description: OpenPGP digital signature


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Alex Suykov
Fri, Dec 27, 2019 at 04:29:13PM -0500, Steve Litt wrote:

> What is slew?

I'd guess this thing: https://gitlab.com/CasperVector/slew


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Steve Litt
On Fri, 27 Dec 2019 21:54:11 +0800
"Casper Ti. Vector"  wrote:

> On Thu, Dec 26, 2019 at 09:57:35PM -0500, Steve Litt wrote:
> > Very, very nice! You should publicize this.  
> 
> .
> It seems to be downvoted, which may be unsurprising given the generic
> sentiment toward systemd in r/linux.  Anyway, as of now I cannot load
> the post page (perhaps because of the Great Firewall, but somehow I
> can partially load the r/linux subreddit page), so I am not sure.

You're not the first to be downvoted. Mahatma Gandhi, Martin Luther
King, Cesar Chavez, Nelson Mandela, and Abraham Lincoln were vigorously
downvoted before they succeeded and raised the collective
consciousness. I'm not equating init system freedom with human freedom:
I'm just saying that it's not unusual for people with great ideas to be
downvoted.

Also, it looked to me like your story was removed by a bot, one of the
reasons being that your account was less than 5 days old. Things might
be different six days from now.

So I repeat: Very, very nice! You should publicize this.

Again and again.

SteveT

Steve Litt 
December 2019 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Steve Litt
On Fri, 27 Dec 2019 01:52:58 +0800
"Casper Ti. Vector"  wrote:

> In addition to providing arguments suitable as stock replies to
> systemd proponents, that post also contains steps to build a small
> example system based on s6/s6-rc/slew, which updates the Ubuntu
> example in ,
> and may be useful to people interested in distro packaging of slew.

What is slew? I searched it for 10 minutes, hit two acronym servers,
and based on context figuring it might be configuration management
software, looked up puppet chef ansible salt slew but got nothing but
the word "a slew of" meaning "a lot of".

Is slew software in the s6 or s6-rc stack, or is it something else?

Thanks,

SteveT

Steve Litt 
December 2019 featured book: Rapid Learning for the 21st Century
http://www.troubleshooters.com/rl21


Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Casper Ti. Vector
On Fri, Dec 27, 2019 at 09:54:11PM +0800, Casper Ti. Vector wrote:
> .
> It seems to be downvoted, which may be unsurprising given the generic
> sentiment toward systemd in r/linux.  Anyway, as of now I cannot load
> the post page (perhaps because of the Great Firewall, but somehow I can
> partially load the r/linux subreddit page), so I am not sure.

Now I know the reason.  "Your account is too young.  Please wait
at least 5 days to begin posting." --- /u/AutoModerator

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Casper Ti. Vector
On Thu, Dec 26, 2019 at 09:57:35PM -0500, Steve Litt wrote:
> Very, very nice! You should publicize this.

.
It seems to be downvoted, which may be unsurprising given the generic
sentiment toward systemd in r/linux.  Anyway, as of now I cannot load
the post page (perhaps because of the Great Firewall, but somehow I can
partially load the r/linux subreddit page), so I am not sure.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Casper Ti. Vector
On Fri, Dec 27, 2019 at 12:32:27PM +, Laurent Bercot wrote:
> Is there real pressure to have this?

AFAIK, the only pressure is from systemd fanboys.  But this is indeed
a biggest criticism from them; we would be able to save quite a lot of
flamewars if the feature was simply there.  Nevertheless I understand
the feature will be, frankly, a vase.

> The problem with such a "babysitter" is that it would need to forward
> signals, much like execline's trap program. It's ugly, and I'd rather
> have people not do that any more than strictly necessary.

We will also need to handle disgusting PID files for double-forking
services.  And in order to be safe in case the service crashes before
the PID file is created, we will perhaps need some kind of startup
deadline.  A big can of worms.

> As for cgroups-related chainloaders, I could probably write some in
> s6-linux-utils, but wasn't the cgroups interface designed to make
> sure those operations are trivial to implement as small scripts?

Well, this is a good idea.  I can even provide such library scripts in
slew, but the libraries will be `rc'-specific, and not used in the
traditional (exec()-based) sense of chainloading.

> I don't, for several reasons, one of which is that Google's homemade
> supervisor (which is... not great) is called "babysitter", and it
> triggers cringey memories.

:)

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Laurent Bercot

I also wonder if someone on this mailing list is interested in actually
implementing a cgroup-based babysitter as is outlined in the post,
perhaps packaged together with standalone workalikes of the cgroup
chainloaders (`create-control-group' etc) from nosh?


Is there real pressure to have this?
The problem with such a "babysitter" is that it would need to forward
signals, much like execline's trap program. It's ugly, and I'd rather
have people not do that any more than strictly necessary.

If there's important pressure to have cgroups support, I will probably
end up applying some version or another of jlyo's patch to s6-supervise,
which makes s6-supervise itself the babysitter. That would allow the
supervised process to operate as usual.

The reason why I didn't want to apply this patch in the first place is
that it's Linux-specific, so it would introduce divergent behaviour in
s6 depending on the system it's running on. But it's probably workable
with some build-time + run-time configuration, I need to think more
about this.

As for cgroups-related chainloaders, I could probably write some in
s6-linux-utils, but wasn't the cgroups interface designed to make
sure those operations are trivial to implement as small scripts?



[And I really like using the word "babysit" here, which comes with a
nice degree of derogatoriness without being excessive.]


 I don't, for several reasons, one of which is that Google's homemade
supervisor (which is... not great) is called "babysitter", and it 
triggers

cringey memories.

--
 Laurent



Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Casper Ti. Vector
On Thu, Dec 26, 2019 at 07:17:02PM +, Laurent Bercot wrote:
> That is awesome, and you are doing very important work.

Thanks :)

> Could somebody volunteer to track down all the similar documents we have,
> and make a list of links on a "meta" page?

I also wonder if someone on this mailing list is interested in actually
implementing a cgroup-based babysitter as is outlined in the post,
perhaps packaged together with standalone workalikes of the cgroup
chainloaders (`create-control-group' etc) from nosh?  I am still quite
a newbie to actual system programming in C...

[And I really like using the word "babysit" here, which comes with a
nice degree of derogatoriness without being excessive.]

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



Re: The "Unix Philosophy 2020" document

2019-12-27 Thread Casper Ti. Vector
On Fri, Dec 27, 2019 at 02:09:48AM +0100, Oliver Schad wrote:
> OMG - how much work was that? Great!

The post is mostly based on materials from the UP2020 document, but
optimised for length and with information added here and there; the
"Linux cgroups" and "Myths" parts are largely new materials.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C