Re: On the feasibility of a shell that includes execline features

[Casper Ti. Vector]

Sorry for bumping this dormant thread, but I found scsh [1] really
interesting (possibly even more so than rc(1)) in this aspect: written
as a Scheme-embedded shell, it can make full use of the Scheme language,
which is both elegant and expressive (unrivaled by any other programming
language I know IMHO); using its syscall interfaces, chainloading would
be fairly easy to implement.

[1] .

There are, inevitably, downsides:
* As of its latest release (v0.6.7 from 2006), scsh does not have a good
  interactive mode.
* scsh is, arguably, not very lightweight:
> -rw-rw-r-- 1 portage portage 9.0M Sep 15  2016 bash-4.4.tar.gz
> -rw-rw-r-- 1 portage portage 211K May 14  2015 rc-1.7.4.tar.gz
> -rw-rw-r-- 1 portage portage 4.2M Aug 31  2006 scsh-0.6.7.tar.gz
  However, considering the power of scsh as a language ("scsh is a Scheme
  system designed for writing useful standalone Unix programs and shell
  scripts: it spans a wide range of application, from 'script'
  applications usually handled with perl or sh, to more standard system
  applications usually written in C" [2]), I think the size of its
  codebase is justified.

[2] .

On Sun, Aug 21, 2016 at 11:00:43PM +0800, Casper Ti. Vector wrote:
> What do we need
> ---
> 
> Unfortunately, I am not a system programmer, and do not my current time
> schedule allow me to spend enough time to systematically learn it; but I
> think that a language that combines the advantages of shell and execline
> is not only a concept, but also a feasible and rewarding goal, which is
> worth a Unix programmer's efforts.  Thus I really wish somebody that is
> interested in process supervision and has the resource to try to realise
> this concept, probably by incorporating execline utilities into rc(1).

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On the feasibility of a shell that includes execline features

[Casper Ti. Vector]

This is a valid and important point; but I do think the arguments in the
`dieshdiedie' (well the name...) page are, though incisive wrt sh(1),
mostly unfair to the shell language.

I understand your pursuit of simplicity in the implementation, and agree
that the overhead is unavoidable; but let's agree to disagree on whether
the overhead is unsatisfying: we are just choosing different balance
between implementational simplicity (from the upstream's viewpoint) and
linguistic power (from the downstream's viewpoint; and not just saving a
few exec()s, really).

A similar example of this kind of discrepancy can be seen in [1]:
supporting /dev/log as SOCK_STREAM requires delicate handling of
fallback issues etc from the upstream, but can enable elegant
applications like [2] on the downstream; the musl developer and you just
chose different balance on that issue.

[1] .
[2] .

On Mon, Aug 22, 2016 at 10:13:38PM +0200, Laurent Bercot wrote:
>   So, yes. In a shell you have no choice: you need to be persistent. And so,
> if you implement "chain-loading" functions as shell builtins, you may
> save a few execs, but the code for your builtins will be significantly more
> complex than what they are for instance in execline, because you'll be
> duplicating the work of the kernel. No matter how well-designed the "rc"
> family of shells is, it's unavoidable and unsatisfying overhead.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On the feasibility of a shell that includes execline features

[Laurent Bercot]

On 21/08/2016 17:00, Casper Ti. Vector wrote:

In [1], I (probably after many other people with similar ideas) came up
with the idea of a "language that somehow combines execline-like
chainloading and shell-like sequential execution".


 I understand how the idea is tempting; and if someone wants to try their
hand at writing something like that, it's certainly an interesting goal.

 However, I think you'd find the benefits aren't quite what they seem
to be at first sight. You'd probably end up with something that is more
complex than the sum of a shell and execline. And the reason for that is
the fundamental difference between a persistent process and one that is
not.

 There is an extremely measurable benefit in writing chain-loading programs,
as far as maintainability is concerned: you do not need to bother with
data structure cleanup at all. You can be incredibly careless with your data,
have pointers into the stack, let entire trees leak - all this doesn't
matter because you're going to execve() right away and the kernel is going
to clean up everything for you. You're not going to have dangling pointers
and you're not going to leak memory. If you have sensitive data, it's gone,
too. (At least it should be. I hope security-conscious kernels actually
wipe the allocated physical pages before giving them to other processes.)

 I use this property a lot, in particular to avoid pulling in malloc()
when I can store all my data into a stack that would be popped in a
persistent program, but won't be here because execve(). This is a really big
factor in making C programs very simple to write and maintain, and also
very resilient - and very efficient. execve() may be slow, but it's powerful,
and the kernel is a wonderful janitor.

 Look at
http://git.skarnet.org/cgi-bin/cgit.cgi/execline/tree/src/execline/export.c :
it doesn't get any simpler than that.

 In a persistent process, you can't afford to do that: you have to keep a
clean accounting of your resources. First, you initialize your runtime,
which is in itself some unavoidable boilerplate, even if your task is
really simple (whereas a launcher such as execlineb doesn't have a runtime
to initialize). Then you loop; for every iteration of the loop, you do some
task that needs to communicate with other tasks; so you allocate resources
whose lifetime may exceed the lifetime of the task. So you can't use a
stack, and you can't yolo allocations in the wild - you have to perform
accurate resource management. Which is what people hate about C, and I
can't blame them.

 Busybox does that. It's painful. It's one of the reasons why toybox is
smaller.

 So, yes. In a shell you have no choice: you need to be persistent. And so,
if you implement "chain-loading" functions as shell builtins, you may
save a few execs, but the code for your builtins will be significantly more
complex than what they are for instance in execline, because you'll be
duplicating the work of the kernel. No matter how well-designed the "rc"
family of shells is, it's unavoidable and unsatisfying overhead.

--
 Laurent

Re: On the feasibility of a shell that includes execline features

[Casper Ti. Vector]

Please forgive my gross carelessness :(

On Sun, Aug 21, 2016 at 11:00:43PM +0800, Casper Ti. Vector wrote:
>   > $ for i in $(seq 500); do loopwhilex /bin/true /bin/true & done
>   > $ for i in $(seq 100); do rc -c 'while (/bin/true) /bin/true' & done
>   and
>   > $ for i in $(seq 100); do ash -c \
>   > 'while /bin/true; do /bin/true; done' & done
s/100/500/g

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On the feasibility of a shell that includes execline features

[Casper Ti. Vector]

On Sun, Aug 21, 2016 at 11:00:43PM +0800, Casper Ti. Vector wrote:
> simple and clear (i.e. not kludgy) shell scripts
To make it clear: "kludgy" is meant in comparison with other shell
scripts, not execline scripts.

> # Think of `redirfd -w /dev/null'.
... should obviously be `redirfd -w 2 /dev/null'.

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C

Re: On the feasibility of a shell that includes execline features

[Casper Ti. Vector]

Always forgetting something :|
This time the attachment...

On Sun, Aug 21, 2016 at 11:00:43PM +0800, Casper Ti. Vector wrote:
> Attached is a simplified tarball of the config of my Alpine server, plus
> instructions for the setup

-- 
My current OpenPGP key:
RSA4096/0x227E8CAAB7AA186C (expires: 2020.10.19)
7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C



s6rc-conf.tgz
Description: GNU Unix tar archive