Re: Environment of expansions and visibility of side-effect assignments

[Casper . Dik]

>On Solaris, vfork() is 3x faster than fork(),

This depends very much on application.

In Solaris, vfork() is more or less O(1) where fork() is a O(n)
where "n" is the size of the address space: each r/w page which isn't 
shared needs to be marked "read-only" so the first modification clones the 
page to a writable copy.

With shared library and the dynamic linker, there is very little that you 
can do between vfork() to exec/_exit in the child.

The current Solaris implementation of posix_spawn() does use vfork() and 
it is known that its use is safe.  For others this is less so.

Ok, vfork() isn't completely O(1): it needs to halt all the other 
threads executing in the same address space, basically it stops to world

Casper

Re: Environment of expansions and visibility of side-effect assignments

[Joerg Schilling]

Jilles Tjoelker  wrote:

> > >From my metering, I know that on Solaris vfork() is 3x faster than fork()
> > even tough the fork() implementation in SunOS uses CopyOnWrite since 1988.
>
> > As a result, I typically see aprox. 30% better performance with a 
> > "configure" 
> > run when vfork() is used.
>
> That difference is a bit higher than I saw in "real" benchmarks on
> FreeBSD, but the differences I saw were still significant, both on PC
> and embedded hardware. I think using vfork() somehow is definitely
> interesting for a scripting shell.

It would be of interest to me what actual difference between a fork() and a 
vfork() call you see on FreeBSD.

On Solaris, vfork() is 3x faster than fork(), but on Linux both are of the same 
speed as Linux only implements the "pitfalls" from vfork() but not the 
advantages that mean:

the address space description of the child is not a copy of the address 
space description from the parent, but the child rather borrows the 
address space description from the parent.

Linux implements vfork() like fork() but with shared memory. Linux fork() 
implementation is based on the concept from SunOS-4.0 from 1988, where the 
child uses a copy-on-write MMU setup. So fork() is faster than it has been on 
UNIX before SunOS-4.0, but Linux vfork() does not benefit from possible 
advantages.

I use a benchmark based on the configure script that I publish with the 
schilytools.

My tests are run from "bosh" with "set -o time" in effect and a shell specific 
test is run as:

shell=SUT-shell
CONFIG_SHELL=$shell $shell ./configure

ksh93 - compiled the way it is on OpenSolaris - is the fastest shell I know.
ksh93 1.5x faster than bash.

BTW: my memory was wrong, bosh is only 10% faster than it's predecessor the 
OpenSolaris Bourne Shell. This however varies between CPU types and with newer 
CPU types, the difference is bigger.

> I think this was deliberate so that side effects from ${foo?} and
> ${foo=bar} affect the outer shell environment instead of just the single
> command.
>
> > You would need:
> > 
> > expredir();
> > forkshell();
> > if (child) {
> > redirect();
> > exec();
> > } else {
> > repair_mods_from_the_child();
> > }
>
> > if you use vfork.
>
> This looks icky because opening a redirection may block (for example if
> the file is a fifo), which leads to the vforked child sticking around in
> its strange state for longer than expected.

> If this approach is used for background simple commands, it is
> definitely broken because the redirection opens should execute in
> parallel with the rest of the script. Things like
>   mkfifo f; something >f & exec 3 would deadlock.

bosh uses vfork() only for the simple cases and never for the cases where the 
shell syntax explicitely mentions a fork.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: Environment of expansions and visibility of side-effect assignments

[Robert Elz]

Date:Wed, 26 Jul 2017 18:27:43 +0200
From:Joerg Schilling 
Message-ID:  <5978c2ff.CLQ2yfNU3QmKah/w%joerg.schill...@fokus.fraunhofer.de>


  | You would need:

  | repair_mods_from_the_child();
  | }
  | 
  | if you use vfork.

When we have done a vfork, we do not generally make any mods that need
repair, and for what little is needed, the mechanism is already in place,
so nothing special is required.   That is, when the vfork code was added
(or perhaps sometime later, I did not investigate the ancient past that
closely) it was all done properly.What this does mean is that in a
vfork'd child, if we would normally free something, we don't (just lose
the memory) if some value would be saved, it isn't - after we have gotten
to the stage where a vfork has been done, we know that almost nothing matters
any more, and that child will not be around for more than a few milliseconds.

kre

Re: Environment of expansions and visibility of side-effect assignments

[Joerg Schilling]

Robert Elz  wrote:

> Date:Tue, 25 Jul 2017 19:06:13 +0200
> From:Joerg Schilling 
> Message-ID:  
> <59777a85.lYd+zInvutTipp/s%joerg.schill...@fokus.fraunhofer.de>
>
>   | ash does not use vfork() at all.
>
> And life is a continuous process of learning new things - I had no idea.
>
> But now I look back through time, I see that vfork() did not appear in
> the NetBSD shell until May 2000 (and not in FreeBSD until 2012).

Interesting to see that other also added vfork() support to their shells.

>From my metering, I know that on Solaris vfork() is 3x faster than fork()
even tough the fork() implementation in SunOS uses CopyOnWrite since 1988.

As a result, I typically see aprox. 30% better performance with a "configure" 
run when vfork() is used.

> Amazing.   In that case, my hypothesis as to why there is the split
> between redirect word expansion, and the following open/close is clearly
> nonsense - the code that does that (as far as NetBSD is concerned anyway,
> I have never seen an original ash shell) was in version 1.1 (March 1993),
> and is roughly
>
>   expredir();
>   forkshell();
>   if (child) {
>   redirect();
>   exec();
>   }
>
> (highly simplified and with lots removed, of course).

You would need:

expredir();
forkshell();
if (child) {
redirect();
exec();
} else {
repair_mods_from_the_child();
}

if you use vfork.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: Environment of expansions and visibility of side-effect assignments

[Robert Elz]

Date:Tue, 25 Jul 2017 19:06:13 +0200
From:Joerg Schilling 
Message-ID:  <59777a85.lYd+zInvutTipp/s%joerg.schill...@fokus.fraunhofer.de>

  | ash does not use vfork() at all.

And life is a continuous process of learning new things - I had no idea.

But now I look back through time, I see that vfork() did not appear in
the NetBSD shell until May 2000 (and not in FreeBSD until 2012).

Amazing.   In that case, my hypothesis as to why there is the split
between redirect word expansion, and the following open/close is clearly
nonsense - the code that does that (as far as NetBSD is concerned anyway,
I have never seen an original ash shell) was in version 1.1 (March 1993),
and is roughly

expredir();
forkshell();
if (child) {
redirect();
exec();
}

(highly simplified and with lots removed, of course).

where "expredir" expands the redirect words (if any), forkshell() is
what does the fork (and all associated bookkeeping) and redirect()
does the actual file descriptor manipulations of the redirects (including
what's needed to be able to restore things in cases where there was no
fork, and the redirects are only to be temporary.)

That means that the decision to expand the redirects in the parent shell
context was done deliberately, the expredir() could easily have been left
until after the forkshell() (certainly with that only using fork()).

kre

Re: Environment of expansions and visibility of side-effect assignments

[Joerg Schilling]

Robert Elz  wrote:

>   if (vfork() == 0) {
>   for each redirect operator & word
>   expand the word
>   open/close/dup as required
>   execve(...)
>   }
>
> leaves the side effects of the expansions (as they affect memory in the
> shell) visible in the parent.   That is, just a bug in a slightly sloppy
> conversion to vfork().

Yes, if you like to use vfork(), you need to know what modifications you did in 
the child and then undo them in the parent once the child returned the borrowed
image to the parent.

> The ash based shalls do not have that problem, that's BSD originated code,
> and (most, at least old timer) BSD people understand how to use vfork(),
> so the vfork() is not an issue at all.

ash does not use vfork() at all.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: Environment of expansions and visibility of side-effect assignments

[Robert Elz]

Date:Tue, 25 Jul 2017 16:53:54 +0100
From:Geoff Clare 
Message-ID:  <20170725155354.GB2553@lt2.masqnet>

  | The context here is a list of items that comprise a shell or utility
  | environment.  This item in the list is concerned with what files are
  | open.

Agreed, but...

  | Clearly there is no intention that the modifications and
  | additions it refers to means anything other than modifications and
  | additions to the set of open files.

I don't think I agree with that, what is at issue here is when the
redirections are "performed" - in the spec there's no concept of
splitting the expansion, and the subsequent open/dup operation - I
suspect because in the early implementations there is (was) no
separation, each redirection was expanded, and then the open done
immediately after.

That the spec describes things this way is then not at all surprising
given its history ... and that ksh88 (apparently) does it this way.

That is, as written the expansion and open happen in the same context,
if we agree (and I think we do) that the actual fd manipulation happens
in the sub-shell context, then the expansion of the redirection word
happens there as well.

kre

Re: Environment of expansions and visibility of side-effect assignments

[SHwareSyst]

By P2366, L75534, redirections are performed in a subshell when no command  
name results, such as one affecting an entire compound command... It goes  
on that redirections otherwise may fail as part of the current execution  
environment, not the pending command environment. Maybe it needs to be more  
explicit, but from that I construe it as when there is a command name 
redirects  are expected to modify the current execution environment and the 
utility 
 environment inherits them.
 
 
In a message dated 7/25/2017 11:38:56 A.M. Eastern Daylight Time,  
k...@munnari.oz.au writes:

Date:Tue, 25 Jul 2017 17:12:28  +0200
From:Joerg Schilling  
Message-ID:   
<59775fdc.mbda4ty+ekbd7zlj%joerg.schill...@fokus.fraunhofer.de>


| Given that Robert came up with this kind of tests:
| 
|  unset X 
| cat <<-EOF 
| ${X=2} 
| EOF  
| echo "${X-1}" 

That was because I rewrote the way the  NetBSD sh does here doc expansions
a year or so ago, and I made those  happen in the sub-shell environ, which
I believed then (and still believe  is correct now) is where all 
redirections
should be performed - which  currently includes the expansion part 
(according
to the spec, as it is  currently - I am not sure I see any particularly
good reason why it must be  that way however.)

So we have tests for that, in our test suite ... I  ran that against bosh,
and sent the (relevant) results to Joerg (many of  the tests check NetBSD
specified, POSIX unspecified, behaviour, like  ${} and more, that those
behaved differently than we expect, I just  ignored, similarly for tests of
NetBSD extensions.)

And of course,  when any application behaves differently when it uses 
vfork()
than it does  using fork, perhaps just compiled with -Dvfork=fork then I
think that's  always a  bug.

kre

Re: Environment of expansions and visibility of side-effect assignments

[Geoff Clare]

Robert Elz  wrote, on 25 Jul 2017:
>
> Then 2.12 .. (after the first group of bullet points)
> 
> Utilities other than the special built-ins (see Section 2.14) shall be
> invoked in a separate environment that consists of the following.
> 
> Then the first following bullet point
> 
> Open files inherited on invocation of the shell, open files controlled by
> the exec special built-in plus any modifications, and additions specified
> by any redirections to the utility
> 
> where the final "and" clause is what matters, 

The context here is a list of items that comprise a shell or utility
environment.  This item in the list is concerned with what files are
open.  Clearly there is no intention that the modifications and
additions it refers to means anything other than modifications and
additions to the set of open files.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: Environment of expansions and visibility of side-effect assignments

[Geoff Clare]

shwares...@aol.com  wrote, on 25 Jul 2017:
>
> I disagree about $M

The current wording is the result of Mantis bug 255 and was written that
way specifically to allow all of the observed behaviours.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: Environment of expansions and visibility of side-effect assignments

[Geoff Clare]

Joerg Schilling  wrote, on 25 Jul 2017:
>
> unset X 
> cat <<-EOF 
> ${X=2} 
> EOF 
> echo "${X-1}" 
> 
> it turned out that ksh88 and a classical Bourne Shell both print 
> 
> 2
> 1
> 
> but ksh93 and my "bosh" print 
> 
> 2
> 2
> 
> but when I recompile bosh with -DNO_VFORK, it prints
> 
> 2
> 1
> 
> Could you comment whether this is undefined as well?

Yes it is, as far as I can see.  The relevant text in 2.7.4 is:

If no part of word is quoted, all lines of the here-document shall
be expanded for parameter expansion, command substitution, and
arithmetic expansion.

It says nothing about which environment the expansion must be done in.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: Environment of expansions and visibility of side-effect assignments

[Robert Elz]

Date:Tue, 25 Jul 2017 15:11:58 +0100
From:Geoff Clare 
Message-ID:  <20170725141158.GA6514@lt2.masqnet>

  | The $M value is explicitly unspecified (2.9.1, second sub-bullet of the
  | second bullet item).

Ah, yes, I see that now, thanks - the var-assign part was an afterthought
in the test, and I had not really researched the spec for that one, this
all started with a question related to redirects.

That also justifies the "bla:bla::" result, and makes that shell (along
with zsh's ":bla::" clearly conformant.)   (That was bash incidentally.)

  | For $Q, I can't find any clear requirement about which environment the
  | expansions within the "word" of a redirection associated with a
  | non-built-in utility have to be done in.

I'd agree with that, with the emphasis on "clear", but there is a derivation
from reading several sections together, which I think specifies that they
should be done in a subshell

  | The file descriptor operations
  | must not affect the current shell environment,

Yes, I don't think that is an issue.

  | but I don't think anything forbids the expansions affecting it.

If Joerg still has the e-mail I sent him (I don't keep copies of mail
I send, unless it gets sent back to me...) then my full reasoning might
still be available, but it is (roughly):

2.9.1.1  1. e. i. b.  (that should be banned - 8 level deep numbering!)
 
Otherwise, the shell executes the utility in a separate utility
environment (see Section 2.12)

Then 2.12 .. (after the first group of bullet points)

Utilities other than the special built-ins (see Section 2.14) shall be
invoked in a separate environment that consists of the following.

Then the first following bullet point

Open files inherited on invocation of the shell, open files controlled by
the exec special built-in plus any modifications, and additions specified
by any redirections to the utility

where the final "and" clause is what matters, combining the relevant parts
of the above we get

"a separate environment that consists of [...] and additions specified
 by any redirections"

And then back to 2.9.1

the following expansions, assignments, and redirections shall all be
performed from the beginning of the command text to the end:

3. Redirections shall be performed as described in Section 2.7.

And then to 2.7

For the other redirection operators,

(that is, not << and <<- which have just been considered, and which
are not relevant right now)

   the word that follows the redirection operator shall be subjected
   to tilde expansion, parameter expansion, command substitution, arithmetic
   expansion, and quote removal.

(and then goes on about pathname expansion, also not relevant here).

That tells us that the expansions are part of "performing the redirection"
(since that is what 2.7 is used for, as explained in 2.9.1 step 3), and
then back to 2.12, that should happen in the command's new execution environ.

So, definitely not "clear", but it is there I think.

And then, while the following alone would not be enough:

  | However, in the no-command-name
  | case (and without using "exec") 2.9.1 clearly says redirections "shall be
  | performed in a subshell environment".

I think we have an additional clue that (except for the "exec" special
case) all redirections are "performed" in a subshell environ.   This case
needed to be explicitly stated, otherwise there would not be a subshell,
without any command name to create it, but if you accept the convoluted
reasoning above, what this is doing is just reinforcing the general
method for this special case.

And if you combine that with what the ancient shells did (as Joerg's
message showed - none of those shells are available to me, and so were
not ones I tested, except bosh, and that I did not compile in the special
way needed to make it behave like he did) it is perhaps not surprising
that this is what was always intended.

I guess the question now, given that so many shells do not do it that
way, is whether this should be maintained (perhaps made a little more
clear, if so), or whether we should just give up, as was done with the
var assignments, and make it explicitly unspecified when the expansion
part of the redirection operators is done (which would logically extend
to the no-command-name case as well - there is no reason for it to be
different.)


  | If any of the shells which set Q
  | in the above command also set it when just executing:
  | 
  |  > /tmp/JUNK-${Q=bla}
  | 
  | then that's a bug (or at least a non-conformance) in those shells.

sh -c ' > /tmp/JUNK-${Q=bla}; echo $Q'
bla

That's the NetBSD shell.   I suspect all ash derived shells are the
same (but as I said earlier, I have long believed it to be incorrect.)
The separation of expansion, and fd manipulation (actually doing the
open() or dup() calls) is (was) explicit in ash, and has not been
changed by 

Re: Environment of expansions and visibility of side-effect assignments

[SHwareSyst]

I disagree about $M, as I see those sub-bullets applying during  evaluation 
of that command or assignments only, in that order of evaluation  isn't 
guaranteed except as a consequence of expansion nesting, and so a  reference 
may get either a pre- or post-assignment value; but by the definition  of the 
${=} and ${:=} expansions the assignments should all be visible  when 
evaluation of the next command begins, as the specified behavior, since no  
nesting of command substitutions is done in the example.
 
 
In a message dated 7/25/2017 10:12:37 A.M. Eastern Daylight Time,  
g...@opengroup.org writes:

Robert  Elz  wrote, on 25 Jul 2017:
>
> Given the  following command sequence ...
> 
> VAR=${M=bla}  /bin/echo ${N=bla} > /tmp/JUNK-${Q=bla}
> echo  $M:$N:$VAR:$Q
> 
> what is the "echo" on the 2nd line supposed to  print?
> 
> It is clear (I think) that expanding VAR='' and N=bla  is correct there,
> and all shells I tested did that,
>  
[...]
> 
> The most common result is
> 
>bla:bla::bla
> 
> which is at least consistent, but to me  incorrect, I believe from reading
> the spec, that the var-assigns and  redirects are supposed to be evaluated
> in the context of the command  about to be executed, which would mean that
> the side-effects from  evaluating them would not be visible in the parent
> shell (which later  runs the 2nd command - at least not when, as in this
> case, the command  is clearly not a builtin utility or function.)

The $M value is  explicitly unspecified (2.9.1, second sub-bullet of the
second bullet  item).

For $Q, I can't find any clear requirement about which  environment the
expansions within the "word" of a redirection associated  with a
non-built-in utility have to be done in.  The file descriptor  operations
must not affect the current shell environment, but I don't think  anything
forbids the expansions affecting it.  However, in the  no-command-name
case (and without using "exec") 2.9.1 clearly says  redirections "shall be
performed in a subshell environment".  If any  of the shells which set Q
in the above command also set it when just  executing:

> /tmp/JUNK-${Q=bla}

then that's a bug (or at  least a non-conformance) in those shells.

-- 
Geoff Clare  
The Open Group, Apex Plaza, Forbury Road,  Reading, RG1 1AX, England

Re: Environment of expansions and visibility of side-effect assignments

[Joerg Schilling]

Geoff Clare  wrote:

> Robert Elz  wrote, on 25 Jul 2017:
> >
> > Given the following command sequence ...
> > 
> > VAR=${M=bla} /bin/echo ${N=bla} > /tmp/JUNK-${Q=bla}
> > echo $M:$N:$VAR:$Q
> > 
> > what is the "echo" on the 2nd line supposed to print?
> > 
> > It is clear (I think) that expanding VAR='' and N=bla is correct there,
> > and all shells I tested did that,
> > 
> [...]
> > 
> > The most common result is
> > 
> > bla:bla::bla
> > 
> > which is at least consistent, but to me incorrect, I believe from reading
> > the spec, that the var-assigns and redirects are supposed to be evaluated
> > in the context of the command about to be executed, which would mean that
> > the side-effects from evaluating them would not be visible in the parent
> > shell (which later runs the 2nd command - at least not when, as in this
> > case, the command is clearly not a builtin utility or function.)
>
> The $M value is explicitly unspecified (2.9.1, second sub-bullet of the
> second bullet item).

Given that Robert came up with this kind of tests:

unset X 
cat <<-EOF 
${X=2} 
EOF 
echo "${X-1}" 

it turned out that ksh88 and a classical Bourne Shell both print 

2
1

but ksh93 and my "bosh" print 

2
2

but when I recompile bosh with -DNO_VFORK, it prints

2
1

Could you comment whether this is undefined as well?
What I definitely see is that ksh93 and bosh in standard vfork() mode both fail 
to repeat the classical Bourne Shell behavior because they do not restore 
modifications done by the vfork() child.

> For $Q, I can't find any clear requirement about which environment the
> expansions within the "word" of a redirection associated with a
> non-built-in utility have to be done in.  The file descriptor operations
> must not affect the current shell environment, but I don't think anything
> forbids the expansions affecting it.  However, in the no-command-name
> case (and without using "exec") 2.9.1 clearly says redirections "shall be
> performed in a subshell environment".  If any of the shells which set Q
> in the above command also set it when just executing:
>
>  > /tmp/JUNK-${Q=bla}
>
> then that's a bug (or at least a non-conformance) in those shells.

So you verify a ksh93 bug that is most likely caused by the same vfork() 
problem 
as with bosh.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: Environment of expansions and visibility of side-effect assignments

[Geoff Clare]

Robert Elz  wrote, on 25 Jul 2017:
>
> Given the following command sequence ...
> 
>   VAR=${M=bla} /bin/echo ${N=bla} > /tmp/JUNK-${Q=bla}
>   echo $M:$N:$VAR:$Q
> 
> what is the "echo" on the 2nd line supposed to print?
> 
> It is clear (I think) that expanding VAR='' and N=bla is correct there,
> and all shells I tested did that,
> 
[...]
> 
> The most common result is
> 
>   bla:bla::bla
> 
> which is at least consistent, but to me incorrect, I believe from reading
> the spec, that the var-assigns and redirects are supposed to be evaluated
> in the context of the command about to be executed, which would mean that
> the side-effects from evaluating them would not be visible in the parent
> shell (which later runs the 2nd command - at least not when, as in this
> case, the command is clearly not a builtin utility or function.)

The $M value is explicitly unspecified (2.9.1, second sub-bullet of the
second bullet item).

For $Q, I can't find any clear requirement about which environment the
expansions within the "word" of a redirection associated with a
non-built-in utility have to be done in.  The file descriptor operations
must not affect the current shell environment, but I don't think anything
forbids the expansions affecting it.  However, in the no-command-name
case (and without using "exec") 2.9.1 clearly says redirections "shall be
performed in a subshell environment".  If any of the shells which set Q
in the above command also set it when just executing:

 > /tmp/JUNK-${Q=bla}

then that's a bug (or at least a non-conformance) in those shells.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: Environment of expansions and visibility of side-effect assignments

[SHwareSyst]

Only the explicit assignment preceding the command_word is evaluated in the 
 new context, and uses the value from the caller's context temporarily. The 
 expansions all occur in, and assignments affect, the caller's context  
first. So I agree VAR should stay an empty string or unset, but M, N,  and Q 
getting set to bla is the conforming behavior, and this is what the most  
common result reflects. This is one of the reasons, afaik, that redirect  
evaluation can be swapped with assignment evaluation in XCU 2.9.1,  because 
actual 
value assignment may need to be deferred to the new context after  
expansions related to the redirects are evaluated in the caller's context. 
 
If you add $VAR before the '>', it should put bla bla into JUNK-bla, not  a 
single bla; if you want to check which value the first echo sees.
 
 
In a message dated 7/24/2017 1:51:37 P.M. Eastern Daylight Time,  
k...@munnari.oz.au writes:

Given  the following command sequence ...

VAR=${M=bla} /bin/echo  ${N=bla} > /tmp/JUNK-${Q=bla}
echo  $M:$N:$VAR:$Q

what is the "echo" on the 2nd line supposed to  print?

It is clear (I think) that expanding VAR='' and N=bla is correct  there,
and all shells I tested did that,

Beyond that, little  consistency - there was a clear winning result,, but
not one that is in  accordance with the way I read the spec.

The NetBSD sh (the one I work  on) is in that group - I have long had an
item on my todo list to fix this,  as I consider us to be broken there.

>From my reading of the spec, what  I expect the output is intended to be
is
:bla::

Of  the shells I tested, only zsh managed that (zsh with no options, other
than  zsh -c '...').   That would make zsh (based just on this one  test)
the only possible candidate (among the shells I tested) as being  posix 
conformant!

The most common result is

bla:bla::bla

which is at least consistent, but to me incorrect, I  believe from reading
the spec, that the var-assigns and redirects are  supposed to be evaluated
in the context of the command about to be  executed, which would mean that
the side-effects from evaluating them would  not be visible in the parent
shell (which later runs the 2nd command - at  least not when, as in this
case, the command is clearly not a builtin  utility or function.)

The result
bla:bla::
was also  observed, which looks closer to correct, but inconsistent, I 
cannot
find  any way to read the spec in which that can be the result.

I appreciate  that this kind of thing is not common (which is why it is
just an entry on  my todo list, not even elevated to the status of being
a current project to  consider - just something for the future when I run
out of more useful  things to do) but we should really know what should
happen in cases like  this.

kre

Re: Environment of expansions and visibility of side-effect assignments

[Joerg Schilling]

Robert Elz  wrote:

> Given the following command sequence ...
>
>   VAR=${M=bla} /bin/echo ${N=bla} > /tmp/JUNK-${Q=bla}
>   echo $M:$N:$VAR:$Q
>
> what is the "echo" on the 2nd line supposed to print?
>
> It is clear (I think) that expanding VAR='' and N=bla is correct there,
> and all shells I tested did that,
>
> Beyond that, little consistency - there was a clear winning result,, but
> not one that is in accordance with the way I read the spec.
>
> The NetBSD sh (the one I work on) is in that group - I have long had an
> item on my todo list to fix this, as I consider us to be broken there.
>
> From my reading of the spec, what I expect the output is intended to be
> is
>   :bla::

This is what I get with:

ksh88
Svr4 Bourne Shell
bosh compiled with:
cd sh
smake COPTX=-DNO_VFORK clean all

So I confirm your interpretation.

The behavior from ksh93 looks like a result of a missing vfork() workaround.


Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'