Re: Environment and pipes
Jean Delvare dixit: >I will check with the customer. The example they provided was clearly >made up to illustrate the problem (and I do appreciate when customers >make the problem easy to understand) so I don't know exactly what their >actually script is doing. Okay. It’s probably better for all involved if actual script code is worked on, but in this case, for a general inquiry, this was good. >Thanks for the hint. Customers are just starting to migrate, on our >request (as it turns out that ksh-93 has a lot of bugs and upstream is >hardly active so we did not feel like supporting it any more) and this >is the first incompatibility I hear about. I see. >> Scripts can use this, e.g: >> >> # here, set +o foo is active >> cat "$@" | while …; do set -o foo; …; done >> # here, set +o foo is active again > >Sorry but I do not understand how this relate to the problem at hand? >Just an illustration of another change in subshells that do not >propagate to the main shell? Yes, exactly. Since a few years, scripts may rely on this. (Although the example is dated: functions defined with “function” have function-local scope for shell options since mksh R51, like recent ksh93 versions have.) bye, //mirabilos -- „Cool, /usr/share/doc/mksh/examples/uhr.gz ist ja ein Grund, mksh auf jedem System zu installieren.“ -- XTaran auf der OpenRheinRuhr, ganz begeistert (EN: “[…]uhr.gz is a reason to install mksh on every system.”)
Re: Environment and pipes
Le Friday 11 September 2015 à 20:23 +, Thorsten Glaser a écrit :
> Dixi quod…
>
> >I think coprocesses are pretty usable for this in almost all cases
> >(I did have one where they weren’t, but you can still often background
> >a part, then play with fd redirection).
>
> Another thing that may help you, if it’s absolutely needed:
>
> Before we had ${PIPESTATUS[*]} I wrote things like this:
>
> foo | (bar; echo $? >bar.rv) | baz
>
> Similarily, you can do tricks with high file descriptors.
>
> x=$( (echo foo | (tr a-z A-Z >&4) | (echo bla >&5)) 4>&1) 5>&1
> echo x=$x
>
> This gives:
>
> bla
> x=FOO
"Nice" :]
> Both not as efficient as direct variable assignment, but…
>
> And let's not forget using…
> while …; do …; done … instead of…
> cat foo | while …; do …; done
> … (which is cat abuse anyway).
Amen. I've been working with a lot of bash scripts and I yell each time
I see the "cat foo |" construct.
> Often, some not-so-basic or more modern scripting approaches
> can eliminate the need for a construct like you were asking for.
I will check with the customer. The example they provided was clearly
made up to illustrate the problem (and I do appreciate when customers
make the problem easy to understand) so I don't know exactly what their
actually script is doing.
--
Jean Delvare
SUSE L3 Support
Re: Environment and pipes
Hi Thorsten, Thanks for your fast and complete answers. Le Friday 11 September 2015 à 20:07 +, Thorsten Glaser a écrit : > Jean Delvare dixit: > > >Dear mksh developers, > > Feel free to use the singular ☺ we’re all just one person, be > it mksh, bash, ksh93, lynx, ncurses, xterm… all those projects > are done by one person (sometimes even the same). I know the feeling ;-) > >A customer of ours is migrating from ksh-93 to mksh. They reported the > > Interesting, but, depending on the scripts, probably hard (lots > of other ksh93 extensions, such as float, will not be there either). Thanks for the hint. Customers are just starting to migrate, on our request (as it turns out that ksh-93 has a lot of bugs and upstream is hardly active so we did not feel like supporting it any more) and this is the first incompatibility I hear about. > [ all parts of a pipeline are run in subshells ] > >I would like to ask if this is: > > > >1* A deliberate design decision, and this will never change. > > An implementation choice, and while the status of this was > not set in stone for several years, some time ago (I’d have > to look when) I decided to make it into a formal language > feature, i.e. it will never change. OK, noted. I'll let the customer know. > Scripts can use this, e.g: > > # here, set +o foo is active > cat "$@" | while …; do set -o foo; …; done > # here, set +o foo is active again Sorry but I do not understand how this relate to the problem at hand? Just an illustration of another change in subshells that do not propagate to the main shell? -- Jean Delvare SUSE L3 Support
Re: Environment and pipes
Dixi quod…
>I think coprocesses are pretty usable for this in almost all cases
>(I did have one where they weren’t, but you can still often background
>a part, then play with fd redirection).
Another thing that may help you, if it’s absolutely needed:
Before we had ${PIPESTATUS[*]} I wrote things like this:
foo | (bar; echo $? >bar.rv) | baz
Similarily, you can do tricks with high file descriptors.
x=$( (echo foo | (tr a-z A-Z >&4) | (echo bla >&5)) 4>&1) 5>&1
echo x=$x
This gives:
bla
x=FOO
Both not as efficient as direct variable assignment, but…
And let's not forget using…
while …; do …; done emacs als auch vi zum Kotzen finde (joe rules) und pine für den einzig
> bedienbaren textmode-mailclient halte (und ich hab sie alle ausprobiert). ;)
Hallo, ich bin der Holger ("Hallo Holger!"), und ich bin ebenfalls
... pine-User, und das auch noch gewohnheitsmäßig ("Oooohhh"). [aus dasr]
Re: Environment and pipes
Jean Delvare dixit: >Dear mksh developers, Feel free to use the singular ☺ we’re all just one person, be it mksh, bash, ksh93, lynx, ncurses, xterm… all those projects are done by one person (sometimes even the same). >A customer of ours is migrating from ksh-93 to mksh. They reported the Interesting, but, depending on the scripts, probably hard (lots of other ksh93 extensions, such as float, will not be there either). [ all parts of a pipeline are run in subshells ] >I would like to ask if this is: > >1* A deliberate design decision, and this will never change. An implementation choice, and while the status of this was not set in stone for several years, some time ago (I’d have to look when) I decided to make it into a formal language feature, i.e. it will never change. Scripts can use this, e.g: # here, set +o foo is active cat "$@" | while …; do set -o foo; …; done # here, set +o foo is active again Dr. Werner Fink dixit: >the mksh can use co-processes as ksh can do Correct. >Btw: this behaviour of the mksh is docmented in the manual page of Correct. >The question rises, if it would be possible to a) let the last pipe >of the mksh not be a subshell and/or use shared memory area to make No. There is no “shared memory” on many mksh targets, and one of the biggest strength of mksh is that it behaves exactly the same across all supported targets. I think coprocesses are pretty usable for this in almost all cases (I did have one where they weren’t, but you can still often background a part, then play with fd redirection). >Werner > >-- > "Having a smoking section in a restaurant is like having > a peeing section in a swimming pool." -- Edward Burr Ah, yours too ☺ so let me pick a different sig… //mirabilos -- Solange man keine schmutzigen Tricks macht, und ich meine *wirklich* schmutzige Tricks, wie bei einer doppelt verketteten Liste beide Pointer XORen und in nur einem Word speichern, funktioniert Boehm ganz hervorragend. -- Andreas Bogk über boehm-gc in d.a.s.r
Environment and pipes
Dear mksh developers, A customer of ours is migrating from ksh-93 to mksh. They reported the following difference in behavior: ksh-93: $ echo X | read A $ echo $A X mksh: $ echo X | read A $ echo $A So mksh behaves the same as for example bash, in that changes to the environment of the right side of a pipe aren't reflected into the main process. That difference breaks the customer's scripts. I would like to ask if this is: 1* A deliberate design decision, and this will never change. 2* A bug, mksh should behave the same as ksh-93 in this respect. 3* Something that was not discussed so far, so status is unknown. Additionally, unless answer is 2 above, I would be grateful if you could suggest an alternative way to achieve the same with mksh. I know how I would do it with bash, but I don't know what mksh is capable of. Thanks, -- Jean Delvare SUSE L3 Support
Re: Environment and pipes
On Fri, Sep 11, 2015 at 03:10:13PM +0200, Jean Delvare wrote: > Dear mksh developers, > > A customer of ours is migrating from ksh-93 to mksh. They reported the > following difference in behavior: > > ksh-93: > $ echo X | read A > $ echo $A > X > > mksh: > $ echo X | read A > $ echo $A > > So mksh behaves the same as for example bash, in that changes to the > environment of the right side of a pipe aren't reflected into the main > process. That difference breaks the customer's scripts. > > I would like to ask if this is: > > 1* A deliberate design decision, and this will never change. > > 2* A bug, mksh should behave the same as ksh-93 in this respect. > > 3* Something that was not discussed so far, so status is unknown. > > Additionally, unless answer is 2 above, I would be grateful if you could > suggest an alternative way to achieve the same with mksh. I know how I > would do it with bash, but I don't know what mksh is capable of. Yes, indeed only orignal ksh and zsh can do ksh93 -c 'echo xxx | read A; echo $A' xxx zsh -c 'echo xxx | read A; echo $A' xxx the mksh can use co-processes as ksh can do ksh93 -c 'echo xxx |& read -p A; echo $A' xxx mksh -c 'echo xxx |& read -p A; echo $A' xxx but not bash nor zsh. A long time ago I had written a small patch fo bash which now seems to be upstream but for bash this is disabled by default. To use it the job control has to be disabled as well as the lastpipee shell option has to be enabled: bash -c 'shopt -s lastpipe; echo xxx | read A ; echo $A' xxx bash -c 'echo $BASH_VERSION' 4.2.46(1)-release Btw: this behaviour of the mksh is docmented in the manual page of mksh: CAVEATS mksh only supports the Unicode BMP (Basic Multilingual Plane). mksh has a different scope model from AT&T UNIX ksh, which leads to subtile differences in semantics for identical builtins. This can cause issues with a nameref to suddenly point to a local variable by accident; fixing this is hard. The parts of a pipeline, like below, are executed in subshells. Thus, variable assignments inside them are not visible in the surrounding execution environment. Use co-processes instead. foo | bar | read baz# will not change $baz foo | bar |& read -p baz# will, however, do so mksh provides a consistent set of 32-bit integer arithmetics, both signed and unsigned, with defined wraparound and sign of the result of a modulo operation, even (defying POSIX) on 64-bit systems. If you require 64-bit integer arithmetics, use lksh (legacy mksh) instead, but be aware that, in POSIX, it's legal for the OS to make print $((2147483647 + 1)) delete all files on your system, as it's Undefined Behaviour. The question rises, if it would be possible to a) let the last pipe of the mksh not be a subshell and/or use shared memory area to make new or changed shell variables and of the sub shell visible to the main shell. Werner -- "Having a smoking section in a restaurant is like having a peeing section in a swimming pool." -- Edward Burr signature.asc Description: Digital signature
