Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On 2021/10/30 09:07, Robert Elz wrote:
oguzismailuy...@gmail.com said:
| `break' is not a keyword in the shell, but a special command.
That's true. However, 99%**1 of script writers don't see
it that way,g they believe it is just like "if" or
"while" or "done" or "return".
That's why the "This is counterintuitive." I would guess - in most
cases, the issue isn't quite like yours, but more like
---
Something that supports break/continue being dynamically
scoped: the variables used in the loop are dynamically
scoped. As long as the function can access the local variables,
it is "part" of the loop.
Bash/shell provides dynamic scoping, by default, for
referenced (but undeclared) variables. So why wouldn't
someone expect the functions that can change all the variables
in the loop to also be able to modify loop params?
ps: the NetBSD shell continues to work the way that
you want, and does so by deliberate choice**2 - our test
suite has a whole stack of tests to make sure this
continues to all work "correctly" (doesn't
accidentally get changed).
**1+**2:
So you are saying that NetBSD shell script writers are less
than 1% of script writers? Is NetBSD's market share that
low? Just curious.
Regardless -- it points, at least, to it being something
that I would think should be shopt'd if nothing else.
I find the inconsistent application of loop parameters
to be, at least, a wart -- i.e. loop variables are dynamically
propagated to called functions, but loop control "verbs" aren't.
Perl is a bit schizoid in this area:
#!/usr/bin/perl
use warnings; use strict; use P;
my $x;
sub foo() {
if ($x>=2 && $x<4) { next; }
if ($x==5) { $x=9; }
if ($x>=11) { last;}
}
for ($x=0;$x<20;++$x){
P "b4 foo x=%s", $x;
foo;
P "after foo x=%s", $x;
}
---
results in dynamically scoped execution with commentary:
b4 foo x=0
after foo x=0
b4 foo x=1
after foo x=1
b4 foo x=2
Exiting subroutine via next at /tmp/lex.pl line 5.
b4 foo x=3
Exiting subroutine via next at /tmp/lex.pl line 5.
b4 foo x=4
after foo x=4
b4 foo x=5
after foo x=9
b4 foo x=10
after foo x=10
b4 foo x=11
Exiting subroutine via last at /tmp/lex.pl line 7.
-
I think a shopt would be more flexible. Having
loop vars be dynamic, but verbs not, seem inconsistent.
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On 2021-10-30 at 23:07 +0700, Robert Elz wrote: > For people who don't believe that all programming languages should work > the same way (usually the same way as the one they learned first) this > isn't necessarily as important - but that's a tiny majority of people. Uh? That's a quite different statement. I do think it's good that languages are somewhat consistent. And if on language A + is the addition operand, it doesn't become the division one on a different one, or that you don't write 'if' to produce a loop. I think that's what is being termed as counterintuitive here. If every languages would have to be the same as every other, then they wouldn't be different ones.
Re: Arbitrary command execution in shell - by design!
On 2021-10-29 at 22:41 -0700, L A Walsh wrote: > > This is quite unfair. > Huh? It's true--look at how functions have to be stored in > the environment because someone was able to hack "their > own system" where they already have unrestricted shell access. > > If that isn't ugly or lame, what is? But it was the fault > of taking actions that a hacker could do on a system where > they already had shell access to corrupt their own environment. No. The big issue was that web servers convert HTTP headers into environment variables when calling a CGI script (adding a HTTP_ prefix which didn't help here), but that allowing a *remote* user to set certain environment variables that lead to execution if the CGI itself was a shell script run under bash or bash got called from it. Two features were safe when taken separately, but turned out to be dangerous when combined. A bug that is only self-exploitable isn't really a security issue. > If permissions and paths were correctly set to never execute > files owned by that user, I don't see how that exploit > would gain root access and affect anyone other the user > who was injecting the code into their own function. That exploit didn't allow escalating to root (by itself). It would typically allow execution as a web user (www-data, apache…) or the owner of the CGI file. Escalation to root would need to be a separate step. Regards
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On Sat, Oct 30, 2021 at 08:45:05PM +0300, Oğuz wrote: > I know, it's great. Though I still couldn't figure out how to have a > command run on login and it drives me mad. Depends on *how* you log in.
Re: hash not restored after running command -p
On 10/29/21 6:06 PM, Roger Morris wrote: Bash Version: 5.0 Patch Level: 17 Release Status: release Description: I believe there's a bug in bash 4.4 (and still in 5.1) that wasn't there in 4.3.30 When 'command -p' runs, it no longer seems to restore the hash to its previous value. This came in in mid-2015 as part of a set of fixes to `command -p'. I think the bash-4.4 and later behavior is correct: just because you're telling command to use a standard path it doesn't mean it should not act like it would in any other circumstance. There's nothing in POSIX that contradicts that. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On Sat, Oct 30, 2021 at 7:07 PM Robert Elz wrote:
>
> What POSIX says about this (which you probably know already) is:
>
> A loop shall enclose a break or continue command if the loop
> lexically encloses the command. A loop lexically encloses a
> break or continue command if the command is:
>
> [and just paraphrasing the conditions]
>
> - same exxecution environment (ie: not in a subshell)
> - inside a compound list associated with a loop (including
> the test list for while/until)
> - not in the body of a function ...
>
> (more or less the obvious useful definition of lexically enclosing).
>
> That is the case in which break & continue are required to work.
>
> It goes on to say:
>
> If n is greater than the number of lexically enclosing loops
> and there is a non-lexically enclosing loop in progress in the
> same execution environment as the break or continue command, it is
> unspecified whether that loop encloses the command.
>
> In your example, the number of lexically enclosing loops is 0, 'n' is 1,
> so the shell is allowed to break from the loop, but not required to.
>
Yes, I read all about this, thanks for the summary.
> Or in other words, you should not write code like that, because it
> is not guaranteed to work.
Pity. But at least for now, (in bash, not other shells) I can place
the function call between `shopt -s compat43' and `shopt -u compat43'
and have the behavior I want without side effects (I believe).
>
>
> oguzismailuy...@gmail.com said:
>| `break' is not a keyword in the shell, but a special command.
>
> That's true. However, 99% of script writers don't see it that way,
> they believe it is just like "if" or "while" or "done" or "return".
>
> That's why the "This is counterintuitive." I would guess - in most
> cases, the issue isn't quite like yours, but more like
>
> f() {
> for x in a b c; do
> for y in 1 2 3; do
> case "$x$y" in
> (b3)break;;
> (a2)continue 2;;
> (c1)break 3;;
> esac
> done
> done
> }
>
> and then called something like
>
> for i in 1 2 3 4 5 6
> do
> echo $i; f
> done
>
> and people wonder why that loop never iterates the 2nd time (prints just 1).
>
> All this because the script author probably thought that they were
> writing C, and the b3 break is for the case statement, a2 continue
> is the y loop, and c1 break the x loop...
>
> They can be taught that that's wrong, break/continue do nothing for
> case statements, and don't count them - but they simply refuse to believe
> that whatever happens inside the function should affect the caller's
> flow - an error from the "break 3" is something they can understand,
> as is just ignoring the "overflow", but not more than that.
Yeah. Not about break/continue, but I met many complaining about how
there should be spaces around `[' and `]', how `if [ grep foo file ]
...' never reaches the else part, etc. and it's all tiresome.
Thankfully I'm not writing a shell.
>
> For people who don't believe that all programming languages should work
> the same way (usually the same way as the one they learned first) this
> isn't necessarily as important - but that's a tiny majority of people.
>
> kre
>
> ps: the NetBSD shell continues to work the way that you want, and does
> so by deliberate choice - our test suite has a whole stack of tests to
> make sure this continues to all work "correctly" (doesn't accidentally
> get changed).
>
I know, it's great. Though I still couldn't figure out how to have a
command run on login and it drives me mad.
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On Sat, Oct 30, 2021 at 6:55 PM Chet Ramey wrote: > You might be interested in > > https://www.austingroupbugs.net/view.php?id=842 The only argument against dynamic scoping there is that one might use `break 1000' for exiting the outermost lexically enclosing loop and I don't think anyone does that, you'd just count loops and specify the exact number. Had it not been too late, I'd suggest `break 0' have that effect (and `break -1' continue the outermost loop in progress, and so on). > > There was a discussion on the austin-group mailing list back in 2016 > accompanying this defect report. This I was interested in, but mail-archive doesn't have a copy, and the austin group collaboration site doesn't show many of the replies to that thread, including yours. > > Chet > > -- > ``The lyf so short, the craft so long to lerne.'' - Chaucer > ``Ars longa, vita brevis'' - Hippocrates > Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
What POSIX says about this (which you probably know already) is:
A loop shall enclose a break or continue command if the loop
lexically encloses the command. A loop lexically encloses a
break or continue command if the command is:
[and just paraphrasing the conditions]
- same exxecution environment (ie: not in a subshell)
- inside a compound list associated with a loop (including
the test list for while/until)
- not in the body of a function ...
(more or less the obvious useful definition of lexically enclosing).
That is the case in which break & continue are required to work.
It goes on to say:
If n is greater than the number of lexically enclosing loops
and there is a non-lexically enclosing loop in progress in the
same execution environment as the break or continue command, it is
unspecified whether that loop encloses the command.
In your example, the number of lexically enclosing loops is 0, 'n' is 1,
so the shell is allowed to break from the loop, but not required to.
Or in other words, you should not write code like that, because it
is not guaranteed to work.
oguzismailuy...@gmail.com said:
| `break' is not a keyword in the shell, but a special command.
That's true. However, 99% of script writers don't see it that way,
they believe it is just like "if" or "while" or "done" or "return".
That's why the "This is counterintuitive." I would guess - in most
cases, the issue isn't quite like yours, but more like
f() {
for x in a b c; do
for y in 1 2 3; do
case "$x$y" in
(b3)break;;
(a2)continue 2;;
(c1)break 3;;
esac
done
done
}
and then called something like
for i in 1 2 3 4 5 6
do
echo $i; f
done
and people wonder why that loop never iterates the 2nd time (prints just 1).
All this because the script author probably thought that they were
writing C, and the b3 break is for the case statement, a2 continue
is the y loop, and c1 break the x loop...
They can be taught that that's wrong, break/continue do nothing for
case statements, and don't count them - but they simply refuse to believe
that whatever happens inside the function should affect the caller's
flow - an error from the "break 3" is something they can understand,
as is just ignoring the "overflow", but not more than that.
For people who don't believe that all programming languages should work
the same way (usually the same way as the one they learned first) this
isn't necessarily as important - but that's a tiny majority of people.
kre
ps: the NetBSD shell continues to work the way that you want, and does
so by deliberate choice - our test suite has a whole stack of tests to
make sure this continues to all work "correctly" (doesn't accidentally
get changed).
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On 10/30/21 11:02 AM, Oğuz wrote: On Sat, Oct 30, 2021 at 4:50 PM Greg Wooledge wrote: As Chet said, it's counterintuitive. Most people don't expect function A to be able to affect loops inside function B. I do, and a subshell can prevent function A from affecting loops inside function B. But that is not a real problem, you wouldn't call, say `break 3', when you're only 2 loop levels deep in a function unless you wanted to exit from the loop from within the function is called after returning. It's a violation of scope. It's a violation of lexical scope, I'm asking why not implement dynamic scope, what's wrong with it? You might be interested in https://www.austingroupbugs.net/view.php?id=842 There was a discussion on the austin-group mailing list back in 2016 accompanying this defect report. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
such stuff could be with aliases maybe accomplished, not sure of your req's On Sat, Oct 30, 2021, 17:03 Oğuz wrote: > On Sat, Oct 30, 2021 at 4:50 PM Greg Wooledge wrote: > > As Chet said, it's counterintuitive. Most people don't expect function A > > to be able to affect loops inside function B. > > I do, and a subshell can prevent function A from affecting loops > inside function B. But that is not a real problem, you wouldn't call, > say `break 3', when you're only 2 loop levels deep in a function > unless you wanted to exit from the loop from within the function is > called after returning. > > > It's a violation of scope. > > It's a violation of lexical scope, I'm asking why not implement > dynamic scope, what's wrong with it? > > > > > Can you name *any* other language where functions can break out of their > > caller's loops? The only thing that comes to mind for me is C's > "longjmp", > > which I've never used even once. (Not that I do any C programming these > > days, but back in the 1990s, I did.) > > As far as I know `longjmp' is far more advanced than shell's > break/continue. I can't name another language, but netbsd sh, busybox > sh, and zsh are shells that implement dynamically scoped break and > continue. > > > > > What are you actually trying to do? > > > > Nothing. Idle curiosity. > >
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On Sat, Oct 30, 2021 at 4:50 PM Greg Wooledge wrote: > As Chet said, it's counterintuitive. Most people don't expect function A > to be able to affect loops inside function B. I do, and a subshell can prevent function A from affecting loops inside function B. But that is not a real problem, you wouldn't call, say `break 3', when you're only 2 loop levels deep in a function unless you wanted to exit from the loop from within the function is called after returning. > It's a violation of scope. It's a violation of lexical scope, I'm asking why not implement dynamic scope, what's wrong with it? > > Can you name *any* other language where functions can break out of their > caller's loops? The only thing that comes to mind for me is C's "longjmp", > which I've never used even once. (Not that I do any C programming these > days, but back in the 1990s, I did.) As far as I know `longjmp' is far more advanced than shell's break/continue. I can't name another language, but netbsd sh, busybox sh, and zsh are shells that implement dynamically scoped break and continue. > > What are you actually trying to do? > Nothing. Idle curiosity.
Re: Why should `break' and `continue' in functions not break loops running outside of the function?
On Sat, Oct 30, 2021 at 02:39:19PM +0300, Oğuz wrote: > I found that this behavior had been introduced in 2014, with the > following commit message: > > > set loop_level to 0 > > when entering a function so break and continue in functions don't > > break loops running outside of the function. Fix picked up from > > dash > > and the commit message for the fix in question reads: > > > As it is if you do a multi-level break inside a function it'll actually > > include loops outside of the function call. This is counterintuitive. > So, does anyone know a case where the new behavior can be useful/the > old behavior can be harmful? Am I missing an obvious problem with the > old one? As Chet said, it's counterintuitive. Most people don't expect function A to be able to affect loops inside function B. It's a violation of scope. Can you name *any* other language where functions can break out of their caller's loops? The only thing that comes to mind for me is C's "longjmp", which I've never used even once. (Not that I do any C programming these days, but back in the 1990s, I did.) What are you actually trying to do?
Why should `break' and `continue' in functions not break loops running outside of the function?
I would expect the following to print only 1 and return,
f() { break; }
for x in 1 2 3; do
echo $x
f
done
because `f' is called from within a loop, and executed in the same
environment as the loop. But it doesn't. It prints all 1, 2, and 3;
and complains about `break' being only meaningful in a loop, which is
confusing to me, because it _is_ in a loop.
I found that this behavior had been introduced in 2014, with the
following commit message:
> set loop_level to 0
> when entering a function so break and continue in functions don't
> break loops running outside of the function. Fix picked up from
> dash
and the commit message for the fix in question reads:
> As it is if you do a multi-level break inside a function it'll actually
> include loops outside of the function call. This is counterintuitive.
No further explanation as to how it is counterintuitive. I think it is
intuitive though; unlike with other languages, `break' is not a
keyword in the shell, but a special command. And its scope, in my
opinion, shouldn't be limited to the function body it is called from.
What is counterintuitive instead is that `break 999' (for example)
breaks the outermost loop unless you have a thousand loops nested, but
`break' (and `continue') breaking across functions is not
counterintuitive at all, it feels far more natural and is far easier
to explain than the behavior implemented now.
So, does anyone know a case where the new behavior can be useful/the
old behavior can be harmful? Am I missing an obvious problem with the
old one?
Oğuz
Re: Arbitrary command execution in shell - by design!
Le 30/10/2021 à 07:41, L A Walsh écrivait : On 2021/10/29 12:33, Greg Wooledge wrote: On Fri, Oct 29, 2021 at 11:59:02AM -0700, L A Walsh wrote: How much lameness Chet has introduced into bash to accommodate the wrong users. This is quite unfair. Huh? It's true--look at how functions have to be stored in the environment because someone was able to hack "their own system" where they already have unrestricted shell access. ... Expect to see more of those misplaced security rants now that Bash becomes popular with git-bash and bash in Windows environment. Indeed, if a bash script is able to be abused, it is not bash's fault, neither the script's fault, but the system's security policies granting those privileges to the user running the script. -- Léa Gris
