Re: Bash parameter expansion (remove largest trailing match, remove largest leading match, pattern replacement) does not work
On 8/29/20 9:12 PM, Lawrence Velázquez wrote: >> Actually, it works (portably) with >> separator2=$'\057' > > (a) $'...' is not POSIX. For instance, dash does not recognize it. I believe it made the cut and will be in the next edition of the standard. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash parameter expansion (remove largest trailing match, remove largest leading match, pattern replacement) does not work
On 8/29/20 3:54 PM, Bruce Lilly wrote: > It's a bit more complicated than that; if, for example, some excerpt ended > up in regression tests, there would be a question about whether or not > there was a copyright violation. This is not at all the same thing, and could (and would) be handled separately. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash parameter expansion (remove largest trailing match, remove largest leading match, pattern replacement) does not work
On 8/29/20 3:25 PM, Bruce Lilly wrote: > Unfortunately, because bash is GPL, I can't post the copyrighted script > which is covered by a non-GPL license. This is ridiculous on its face. A script that bash executes doesn't have any relevance to bash's copyright. You might have reasons for not wanting to publicly post a copyrighted script, but bash being GPL is not one of them. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash parameter expansion (remove largest trailing match, remove largest leading match, pattern replacement) does not work
On 8/29/20 10:22 AM, Bruce Lilly wrote: > Bash Version: 5.0 > Patch Level: 17 > Release Status: release > > Description: >Bash parameter expansion (remove largest trailing match, remove > largest leading match, pattern replacement) does not work > Tested on OpenSUSE Leap 15.2, bash version 4.4.2.3(1)-release > (x86_64-suse-linux-gnu) > OpenBSD 6.7 bash version 5.0.17(1)-release (x86_64-unknown-openbsd6.7) > NetBSD 9.0 bash version 5.0.17(1)-release (x86_64--netbsd) > FreeBSD 12.1-STABLE bash version 5.0.18(2)-release (amd64-portbld-freebsd12.1) > > Same results in all cases; this report posted from NetBSD 9.0. There are a number of things wrong with this report; I think later messages in the thread cover them. In short, 1. BRE bracket expression matching doesn't perform backslash-interpretation of anything, except to quote the next character, much less octal constant expansion; 2. You need `shopt -s extglob' to enable the pattern matching you want to use. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Incorrect / Inconsistent behavior with nameref assignments in functions
On 8/28/20 11:28 AM, Greg Wooledge wrote: > > You've got a local variable with the same name as the global variable > that you're attempting to pass by reference. This will not work. > > Namerefs (declare -n) in bash are *not* like uplevel commands in Tcl. > They cause the referenced variable name to be evaluated just like any > other variable would be, starting at the current function scope, then > going up to the caller, and so on. Namerefs don't change bash's dynamic variable scoping. The two conflict and make namerefs less useful than they might be. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Incorrect / Inconsistent behavior with nameref assignments in functions
On 8/28/20 4:56 AM, Binarus wrote: > Bash Version: 4.4 > Patch Level: 12 > Release Status: release > > > Description: > > > Under certain circumstances, assignments of namerefs to local variables > in functions behaves in a way which makes namerefs completely useless. > Furthermore, the behavior is not consistent. There is an order of evaluation problem as explained later in the thread, not specific to namerefs. It's fixed in bash-5.1. The underlying issue is making `declare [options] foo=bar' expand the argument like an assignment statement as POSIX specifies. This makes `declare' more like a hybrid reserved word instead of a builtin. In some cases, the options matter and affect how the argument gets expanded. You end up having to do something like `declare [options] foo; foo=bar' to get the expansion right, but that introduces several corner cases. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: [PATCH] Fix segfault with self-modifying array PROMPT_COMMAND
On 8/30/20 9:36 PM, Koichi Murase wrote: > Bash Version: 5.1 > Patch Level: 0 > Release Status: release > > Description: > > In the devel branch (e4d38c2d: commit bash-20200819 snapshot), a > segmentation fault occurs when a prompt command stored in the array > version of PROMPT_COMMAND unsets the corresponding element in > PROMPT_COMMAND. Thanks for the report. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: [PATCH] Bash 5.0+: Fix a problem that interactive sessions close with `eval {'
On 8/27/20 8:33 PM, Koichi Murase wrote: > Bash Version: 5.1 > Patch Level: 0 > Release Status: release > > Description: > > When a command string with an incomplete construct is supplied to > the `eval' builtin in interactive sessions, the process will be > terminated. This does not happen before Bash 5.0. This does not > happen in non-interactive modes. Thanks for the report. I appreciate your attention to detail. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash parameter expansion (remove largest trailing match, remove largest leading match, pattern replacement) does not work
On Sat, Aug 29, 2020 at 03:25:44PM -0400, Bruce Lilly wrote: > Unfortunately, because bash is GPL, I can't post the copyrighted script > which is covered by a non-GPL license. On the extremely slim chance that you are genuinely confused, and not just a troll: The GPL covers bash. The GPL does NOT cover your script, written by you, which includes #!/bin/bash or whatever shebang at the top. Your script is your own work, and you hold the copyright. You get to decide under what license other people may distribute copies of your script. However, because I feel the chances of you not being a troll are so very slim, I'm just going to delete the rest of the unread messages in this thread immediately.
Re: Incorrect / Inconsistent behavior with nameref assignments in functions
On 30.08.2020 20:01, Koichi Murase wrote:
> 2020-08-30 19:24 Binarus :
> Yes, I recognize the problem when the function isn't properly used.
> But, the use of eval itself is not fatal. When another user can call
> the function as in your example,
>
> Dummy 'myArray1[@]}"); echo Gotcha!; #'
>
> that means that the user can already run arbitrary commands. The user
> could have directly written
>
> echo 'Gotcha!'
>
> The real problems occur when the user write like
>
> Dummy "$input_to_program"
>
> with `input_to_program' provided by the third user who should not be
> able to run arbitrary commands, and the input is not checked nor
> sanitized. In this case, the problem should be evaded by checking or
> sanitizing the input. This check can be made inside the function
> Dummy, but it is also possible to make it at the time when the shell
> receives the input.
Thank you very much for the clarification. Of course, you are right. If
I would provide the functions in a file which can be sourced by users,
security isn't a problem at all (except in cases where users
accidentally make mistakes which lead to the HDD being purged ...). But
if I wrap them in a script and provide the interface via command line
parameters, and that script runs SUID, this imposes a new class of problems.
>> declare -a -i myArray1=('1' '2' '3')
>> Dummy 'myArray1[@]}"); echo Gotcha!; #'
>>
>> Output:
>>
>> root@cerberus:~/scripts# ./test6
>> Gotcha!
>> declare -a myArray=([0]="1" [1]="2" [2]="3")
>
> Unfortunately, your original function `Dummy' also has the same
> vulnerability. As Greg has written, there are many other places that
> cause the command execution in the shell because that is the purpose
> of the shell. With your original method,
>
> $ cat testR2d.sh
> function Dummy {
> local -n namerefArray="$1"
> local -a -i myArray=("${namerefArray[@]}")
> }
> myArray1=("$@")
> Dummy 'myArray1'
> $ bash testR2d.sh 'a[$(echo Gotcha1 >/dev/tty)]'
> Gotcha1
>
> This is caused by the arithmetic evaluation caused by `-i' flag. In
> arithmetic evaluations, the array subscripts are subject to the extra
> expansions so that the string $(echo Gotcha1 >/dev/tty) is expanded as
> a command substitution.
>
> Actually, the nameref also has the same behavior, so the use of
> `nameref' is not much safer than the use of `eval'.
>
> $ cat testR2e.sh
> function Dummy2 {
> local -n namerefScalar=$1
> local var=$namerefScalar
> }
> Dummy2 "$1"
> $ bash testR2e.sh 'a[$(echo Gotcha2 >/dev/tty)]'
> Gotcha2
>
> Yes, I guess it would be a valid strategy to disallow any use of
> `eval' because humans will make mistakes no matter how careful we are.
> But, there are still different traps, so anyway we need to carefully
> check or sanitize inputs even when we don't use `eval'.
Thank you very much for these insights! Actually, I didn't think
systematically about command line interfaces and safety yet, because
these scripts won't run SUID and don't provide command line interfaces.
However, the eval code injection stared me in the face, so I thought I'd
comment about it. Probably I shouldn't have done so, because I actually
at least don't have that problem at the moment, and because I don't
intend to to harden theses scripts anyway; as it currently stands, only
root will be able to execute them.
>> I guess it would be very complicated, if possible at all, to protect
>> the code inside eval against every sort of such attacks.
>
> I think the standard way is to check the input before passing it to
> `eval' and is not complicated. You can just check if the array name
> has an expected form:
>
> function is-valid-array-name {
> local reg='^[_[:alpha:]][_[:alnum:]]*$'
> [[ $1 =~ $reg ]]
> }
>
> # Check it inside Dummy
> function Dummy {
> is-valid-array-name "$1" || return 1
> [[ $1 == myArray ]] || eval "local -a myArray=(\"\${$1[@]}\")"
> declare -p myArray
> }
>
> # Or check it when it receives the array name (I prefer this)
> is-valid-array-name "$1" || exit 1
> input_data=$1
> Dummy "$input_data"
Thank you very much. These snippets are nice and clean. I'll file all
your messages and code patterns and use them as reference if I ever
write a bash script again.
>>> * If you want to use namerefs to eliminate the use of `eval', maybe
>>> you could do like the following [...]
>>
>> However (hoping that I don't get flamed for a dumb question), I
>> don't understand why we need inputArray at all in that code.
>
> Sorry, I should have explained it in detail. The step of `inputArray'
> is only needed when you want to modify `myArray' locally in the
> function `Dummy' keeping the original array unmodified. Without
> `inputArray',
>
> $ cat testR2f.sh
> function Dummy {
> [[ $1 == refArray ]] || local -n refArray=$1
> [[ $1 == myArray ]] || local -ia myArray=("${refArray[@]}")
> myArray[0]=${myArray[0]%/}
> }
> myArray=(my/dir/)
>
Re: Incorrect / Inconsistent behavior with nameref assignments in functions
On 30.08.2020 16:50, Greg Wooledge wrote: > The evil thing here is code injection. Obviously eval is one way to > perform code injection, but it's not the *only* way. Eval itself isn't > evil; if anything, it's all of the other forms of code injection, > which people don't suspect, that are truly insidious. > > https://mywiki.wooledge.org/CodeInjection > https://mywiki.wooledge.org/BashWeaknesses > > You're trying to do something that you feel should be possible -- passing > an array to a function by reference. Every other language can do this, > right? So bash should be able to do this... right? Nope. > > Passing variables by reference (especially arrays) is one of the > major missing features of bash. Everyone wants it. Many, many people > have attempted it. The sheer insanity of some of the attempts is > astounding. > > https://fvue.nl/wiki/Bash:_Passing_variables_by_reference > > That's a slightly older page, but he found an exploit in "unset" which > does bizarre things when called at different function scope levels, and > managed to use it to manipulate the existence of variables at various > function scopes. > > If you absolutely *need* to pass a variable by reference, don't use bash. > That's the best advice I can give you. You are absolutely right, and I have understood this in the meantime. Unfortunately, there is a substantial amount of work (and thus, money) in these scripts, and there is a time line, so the moment where I could dump bash for Perl or Python has passed some time ago. Hence, I really have to finish these bash scripts, but I have learned my lesson and in the future won't use bash for anything that is more complex than a one-liner. Even though bash 5.1 seems to solve my current problem, I suspect that there are more surprises like this which I just haven't come across yet. Thank you very much, and best regards, Binarus
