On 3/7/20 3:38 PM, Chet Ramey wrote: > On 3/6/20 9:05 PM, Rob Landley wrote: >> It turns into echo "=a" "" "bf=" and when I do that from the command line I >> get >> a space before and a space after the NULL argument? But with bash there are >> three? Is it turning into two NULL entries? Ah, it SHOULD do that because >> there >> are two consecutive non-whitespace IFS separators that don't bind to an >> existing >> string (like the first one does). So why _isn't_ mine doing that... > > Your analysis is pretty much spot on. > > The first thing, as you already noted, is that the argument to `echo' is > also being split on $IFS. If you quote that, you see that what you get out > of the for loop is > > =axyxbf= > == > =abc= > > That's because the $@ expands to three arguments, concatenated to what > comes before and after, as described in > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_05_02 > > which can be further split, under certain circumstances. POSIX isn't > exactly precise about which expansions the `word list' following `in' > undergoes ("the list...shall be expanded"), but those don't include word > splitting, so this isn't one of those circumstances.
I read through the posix shell bits long enough ago it was probably SUSv3 rather than v4, but at the moment I'm taking bash as my standard and just doing whatever that does. I'm not looking at the bash source but I'm _scrutinizing_ the bash man page and testing all the corner cases I can think of under both 4.4.12 and 2.05b. (I still need to collate the attached notes with https://github.com/landley/toybox/blob/master/tests/sh.test, and together they're about 1/3 of the tests I need). I should do another pass reading posix afterwards, but after https://landley.net/notes-2016.html#11-03-2016 I've been much less interested in interacting with the posix committee due to the risk of another Schilling, and have pretty much backed up to https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/ in much the same way Debian backed up to LSB 4.1 ala https://lwn.net/Articles/658809/ I still _sort_ of care about newer posix, but I got {bracket,expansion} working last year and last month taught my $IFS splitting to understand utf8 characters (and have a TODO item that if IFS is an array it should understand strings), and I honestly don't expect to live long enough for either NOT to be a divergence from Posix. Sorry to bother you, I should have (and eventually did) figure this out for myself. Rob
https://bash.cyberciti.biz/guide/Select_loop /* alias bg cd command fc fg getopts hash jobs kill read type ulimit umask unalias wait exit if while for case export set unset trap exec function source TOY_MAYFORK: echo test printf no memory allocations, filehandles open... if signaled, no cleanup. maybe: help ulimit .toysh_history file {abc,def} $LINENO $RANDOM "source" and "." check current dir after $PATH job control: CTRL-Z, CTRL-C, fg, bg, kill %jobspec, wait $(command) $((math)) # integer attribute for variables, assignment does that $((abc"def)) the " doesn't take effect so )) is recognized? if then elif else fi for while until do done case esac function () { commands ; } local variables in functions set -x (and -x command line option) to trace export -n (unexports) readonly read set unset umask time (whole pipeline) command trap type times getopts todo: echo | (cat) # as far as cat's concerned previous end was NULL What to do about builtin vs $PATH: check $PATH first _then_ builtin? config opt? if files exist use 'em, otherwise: /dev/fd/fd stdin stdout stderr tcp/host/port udp/host/port exec redirects in current shell n<word n>word n>>word (append) &>word # stdout and stderr, same as >word 2>&1 &>>word (append) >&word # note: >&$BLAH when $BLAH is number or - n<&word # number = duplicate, - = close, number- = move n>&word # special case: no n and word not number or -, redirect stderr & out. n<<word # only quote removal, no expansion # unquoted: parameter ex, command subst, arith ex, \\ \$ \` \newline <<-word # strips leading tabs <<<word # all expansions _except_ pathname expansion and word splitting n<>word # open file word for reading & writing, create if necessary, n def 0 // &> or >& = >word 2>&1 // &>> append stdout & stderr == >> word 2>&1 // <&# copy input fd (error if # not open for read) // >&# copy output fd (error if # not open for write) // <&#- move fd# to n // >&#- move fd# to n What does this mean? env {abcdef}>&2 | grep abcdef Instead of number, precede with {varname}? brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, quote removal, pathname expansion, and word splitting. << no path ex or word split test should work as a nofork, handle [[ what does $(( )) handle ( ( echo hello ) ) | cat and (echo)|cat if (then) else elif (then) fi for select while until (do) done case (esac) ( ) { } [[ ]] } // function // HERE document // ( ) - ) is always last argument of line // | & && || |& // "if", "for", "while", "until", "select" // "do" : "then"; // then else elif fi // continue break case esac return do done function // in time { } ]] ]] // ! coproc // function () // . source [ test [[ ( { (( // [ pwd ulimit -- kill time // cd exit -- pushd popd logout umask getopts eval exec // fg bg jobs disown wait suspend // alias unalias set export unset let local readonly read shift trap # Because of HERE documents we can't re-parse, have to retain state. # -- either that or our caller has to know about here document servicing? # -c input must be split into lines for HERE documents < > >> <<< WORD << HERE [HERE] # eat lines until HERE as _only_ word on line if [then] for while until select [do] case [esac] ( [)] # funky because ) not first word { [}] [[ []]] function NAME ( ) { # note expect { doesn't eat { be because need to expect } /* 4 cases: redirect now: from -> to, saving displaced to (+, +) = hfd to explicitly close (now), saving displaced to (-1, +) = hfd to close in child only: (+, -1) redirect saving to, close saving from: (+, +) = hfd to then (-1, +) = hfd to {var} leak - nothing to save? Except the from open is deferred (+, +) but to == hfd */ // TODO job control: & backgrounding // TODO && || // TODO | |& // TODO add redir segment, redir pipes[1] to pipes[0] with close, pop again later // TODO pipe segments are subshells ala ( ) // TODO: don't () around single process pipeline, shell otherwise // this turns into exec for sh -c "echo" // single pipe segments that _aren't_ builtins run directly, // compound pipe segments and builtins run via subshell. // I.E. subshell implicit exec // TODO we're not running a command, we're running a block stack? // echo one two three | while read i; do echo hello; done // TODO free/cleanup partial pipeline on NULL return? // TODO -o pipefail // TODO pipes don't connect commands, they connect arbitrary function chunks. */ # can't have space before first : but arguments can have lead/trail $ BLAH=abcdefghi; echo ${BLAH: 1 : 3 } # leading assignments don't affect current command line $ VAR=12345 echo $VARa $ bash -c 'ls $(' bash: -c: line 0: unexpected EOF while looking for matching `)' bash: -c: line 1: syntax error: unexpected end of file $ ; bash: syntax error near unexpected token `;' $ ABC= ; env | grep ABC= ; unset ABC ; env | grep ABC= ABC= $ cat blah<(echo hello)thing cat: blah/dev/fd/63thing: No such file or directory $ "AB"="CD" echo $AB bash: AB=CD: command not found landley@driftwood:~/toybox/toybox$ AB="CD" echo $AB $ 'AB'="CD" echo $AB bash: AB=CD: command not found $ \AB="CD" echo $AB bash: AB=CD: command not found $ \AB="CD" echo $AB $ if echo hello; then fi bash: syntax error near unexpected token `fi' $ echo ) bash: syntax error near unexpected token `)' $ ( echo hello ) | cat hello $ if > true > then > echo hello > fi hello $ if true; then echo hello; fi hello $ if false; then echo hello; fi $ false; X=47; echo $? 0 $ if; bash: syntax error near unexpected token `;' $ if > true > echo > hello > fi bash: syntax error near unexpected token `fi' $ echo ))) bash: syntax error near unexpected token `)' $ echo &&& bash: syntax error near unexpected token `&' $ echo ||| bash: syntax error near unexpected token `|' $ echo hello | if true; then read i; echo i=$i; fi i=hello $ if true && false; then echo hi; fi while true ; do echo hello; done | tee blah if { echo hello; }; then echo hi; fi if if true; then true; fi; then echo hello; fi boom(){ echo hello; }; boom ( ( echo; thingy ) ) | cat $ echo hello | X=y env | grep -w X; echo $X X=y ;|x ;x| (echo)also (echo)(echo) $ if true esac; then echo hi; fi hi # note: exits top level shell anyway! $ (exit walrus) bash: exit: walrus: numeric argument required $ echo $? 2 $ exit walrus 2>blah.txt $ echo ${abc:?error m essage} bash: abc: error m essage $ if blah () { echo bang; true; }; blah; then echo hello; fi; blah bang hello bang $ meep() { echo hello; klarg () { echo helloier; }; klarg; } $ meep hello helloier $ meep() { echo hello; klarg() { echo helloier; } ; } $ klarg bash: klarg: command not found $ meep hello $ klarg helloier $ landley@driftwood:~/toybox/toybox$ thingy () > ^C landley@driftwood:~/toybox/toybox$ echo ( bash: syntax error near unexpected token `newline' landley@driftwood:~/toybox/toybox$ function bash: syntax error near unexpected token `newline' landley@driftwood:~/toybox/toybox$ function name () { > } bash: syntax error near unexpected token `}' landley@driftwood:~/toybox/toybox$ function name () { ;} bash: syntax error near unexpected token `;' landley@driftwood:~/toybox/toybox$ if ; then; echo hello; fi bash: syntax error near unexpected token `;' landley@driftwood:~/toybox/toybox$ func (); { echo hello; } bash: syntax error near unexpected token `;' $ if true | > then bash: syntax error echo one && if true; then echo hello; fi one hello $ echo one two \ > three $ bash -c "$(echo -e 'cat << HERE\none two\nHERE')" one two $ cat << HERE; echo hello > boing > HERE boing hello $ cat << one << two > abc $ ls > file 2>&1 #redirects both $ ls 2>&1 >file # redirects only stdout, stderr goes to original stdout $ echo hello < nowhere bash: nowhere: No such file or directory $ cat << E"O"F > $PATH > EOF $PATH $ cat << HERE filename > HERE cat: filename: No such file or directory $ cat <<< here filename cat: filename: No such file or directory landley@driftwood:~/toybox/toybox$ bash -c "cat <<< HERE; echo hello; HERE" HERE hello bash: HERE: command not found landley@driftwood:~/toybox/toybox$ bash -c "cat << HERE; echo hello; HERE" bash: warning: here-document at line 0 delimited by end-of-file (wanted `HERE') hello bash: HERE: command not found landley@driftwood:~/toybox/toybox$ bash -c "$(echo 'cat << HERE\necho hello\nHERE')" bash: warning: here-document at line 0 delimited by end-of-file (wanted `HEREnecho') cat: hellonHERE: No such file or directory landley@driftwood:~/toybox/toybox$ bash -c "$(echo -e 'cat << HERE\necho hello\nHERE')" echo hello landley@driftwood:~/toybox/toybox$ X=3; echo $((X)) 3 landley@driftwood:~/toybox/toybox$ X=3; echo $((X))^C landley@driftwood:~/toybox/toybox$ echo | > > cat landley@driftwood:~/toybox/toybox$ echo && > && bash: syntax error near unexpected token `&&' landley@driftwood:~/toybox/toybox$ echo && > ; bash: syntax error near unexpected token `;' landley@driftwood:~/toybox/toybox$ if > true > then > echo hello > fi hello landley@driftwood:~/toybox/toybox$ << EOF > echo hello > EOF $ echo &<2 [1] 17850 bash: 2: No such file or directory landley@driftwood:~/toybox/toybox$ [1]+ Done echo landley@driftwood:~/toybox/toybox$ { echo hello; } hello landley@driftwood:~/toybox/toybox$ { { echo hello; };} hello landley@driftwood:~/toybox/toybox$ {{ echo hello; };} bash: syntax error near unexpected token `}' landley@driftwood:~/toybox/toybox$ { {echo hello; };} bash: syntax error near unexpected token `}' landley@driftwood:~/toybox/toybox$ { { echo hello; };} hello landley@driftwood:~/toybox/toybox$ { > { > echo hello > } > } hello $ if cat << EOF ; then > one two three > EOF > echo hello > fi one two three hello cat <(ls "$PWD"/a{b,c}*) &> /dev/tcp/127.0.0.1/80 $ while grep -q << EOF walrus; do > walrus > EOF > echo hello; done | head -n 3 hello hello hello $ if echo hello; then echo also; fi | tee and.txt # both get redirected hello also $ cat - << EOF <(echo > hello) > boing > EOF bash: hello: command not found boing $ ( ( ( echo abc ) echo ) ) bash: syntax error near unexpected token `echo' $ ( ( ( echo abc ) > echo also ) ) abc also $ ( ( echo one ) > file ) $ ( ( echo one ) > two ) > three $ ( echo ) >> blah $ if true; then echo; fi >> blah $ cat - /proc/self/fd/3 << BOING 3<<MEEP > one > BOING > aha > MEEP one aha $ cat - << BOING $( echo hello) abc BOING abc blah $ ( cat << EOF > hello > EOF > ) hello $ if cat << EOF > hello > EOF > then > echo fi > fi hello fi $ echo 2>&1 $ echo 2 >&1 $ echo 2 >& 1 $ if while true; do echo hello; done; then echo hi; fi $ (echo &&) $ (echo ;) $ if true; then echo $X; fi {X}</dev/null 10 $ X=2; {X}<&-; boing # does not close stderr? bash: boing: command not found $ ls /proc/$$/fd $ exec 10<& - # >&; ;&> # prompts before echoing $ echo hello; if > echo two; then echo three > fi hello two three $ echo )hello bash: syntax error near unexpected token `)' $ echo {abc {abc $ echo {abc</dev/null {abc # innermost redirect wins $ if cat <<< moo ; then cat <<< also; fi <<< potato moo also $ (echo &&) bash: syntax error near unexpected token `)' $ funkiness() { env ; } $ POTATO=blah funkiness | grep POTATO POTATO=blah $ echo $POTATO $ chicken() { echo hello; chicken() { echo also ;}; chicken;} $ chicken hello also $ chicken also $ thingy() { echo hello; } | cat $ thingy bash: thingy: command not found $ echo hello > >(sed 's/.*/abc&def/') > >(sed 's/.*/ghi&jkl/') abcghihellojkldef $ echo hello > one > two $ yes | head -n 3 > >(wc -l >two) > >(wc -l >one) landley@driftwood:~$ cat one 3 landley@driftwood:~$ cat two 0 $ </dev/null echo hello hello $ echo {+}</dev/null {+} $ echo {}</dev/null {} $ echo < bash: syntax error near unexpected token `newline' $ murgle() { echo > "$@" ;} $ murgle one two three bash: "$@": ambiguous redirect $ murgle one $ cat << " " > abc > abc $ cat << \" > boing > " boing $ echo hello &>> blah.txt $ potato() { echo "$@"; }; IFS=1234 potato one two three one two three $ cat <<< {one,two,three} {one,two,three} $ cat <<< one two three $ echo hello >&2-x && ls 2-x 2-x $ while true; do sleep 1; read a; echo $a; [ -z "$a" ] && break; done << EOF & > one > two > three > EOF [1] 30590 $ (set -o noclobber; echo > /dev/null) $ (ln -s /dev/null blah; set -o noclobber; echo > blah) $ (touch walrus; ln -s walrus penguin; set -o noclobber; echo > penguin) $ funky() { A="$@" ;}; funky one two three; echo $A one two three $ export "()=42" bash: export: `()=42': not a valid identifier $ set hello="$@" > walrus $ X=$(false) || echo true true $ for((i=1;i<5;i++));do echo $i;done 1 2 3 4 # when is (( a token vs ( (? $ ((echo a) | sed s/a/b/) b $ ((1 > +2<1)); echo $? 1 $ X=ii;for $X in a b c; do echo $ii; done bash: `$X': not a valid identifier $ for i in 1 2 3 & do echo hello; done bash: syntax error near unexpected token `&' $ if true; then false; else true; elif true; then false; fi bash: syntax error near unexpected token `elif' $ while false; true; do echo hello; done | head -n 3 hello hello hello $ せ=42 bash: せ=42: command not found $ function abc=def () {echo hello;} ; "abc=def" hello $ ((echo hello) ) hello $ abc() { for name; do echo $name; done;}; abc X Y Z X Y Z # In a for loop ;; breaks down to ; ; $ for((i=0;;i++)); do echo $i; done | head -n 30 1 2 $ (()); echo $? 1 $ ((x=12)); echo $x 12 $ (echo 123) # redir prefix not used for non-redir end tokens 123 $ x=42; ! ((x<3)) && ((x<43)) && echo yes yes # redirects aren't in math $ echo $((2<3)) 1 $ echo $((2&3)) 2 $ ((3<2)); echo $? 1 $ X=X; echo $((X+2)) bash: X: expression recursion level exceeded (error token is "X") $ X=3 ((3<2)) || echo hello bash: syntax error near unexpected token `(' $ echo $((`echo 1`)) 1 $ X=3; (($X<3)) || echo no; (($X<4)) && echo yes no yes $ cat <(echo "$(((1<2)) && echo "hello")") hello $ THINGY=whoami; echo "$("${THINGY}")" landley $ burble() #comment > { echo hello;};burble hello $ for i=3 > do echo hello; done bash: `i=3': not a valid identifier $ ((1<2)) > blat $ boing () { for name do echo $name; done }; boing one two three one two three $ cat <( "(" ) bash: (: command not found $ break > potato 2>/dev/null || ls potato potato # too many error messages $ break 1 37 bash: break: only meaningful in a `for', `while', or `until' loop $ for i in 1; do break walrus; done bash: break: walrus: numeric argument required $ for i in 1; do break 37 walrus; done bash: break: too many arguments $ for i in 1; do break 0; done bash: break: 0: loop count out of range $ if true; then if false; then echo one; elif ! echo two; then echo three; else echo four; fi; fi two four $ cat boing |& sed s/boing/walrus/ cat: walrus: No such file or directory # runs printing "hi" until segfault bash -c "abc() { echo hi; def() { abc;echo lo;};def;}; abc" # word splitting test $ X="one two"; printf '%s\n' $X 2 3 | while read a; do echo $a; done one two 2 3 $ fff() { echo abc$*def;}; fff one two three | xargs -n 1 echo abcone two threedef $ ABC=abcdefg $ echo ${ABC:+"}"} } $ for i in a b c d e; do break & done [1] 25533 [2] 25534 [3] 25535 [4] 25536 [5] 25537 $ walrus=1; echo ${walrus+blah} blah $ unset walrus; echo ${walrus+blah} $ $ echo echo hello | sh hello $ { { { echo hello;} ;} >blat ;} ; echo one; cat blat one hello $ ABC=1 $ echo ${"AB"C} bash: ${"AB"C}: bad substitution $ echo ${"ABC"} bash: ${"ABC"}: bad substitution $ X='*'; echo $X $ <&2-; ls boing # redirect undone at end of command ls: cannot access 'boing': No such file or directory $ ls boing <&2- # redirect applies to command $ X=42 </dev/null ; echo $X # local assignment persists even with redirect 42 $ bash -c 'read i >/dev/null <<< boing; echo $i' boing # The expansions unquoted << _doesn't_ do. $ cat << TEST > ~landley > * > potato/{one,two,three} > "hello" > TEST ~landley * potato/{one,two,three} "hello" $ X="|"; echo hello $X and hello | and $ for i in a b c; do for j in d e f; do for k in g h i; do echo $i $j $k; continue 3; done done done a d g b d g c d g $ sh -c '<&3-' sh: 1: Syntax error: Bad fd number $ {abc}<&2- # leaves $abc open even when 2- unwound $ {def}<&2 # doesn't close 2 $ {bcd}<walrus $ {cde}<<EOF hello EOF $ echo {fgh}<<<potato another; echo hi; cat <&$fgh another hi potato $ <&2- <&37 ; echo $? 1 $ echo hello {var}<nonexistent # doesn't set var on error $ var=wurble; echo hello {var}<nope # doesn't blank either $ X="echo hello"; $X hello $ > /does/not/exist && echo $? 1 $ { echo -e "one\ntwo" ;} | { if true; then read i; echo a=$i;fi;if true; then read i; echo b=$i;fi;} a=one b=two $ if false; then if false; then echo one; else echo two; else echo three; fi # disabled on block entry disables entire block: else doesn't revive $ if false; then if false; then echo one; else echo two; fi; else echo three; fi three $ { { echo hello ;} | } bash: syntax error near unexpected token `}' $ {|{ echo hello;};} $ echo hello | if false; then cat; fi | cat $ blah() { echo hello; } | echo ha; blah ha bash: blah: command not found $ X=42 | true; echo $X $ if echo hello; then true; fi | tr l x hexxo # assignment persists, redirect does not $ X=1 >woot; echo did not persist; echo $X did not persist 1 $ echo <(if) bash: command substitution: line 22: syntax error near unexpected token `)' bash: command substitution: line 22: `if)' # this shows error immediately but doesn't exit for 10 seconds... $ sleep 10 | cat < notit bash: notit: No such file or directory $ echo {"on,t"w}e {on,tw}e $ func() { echo $0,$1; shift; echo $0,$1;}; func one two three bash,one bash,two $ X=123 cat <(echo $X) $ env -i bash --norc --noprofile -c env | sort PWD=$(pwd -P) SHLVL=1 _=/usr/bin/env $ echo {~,~root}/pwd /home/landley/pwd /root/pwd $ echo \{~,~root}/pwd {~,~root}/pwd $ echo ""{~,~root}/pwd ~/pwd ~root/pwd $ A= ABC=123; echo $A{BC,""} 123 $ A=; echo $A''C C $ echo $((x=3)); echo $x 3 3 $ y=-4; echo $((x=y)); echo $y -4 -4 $ x=x; echo $((x+1)) bash: x: expression recursion level exceeded (error token is "x") $ expr 1 + 2x expr: non-integer argument $ echo $((1+_)) # because $_ bash: 2x: value too great for base (error token is "2x") $ echo hello | cat <(read i; echo $i) hello # errors propagate up but return code doesn't $ echo -n thingy $(if true) bash: command substitution: line 2: syntax error near unexpected token `)' bash: command substitution: line 2: `if true)' $ echo $? 1 $ echo -n $(false) $ echo $? 0 $ echo -n $(true < walroid) bash: walroid: No such file or directory landley@driftwood:~/toybox/toybox$ echo $? 0 $ walrus=42; readonly walrus; walrus=7; echo $? bash: walrus: readonly variable 1 $ walrus=42; readonly walrus; walrus=7 echo hello; echo $? bash: walrus: readonly variable hello 0 $ ./bash -c 'echo $_' ./bash $ PATH=$PWD:$PATH bash -c 'echo $_' $PWD/bash $ PATH=$PWD:$PATH PWD=/bin bash -c 'echo $_' $OLDPWD/bash $./sh -c 'cat <(echo hello 2>&1)' hello $ $ chicken() { for i in a"$@"b;do echo =$i=;done;}; chicken 123 456 789 =a123= =456= =789b= # I hate IFS $ THINGY=$'potatoxsalad\tand this\n\n\n' IFS='x '; for i in $THINGY; do echo =$i=; done =potato= =salad and= =this =
_______________________________________________ Toybox mailing list [email protected] http://lists.landley.net/listinfo.cgi/toybox-landley.net
