Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-04 Thread Herbert Xu
On Wed, Apr 04, 2018 at 06:27:45PM +0200, Denys Vlasenko wrote: > > This does not match what happens with this simple heredoc: > > cat < \" > EOF > > # dash zz > \" Right, the special double quote part only comes into play inside backquotes. The reason is that the parsing of backquotes is done

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-04 Thread Denys Vlasenko
On Wed, Apr 4, 2018 at 5:13 PM, Herbert Xu wrote: >> This is the example which is not affected by echo differences: >> >> cat <> -\"-\'-\`-\--\z-\*-\?- >> `echo '-\"-\x-\`-\--\z-\*-\?-'` >> $(echo '-\"-\x-\`-\--\z-\*-\?-') >> EOF >> >> # bash z >> -\"-\'-`-\--\z-\*-\?- >> -\"-\x-`-\--\z-\*-\?- >>

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-04 Thread Herbert Xu
On Wed, Apr 04, 2018 at 01:56:42PM +0200, Denys Vlasenko wrote: > > This is the example which is not affected by echo differences: > > cat < -\"-\'-\`-\--\z-\*-\?- > `echo '-\"-\x-\`-\--\z-\*-\?-'` > $(echo '-\"-\x-\`-\--\z-\*-\?-') > EOF > > # bash z > -\"-\'-`-\--\z-\*-\?- > -\"-\x-`-\--\z-\*-

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-04 Thread Denys Vlasenko
On Tue, Apr 3, 2018 at 3:35 AM, Herbert Xu wrote: > On Mon, Apr 02, 2018 at 08:57:35PM +0200, Denys Vlasenko wrote: >> >> I see this: >> >> $ x='\t'; echo "[$x]" >> [] >> >> I'm pretty sure this isn't okay... > > Why not? dash has always behaved like this and this is explicitly > required by P

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-02 Thread Herbert Xu
On Mon, Apr 02, 2018 at 08:57:35PM +0200, Denys Vlasenko wrote: > > I see this: > > $ x='\t'; echo "[$x]" > [] > > I'm pretty sure this isn't okay... Why not? dash has always behaved like this and this is explicitly required by POSIX: http://pubs.opengroup.org/onlinepubs/9699919799/utilitie

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-02 Thread Denys Vlasenko
On Mon, Apr 2, 2018 at 7:04 PM, Herbert Xu wrote: > On Mon, Apr 02, 2018 at 02:13:03PM +0200, Denys Vlasenko wrote: >> >> I was trying some of the beautiful atrocities from one of earlier >> Harald's emails and this one fails in current dash git: >> >> # x=""; echo "${x#"${x+''}"''}" >> dash:

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-02 Thread Denys Vlasenko
On Mon, Apr 2, 2018 at 7:08 PM, Herbert Xu wrote: > On Mon, Apr 02, 2018 at 03:48:46PM +0200, Denys Vlasenko wrote: >> >> Another probably buggy case: >> >> cat <> -\t-\\-\"-\'-\`-\--\z-\*-\?- >> `echo '-\t-\\-\"-\x-\`-\--\z-\*-\?-'` >> $(echo '-\t-\\-\"-\x-\`-\--\z-\*-\?-') >> EOF >> >> >> # bas

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-02 Thread Herbert Xu
On Mon, Apr 02, 2018 at 03:48:46PM +0200, Denys Vlasenko wrote: > > Another probably buggy case: > > cat < -\t-\\-\"-\'-\`-\--\z-\*-\?- > `echo '-\t-\\-\"-\x-\`-\--\z-\*-\?-'` > $(echo '-\t-\\-\"-\x-\`-\--\z-\*-\?-') > EOF > > > # bash heredoc_backslash.test > -\t-\-\"-\'-`-\--\z-\*-\?- > -\t-\

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-02 Thread Herbert Xu
On Mon, Apr 02, 2018 at 02:13:03PM +0200, Denys Vlasenko wrote: > > I was trying some of the beautiful atrocities from one of earlier > Harald's emails and this one fails in current dash git: > > # x=""; echo "${x#"${x+''}"''}" > dash: 12: Syntax error: Missing '}' Thanks for the report. Thi

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-02 Thread Denys Vlasenko
On Mon, Apr 2, 2018 at 2:13 PM, Denys Vlasenko wrote: > I was trying some of the beautiful atrocities from one of earlier > Harald's emails and this one fails in current dash git: > > # x=""; echo "${x#"${x+''}"''}" > dash: 12: Syntax error: Missing '}' Another probably buggy case: cat

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-04-02 Thread Denys Vlasenko
On Sat, Mar 10, 2018 at 3:04 AM, Harald van Dijk wrote: > On 3/8/18 1:40 AM, Harald van Dijk wrote: >> >> If the syntax stack is to be stored on the actual stack, then real >> recursion could be used instead, as attached. > > > Even though it won't be accepted in dash, I continued with this approa

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-09 Thread Harald van Dijk
On 3/8/18 1:40 AM, Harald van Dijk wrote: If the syntax stack is to be stored on the actual stack, then real recursion could be used instead, as attached. Even though it won't be accepted in dash, I continued with this approach for my own use. I've now got it to about 1800 bytes smaller (at -O

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-08 Thread Harald van Dijk
On 08/03/2018 08:55, Herbert Xu wrote: On Thu, Mar 08, 2018 at 01:40:32AM +0100, Harald van Dijk wrote: If the syntax stack is to be stored on the actual stack, then real recursion could be used instead, as attached. I tried to avoid recursion for the cases that unpatched dash already handled p

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-07 Thread Herbert Xu
On Thu, Mar 08, 2018 at 01:40:32AM +0100, Harald van Dijk wrote: > > If the syntax stack is to be stored on the actual stack, then real recursion > could be used instead, as attached. I tried to avoid recursion for the cases > that unpatched dash already handled properly. It does look a lot simple

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-07 Thread Harald van Dijk
On 3/7/18 8:04 PM, Harald van Dijk wrote: On 3/7/18 5:29 PM, Herbert Xu wrote: On Sun, Mar 04, 2018 at 10:29:25PM +0100, Harald van Dijk wrote: Another pre-existing dash parsing bug that I encountered now is $(( ( $(( 1 )) ) )). This should expand to 1, but gives a hard error in dash, again

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-07 Thread Martijn Dekker
Op 07-03-18 om 16:29 schreef Herbert Xu: > I also didn't quite like the idea of scanning the string backwards > to find the previous syntax. So here is my attempt at the recursive > parsing using alloca. This version introduces a parsing bug: $ src/dash -c 'x=0; x=$((${x}+1))' src/dash: 1: Synt

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-07 Thread Harald van Dijk
On 3/7/18 5:29 PM, Herbert Xu wrote: On Sun, Mar 04, 2018 at 10:29:25PM +0100, Harald van Dijk wrote: Another pre-existing dash parsing bug that I encountered now is $(( ( $(( 1 )) ) )). This should expand to 1, but gives a hard error in dash, again due to the non-recursive nature of dash's par

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-07 Thread Herbert Xu
On Sun, Mar 04, 2018 at 10:29:25PM +0100, Harald van Dijk wrote: > > Another pre-existing dash parsing bug that I encountered now is $(( ( $(( 1 > )) ) )). This should expand to 1, but gives a hard error in dash, again due > to the non-recursive nature of dash's parser. A small generalisation of wh

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-04 Thread Harald van Dijk
On 3/4/18 9:08 PM, Martijn Dekker wrote: Op 04-03-18 om 16:46 schreef Harald van Dijk: FreeBSD sh also prints a blank line here. [...] Like above, FreeBSD sh behaves like ksh. I stand corrected. Is there any port of FreeBSD sh to other operating systems? It would be much more convenient for

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-04 Thread Martijn Dekker
Op 04-03-18 om 16:46 schreef Harald van Dijk: > FreeBSD sh also prints a blank line here. [...] > Like above, FreeBSD sh behaves like ksh. I stand corrected. Is there any port of FreeBSD sh to other operating systems? It would be much more convenient for me to include it in my tests if I didn't h

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-04 Thread Harald van Dijk
On 3/4/18 7:05 PM, Denys Vlasenko wrote: On Fri, Mar 2, 2018 at 7:03 PM, Harald van Dijk wrote: On 02/03/2018 18:00, Denys Vlasenko wrote: On Wed, Feb 14, 2018 at 9:03 PM, Harald van Dijk wrote: Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <> This is what I expect, and also what b

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-04 Thread Denys Vlasenko
On Fri, Mar 2, 2018 at 7:03 PM, Harald van Dijk wrote: > On 02/03/2018 18:00, Denys Vlasenko wrote: >> >> On Wed, Feb 14, 2018 at 9:03 PM, Harald van Dijk >> wrote: >>> >>> Currently: >>> >>> $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' >>> <> >>> >>> This is what I expect, and also what bash, ksh an

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-04 Thread Harald van Dijk
On 3/4/18 4:26 PM, Martijn Dekker wrote: Op 04-03-18 om 11:44 schreef Harald van Dijk: When CTLENDVAR is seen, I double-check that syntax has the expected value. This fixes the handling of "${$+"}"}", where the inner } was seen as ending the variable substitution. Also looks like dash with you

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-04 Thread Martijn Dekker
Op 04-03-18 om 11:44 schreef Harald van Dijk: > When CTLENDVAR is seen, I double-check that syntax has the expected > value. This fixes the handling of "${$+"}"}", where the inner } was > seen as ending the variable substitution. Also looks like dash with your patch considers the "}" to be quoted:

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-04 Thread Harald van Dijk
On 3/2/18 11:58 AM, Harald van Dijk wrote: On 02/03/2018 08:49, Herbert Xu wrote: If we fix this in the parser then everything should just work. Right, that's the approach FreeBSD sh has taken that I referred to in my message from Feb 18, that I'd personally prefer as well. It basically invo

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-02 Thread Harald van Dijk
On 02/03/2018 17:33, Herbert Xu wrote: On Fri, Mar 02, 2018 at 05:05:46PM +0100, Harald van Dijk wrote: But in "${x+${y-}''}", the ' is the literal ' character. In "${x#${y-}''}", the ' is a quote character. This part is hard. If the above is done simply using another local variable, then the p

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-02 Thread Harald van Dijk
On 02/03/2018 18:00, Denys Vlasenko wrote: On Wed, Feb 14, 2018 at 9:03 PM, Harald van Dijk wrote: Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <> This is what I expect, and also what bash, ksh and posh do. With your patch: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' I was looking i

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-02 Thread Denys Vlasenko
On Wed, Feb 14, 2018 at 9:03 PM, Harald van Dijk wrote: > Currently: > > $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' > <> > > This is what I expect, and also what bash, ksh and posh do. > > With your patch: > > $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' > I was looking into this specific example and

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-02 Thread Herbert Xu
On Fri, Mar 02, 2018 at 05:05:46PM +0100, Harald van Dijk wrote: > > But in "${x+${y-}''}", the ' is the literal ' character. In "${x#${y-}''}", > the ' is a quote character. This part is hard. If the above is done simply > using another local variable, then the parse of the nested ${y-} would > cl

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-02 Thread Harald van Dijk
On 02/03/2018 16:28, Herbert Xu wrote: On Fri, Mar 02, 2018 at 11:58:41AM +0100, Harald van Dijk wrote: If we fix this in the parser then everything should just work. Right, that's the approach FreeBSD sh has taken that I referred to in my message from Feb 18, that I'd personally prefer as w

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-02 Thread Herbert Xu
On Fri, Mar 02, 2018 at 11:58:41AM +0100, Harald van Dijk wrote: > > >If we fix this in the parser then everything should just work. > > Right, that's the approach FreeBSD sh has taken that I referred to in my > message from Feb 18, that I'd personally prefer as well. It basically > involves rever

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-02 Thread Harald van Dijk
On 02/03/2018 08:49, Herbert Xu wrote: On Thu, Mar 01, 2018 at 08:24:22PM +0100, Harald van Dijk wrote: On 01/03/2018 00:04, Harald van Dijk wrote: $ bash -c 'x=yz; echo "${x#'"'y'"'}"' z $ dash -c 'x=yz; echo "${x#'"'y'"'}"' yz (That is, they are executing x=yz; echo "${x#'y'}".) POSIX says

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-01 Thread Herbert Xu
On Thu, Mar 01, 2018 at 08:24:22PM +0100, Harald van Dijk wrote: > On 01/03/2018 00:04, Harald van Dijk wrote: > >$ bash -c 'x=yz; echo "${x#'"'y'"'}"' > >z > > > >$ dash -c 'x=yz; echo "${x#'"'y'"'}"' > >yz > > > >(That is, they are executing x=yz; echo "${x#'y'}".) > > > >POSIX says that in "${va

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-01 Thread Harald van Dijk
On 01/03/2018 00:04, Harald van Dijk wrote: $ bash -c 'x=yz; echo "${x#'"'y'"'}"' z $ dash -c 'x=yz; echo "${x#'"'y'"'}"' yz (That is, they are executing x=yz; echo "${x#'y'}".) POSIX says that in "${var#pattern}" (and the same for ##, % and %%), the pattern is considered unquoted regardless

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-28 Thread Harald van Dijk
On 2/28/18 11:14 PM, Denys Vlasenko wrote: Guys, while you work on fixing this, consider taking a look at some more parsing cases in this bbox bug: https://bugs.busybox.net/show_bug.cgi?id=10821 I suppose some of them may fail in dash too (I did not test, sorry). $'...' isn't supported in das

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-28 Thread Denys Vlasenko
On Sat, Feb 24, 2018 at 5:52 PM, Herbert Xu wrote: > On Sat, Feb 24, 2018 at 10:47:07AM +0100, Harald van Dijk wrote: >> >> It seems like the new control character doesn't get escaped in one place >> where it should be, and gets lost because of that: >> >> Unpatched: >> >> $ dash -c 'x=`printf \\\

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-26 Thread Harald van Dijk
On 26/02/2018 09:40, Harald van Dijk wrote: On 26/02/2018 08:03, Harald van Dijk wrote: On 2/13/18 2:53 PM, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ There's another case where this goes wrong, that isn't fixed by your, my, or Herbert's patches: And hope

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-26 Thread Harald van Dijk
On 26/02/2018 08:03, Harald van Dijk wrote: On 2/13/18 2:53 PM, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ There's another case where this goes wrong, that isn't fixed by your, my, or Herbert's patches: $ dash -c 'a=\\a; echo "${a#\*}"' $ bash -c 'a=\\a;

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-25 Thread Harald van Dijk
On 2/13/18 2:53 PM, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ There's another case where this goes wrong, that isn't fixed by your, my, or Herbert's patches: $ dash -c 'a=\\a; echo "${a#\*}"' $ bash -c 'a=\\a; echo "${a#\*}"' \a My patch and Herbert's pr

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-24 Thread Harald van Dijk
On 2/24/18 5:52 PM, Herbert Xu wrote: On Sat, Feb 24, 2018 at 10:47:07AM +0100, Harald van Dijk wrote: It seems like the new control character doesn't get escaped in one place where it should be, and gets lost because of that: Unpatched: $ dash -c 'x=`printf \\\211X`; echo $x | cat -v' M-^IX

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-24 Thread Herbert Xu
On Sat, Feb 24, 2018 at 10:47:07AM +0100, Harald van Dijk wrote: > > It seems like the new control character doesn't get escaped in one place > where it should be, and gets lost because of that: > > Unpatched: > > $ dash -c 'x=`printf \\\211X`; echo $x | cat -v' > M-^IX > > Patched: > > $ src/d

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-24 Thread Harald van Dijk
On 2/24/18 1:33 AM, Herbert Xu wrote: On Wed, Feb 21, 2018 at 11:47:58PM +0100, Harald van Dijk wrote: On 2/21/18 2:50 PM, Denys Vlasenko wrote: I propose replacing this: Agreed, that looks better. Thanks! Good work guys. However, could you check to see if this patch works too? It passes a

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-23 Thread Herbert Xu
On Wed, Feb 21, 2018 at 11:47:58PM +0100, Harald van Dijk wrote: > On 2/21/18 2:50 PM, Denys Vlasenko wrote: > >I propose replacing this: > > Agreed, that looks better. Thanks! Good work guys. However, could you check to see if this patch works too? It passes all my tests so far. ---8<--- The c

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-21 Thread Harald van Dijk
On 2/21/18 2:50 PM, Denys Vlasenko wrote: I propose replacing this: Agreed, that looks better. Thanks! Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-21 Thread Denys Vlasenko
On Mon, Feb 19, 2018 at 11:13 PM, Harald van Dijk wrote: > On 2/18/18 11:50 PM, Harald van Dijk wrote: >> On 2/14/18 11:50 PM, Harald van Dijk wrote: >>> On 2/14/18 10:44 PM, Harald van Dijk wrote: On 2/14/18 9:03 PM, Harald van Dijk wrote: > On 13/02/2018 14:53, Denys Vlasenko wrote: >>>

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-19 Thread Harald van Dijk
On 2/18/18 11:50 PM, Harald van Dijk wrote: On 2/14/18 11:50 PM, Harald van Dijk wrote: On 2/14/18 10:44 PM, Harald van Dijk wrote: On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ [...] Currently:

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-18 Thread Harald van Dijk
On 2/14/18 11:50 PM, Harald van Dijk wrote: On 2/14/18 10:44 PM, Harald van Dijk wrote: On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ [...] Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <>

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-14 Thread Harald van Dijk
On 2/14/18 10:44 PM, Harald van Dijk wrote: On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ [...] Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <> This is what I expect, and also what bash,

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-14 Thread Harald van Dijk
On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ [...] Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <> This is what I expect, and also what bash, ksh and posh do. With your patch: $ dash -c

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-14 Thread Harald van Dijk
On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ Nice catch. The cause: uses "\\*" pattern instead of "\\\*". The fix: /* backslash */ case CBACK: c = pgetc2()

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-13 Thread Martijn Dekker
Op 13-02-18 om 14:53 schreef Denys Vlasenko: > $ >'\' > $ >'\' > $ dash -c 'echo "\*"' > \ \ > > The cause: uses "\\*" pattern instead of "\\\*". Also: $ dash -c 'case \\ab in "\*") echo BUG;; esac' BUG $ dash -c 'case \\a in "\?") echo BUG;; esac' BUG Yup. Full globbing within

dash bug: double-quoted "\" breaks glob protection for next char

2018-02-13 Thread Denys Vlasenko
$ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ The cause: uses "\\*" pattern instead of "\\\*". The fix: /* backslash */ case CBACK: c = pgetc2(); if (c == PEOF) {