Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 11:52, Herbert Xu wrote: On Wed, Mar 28, 2018 at 08:44:28AM +0200, Harald van Dijk wrote: Test case: $v='*\' set -- $v I don't see how this would cause an overrun, can you please explain it for me? Line numbers are from 0.5.9.1. When expanded backslashes are no longer

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 10:16, Herbert Xu wrote: On Wed, Mar 28, 2018 at 09:38:04AM +0200, Harald van Dijk wrote: Also, it's just as logical to restore the case pattern matching to its traditional behaviour to align it with pathname expansion. No I think that makes no sense and clearly contradicts

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Herbert Xu
On Wed, Mar 28, 2018 at 09:38:04AM +0200, Harald van Dijk wrote: > > Also, it's just as logical to restore the case pattern matching to its > traditional behaviour to align it with pathname expansion. No I think that makes no sense and clearly contradicts POSIX. Cheers, -- Email: Herbert Xu

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 09:31, Herbert Xu wrote: On Wed, Mar 28, 2018 at 08:52:57AM +0200, Harald van Dijk wrote: I seriously cannot understand the logic of pushing a break from traditional ash behaviour and from all shells apart from bash, giving POSIX as a reason for doing it, and then giving the

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Herbert Xu
On Wed, Mar 28, 2018 at 08:52:57AM +0200, Harald van Dijk wrote: > > I seriously cannot understand the logic of pushing a break from traditional > ash behaviour and from all shells apart from bash, giving POSIX as a reason > for doing it, and then giving the behaviour of all those other shells as

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 08:18, Herbert Xu wrote: On Tue, Mar 27, 2018 at 08:38:01PM +0200, Harald van Dijk wrote: My inclination is to just drop the /d\ev issue and use the bash behaviour until such a time that bash changes or POSIX changes. What would need to change in POSIX to convince you to

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 08:23, Herbert Xu wrote: On Wed, Mar 28, 2018 at 12:19:17AM +0200, Harald van Dijk wrote: This introduces a buffer overread. When expmeta() sees a backslash, it assumes it can just skip the next character, assuming the next character is not a forward slash. By treating expanded

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Herbert Xu
On Wed, Mar 28, 2018 at 12:19:17AM +0200, Harald van Dijk wrote: > > This introduces a buffer overread. When expmeta() sees a backslash, it > assumes it can just skip the next character, assuming the next character is > not a forward slash. By treating expanded backslashes as unquoted, it >

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Herbert Xu
On Tue, Mar 27, 2018 at 11:32:36PM +0200, Jilles Tjoelker wrote: > > This implies that special characters cannot be escaped using a backslash > in a context where shell quote removal is performed. Instead, special My argument would be that in the context of parameter expansion quote removal is

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Herbert Xu
On Tue, Mar 27, 2018 at 08:38:01PM +0200, Harald van Dijk wrote: > > >My inclination is to just drop the /d\ev issue and use the bash > >behaviour until such a time that bash changes or POSIX changes. > > What would need to change in POSIX to convince you to change dash to match > what you were

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 3/27/18 10:55 AM, Herbert Xu wrote: So going back to dash yes I think we should adopt the bash behaviour for pathname expansion and keep the existing case semantics. This patch does exactly that. Note that this patch does not work unless you have already applied

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 3/27/18 11:32 PM, Jilles Tjoelker wrote: I don't think it is clear at all. Note the final paragraph of 2.13.1: ] When pattern matching is used where shell quote removal is not ] performed [...] This implies that special characters cannot be escaped using a backslash in a context where shell

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Jilles Tjoelker
On Tue, Mar 27, 2018 at 08:19:19PM +0800, Herbert Xu wrote: > On Tue, Mar 27, 2018 at 01:07:21PM +0200, Harald van Dijk wrote: > > If POSIX specifies a result, and a shell applies an optimisation that causes > > a different result to be produced, doesn't that inherently disallow that > >

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 20:24, Herbert Xu wrote: On Tue, Mar 27, 2018 at 08:01:10PM +0200, Harald van Dijk wrote: Right. I guess we could change it so that CTLESC is preserved to distinguish between the backslashes from parameter expansion vs. backslashes from open input. Thinking about it some

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 19:02, Herbert Xu wrote: On Tue, Mar 27, 2018 at 06:48:45PM +0200, Harald van Dijk wrote: Backslashes coming from parameters, sure, but backslashes introduced by preglob(), I'm not so sure. Right. I guess we could change it so that CTLESC is preserved to distinguish between

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Herbert Xu
On Tue, Mar 27, 2018 at 06:48:45PM +0200, Harald van Dijk wrote: > > Backslashes coming from parameters, sure, but backslashes introduced by > preglob(), I'm not so sure. Right. I guess we could change it so that CTLESC is preserved to distinguish between the backslashes from parameter

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 18:04, Herbert Xu wrote: On Tue, Mar 27, 2018 at 05:54:53PM +0200, Harald van Dijk wrote: I was thinking about not making backslashes set metaflag in expmeta(): when the pathname component doesn't include *, ?, or [, but does include backslashes, then the if (metaflag == 0) block

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 17:14, Herbert Xu wrote: On Tue, Mar 27, 2018 at 02:57:12PM +0200, Harald van Dijk wrote: That's a good point and wouldn't have too much of an impact on performance of existing scripts. BTW, that means both expandmeta()'s metachars variable, and expmeta()'s if (metaflag == 0

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Herbert Xu
On Tue, Mar 27, 2018 at 02:57:12PM +0200, Harald van Dijk wrote: > > That's a good point and wouldn't have too much of an impact on performance > of existing scripts. BTW, that means both expandmeta()'s metachars variable, > and expmeta()'s > > if (metaflag == 0 || lstat64(expdir, ) >= 0) > >

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 14:19, Herbert Xu wrote: Nowhere does it say that backslashes that come from the result of parameter expansion is always literal. So it's clear that any shell that treats the backslash as a literal backslash is already broken. By the literal POSIX wording, yes, but the fact

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Herbert Xu
On Tue, Mar 27, 2018 at 01:07:21PM +0200, Harald van Dijk wrote: > > If POSIX specifies a result, and a shell applies an optimisation that causes > a different result to be produced, doesn't that inherently disallow that > optimisation? There may be some confusion and/or disagreement over what >

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 11:44, Herbert Xu wrote: On Tue, Mar 27, 2018 at 11:16:29AM +0200, Harald van Dijk wrote: If you say that quote removal takes place on the original token (meaning before parameter expansion), and if parameter expansion takes place before pathname expansion, then there's nothing

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Herbert Xu
On Tue, Mar 27, 2018 at 11:16:29AM +0200, Harald van Dijk wrote: > > If you say that quote removal takes place on the original token (meaning > before parameter expansion), and if parameter expansion takes place before > pathname expansion, then there's nothing left to allow \* to behave >