[Partial patch] IFS and read builtin

2010-08-22 Thread Harald van Dijk
Hi, as has been reported already dash currently has a bug where the read builtin ignores the read environment's IFS setting. As a result, echo a:b | { IFS=: read a b; echo $a; } will write out a:b. I tried to see what changed between 0.5.5.1 and 0.5.6, and found that the old code used

Re: [Partial patch] IFS and read builtin

2010-08-22 Thread Harald van Dijk
On 23/08/10 01:00, Jilles Tjoelker wrote: On Mon, Aug 23, 2010 at 12:20:12AM +0200, Harald van Dijk wrote: [...] echo a:b | { IFS=: read a b; echo $a; } [...] This has already been fixed in a totally different way in master. See git commits near 95a60b2936e8835963bfb08eadc0edf9dddf0498

Re: [Partial patch] IFS and read builtin

2010-08-23 Thread Harald van Dijk
On 23/08/10 21:35, Jilles Tjoelker wrote: I think you should do what you think is best for the stability of your product. Because dash releases are not extensively tested, I'd recommend a trial build of at least a minimal base system with the new version you choose. A particular feature to be

Re: [Partial patch] IFS and read builtin

2010-08-24 Thread Harald van Dijk
On 25/08/10 00:51, Jilles Tjoelker wrote: Yes, I think that's the proper way to implement LINENO. FreeBSD sh avoids extending the nodes by detecting expansions of LINENO at parse time and storing the line number at that time. However, this is only possible because it does not print a line

[PATCH] Improved LINENO support

2010-11-08 Thread Harald van Dijk
Hi, This patch improves LINENO support by storing line numbers in the parse tree, for commands as well as for function definitions. It makes LINENO behaves properly when calling functions, and has the added benefit of improved line numbers in error messages when the last-parsed command is

Re: [PATCH] Improved LINENO support

2010-11-11 Thread Harald van Dijk
On 10/11/10 22:59, Jilles Tjoelker wrote: What I was thinking of was adding a lineno field to narg instead of to all command types. The lineno variable would then be set by expand.c. I think that leads to a smaller patch and it should still give a sensible value for almost all errors. Downside

Re: [PATCH] Improved LINENO support

2010-11-12 Thread Harald van Dijk
On 12/11/10 19:35, Eric Blake wrote: On 11/12/2010 11:17 AM, Harald van Dijk wrote: ( : : : : : )test-$LINENO should write to file test-1, not test-7, even though the only word is on line 7. bash gets this wrong, pdksh gets this almost right. Umm

Re: [PATCH] Improved LINENO support

2010-11-12 Thread Harald van Dijk
On 12/11/10 20:08, Harald van Dijk wrote: On 12/11/10 19:35, Eric Blake wrote: On 11/12/2010 11:17 AM, Harald van Dijk wrote: ( : : : : : )test-$LINENO Umm - there was more than one word in that example (each : in the subshell is a word) That was because it was a poor example. I meant

Re: static vs. dynamic scoping

2010-11-13 Thread Harald van Dijk
On 09/11/10 23:02, Eric Blake wrote: 2. User aspect: Is anyone aware of a script that intentionally uses the full power of dynamic scoping available through 'local' which would break if scoping switched to static? FWIW, some scripts use local IFS to reset IFS to a sane value, and have the

Re: static vs. dynamic scoping

2010-11-13 Thread Harald van Dijk
On 14/11/10 00:54, ольга крыжановская wrote: Now function a uses typeset IFS=X to set the field separator to X via a local variable IFS. If function b now uses read foo1 foo2 foo3 to read a line of a database the concept of dynamic scoping *BITES*. The way I had seen local IFS used is to

Re: Quoted closing brace in variable default expansion

2010-11-20 Thread Harald van Dijk
On 19/11/10 14:53, Herbert Xu wrote: Harald van Dijkhar...@gigawatt.nl wrote: sh -c 'echo ${x:-\}}' If you need to print anything involving a backspace you should use printf and not echo. It's true that backslashes should not be passed to echo, but assuming unset or empty x, ${x:-\}}

Re: [PATCH] [PARSER] Remove backslash before } in double-quotes in variable

2010-11-23 Thread Harald van Dijk
On 21/11/10 14:41, Jilles Tjoelker wrote: The backslash prevents the closing brace from terminating the substitution, therefore it should be removed. Thanks, that works great. Example: printf %s\n ${$+\}} ${$+\}} ${$+\}} should print } three times, without backslashes. Some tests with

Re: [PATCH] Improved LINENO support

2010-11-27 Thread Harald van Dijk
On 12/11/10 22:29, Jilles Tjoelker wrote: So some more ideas: A per-command LINENO does not require adding the number to all of the node types. Only node types that are commands that perform expansions need it: NCMD, NREDIR, NBACKGND, NSUBSHELL, NFOR, NCASE. This makes sense, and I've tried

Re: [PATCH 1/2] [SHELL] Allow building without LINEO support.

2011-08-17 Thread Harald van Dijk
On Tue, 2011-08-16 at 20:12 -0500, Jonathan Nieder wrote: David Miller wrote: [Subject: [SHELL] Allow building without LINEO support.] Thanks! Debian has been using something like this (but unconditional) to convince autoconf not to use dash as CONFIG_SHELL, to work around bugs in

Re: Crash on valid input

2013-04-09 Thread Harald van Dijk
On 09/04/13 05:27, Eric Blake wrote: On 04/08/2013 09:12 PM, Dan Kegel wrote: Yes, my script was crap, I've fixed it. Here's the reproducer. Called with foo unset. I think it doesn't crash without -x. #!/bin/dash set -x test ! $foo The 'set -x' was indeed the key to reproducing the

Re: command -p does not correctly limit search to a safe PATH

2013-07-14 Thread Harald van Dijk
. If no one else comes up with a working patch or a better approach, I'll try to get that working. Cheers, Harald commit 475e328589fd2e843c138d49fb96699a2a66151d Author: Harald van Dijk har...@gigawatt.nl Date: Sun Jul 14 21:23:01 2013 +0200 command: allow combining -p with -v diff --git a/src

Re: command -p does not correctly limit search to a safe PATH

2013-07-19 Thread Harald van Dijk
On 14/07/13 21:54, Harald van Dijk wrote: On 10/07/13 20:18, Craig Loomis wrote: Dash (0.5.7 and git master) does not implement 'command -p' according to the standard, and opens an intriguing security hole to anyone trying this scheme. [...] the path that 'command -p cmd' uses is a compiled

Re: [PATCH] implement privmode support in dash

2013-08-22 Thread Harald van Dijk
On 22/08/13 19:59, Tavis Ormandy wrote: Hello, this is a patch to add privmode support to dash. privmode attempts to drop privileges by default if the effective uid does not match the uid. This can be disabled with -p, or -o nopriv. Hi Tavis, Your approach definitely has my support (FWTW),

test incorrectly rejecting valid expression with confusing ! placement

2013-08-24 Thread Harald van Dijk
Hi, Now that Herbert fixed the reported crash in test (in a far simpler manner than I had suggested, which I like), I did some more testing, and came across one case that does not currently work, and did not work in the past, but is perfectly valid: $ src/dash -c 'test ! ! = !' src/dash: 1:

Re: [PATCH] \e in echo and printf builtins

2014-06-29 Thread Harald van Dijk
On 28/06/14 19:33, Paul Gilmartin wrote: On 2014-06-28, at 10:52, Harald van Dijk wrote: No comment on whether dash itself should accept \e, but you already found a compiler that doesn't support it at all, and many of the ones that do support it also (optionally) issue a warning for it. Should

Re: [PATCH] [BUILTIN] Require leading '0' on octal escapes in echo

2014-11-01 Thread Harald van Dijk
'printf %b \1' | cat -v \1 If that isn't your aim, if your aim is only to make dash meet POSIX requirements, then don't worry, it already does so. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line unsubscribe dash in the body of a message to majord...@vger.kernel.org More

Re: dash.1 - Confusion between two pages c[h]sh

2014-11-10 Thread Harald van Dijk
that leaves csh but also adds chsh be more appropriate, or do you prefer to leave it as it is now? Cheers, Harald van Dijk Cheers, -- To unsubscribe from this list: send the line unsubscribe dash in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org

Re: dash-0.5.8 bug report

2014-11-21 Thread Harald van Dijk
, Harald van Dijk -- To unsubscribe from this list: send the line unsubscribe dash in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Possibly wrong handling of $_?

2014-12-23 Thread Harald van Dijk
a need for it, but that's just me speaking from personal experience, others may feel differently. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line unsubscribe dash in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo

Re: Possibly wrong handling of $_?

2014-12-25 Thread Harald van Dijk
On 25/12/2014 15:32, Jilles Tjoelker wrote: On Wed, Dec 24, 2014 at 12:33:32AM +0100, Vadim Zeitlin wrote: On Wed, 24 Dec 2014 00:21:09 +0100 Harald van Dijk har...@gigawatt.nl wrote: HvD On 23/12/2014 23:34, Vadim Zeitlin wrote: HvDHello, HvD HvDI'm not exactly sure if this is a bug

Re: getopts doesn't properly update OPTIND when called from function

2015-05-28 Thread Harald van Dijk
a new set of arguments. Cheers, Harald van Dijk diff --git a/src/eval.c b/src/eval.c index 071fb1b..59e7506 100644 --- a/src/eval.c +++ b/src/eval.c @@ -953,8 +953,6 @@ evalfun(struct funcnode *func, int argc, char **argv, int flags) INTON; shellparam.nparam = argc - 1

Re: getopts doesn't properly update OPTIND when called from function

2015-05-28 Thread Harald van Dijk
On 29/05/2015 04:58, Herbert Xu wrote: Harald van Dijk har...@gigawatt.nl wrote: That isn't the problem, not exactly anyway. The problem is that getopts is required to keep internal state separately from the OPTIND variable (a single integer is insufficient to track the progress when multiple

Re: getopts doesn't properly update OPTIND when called from function

2015-06-01 Thread Harald van Dijk
On 01/06/2015 08:29, Herbert Xu wrote: On Fri, May 29, 2015 at 07:50:09AM +0200, Harald van Dijk wrote: But the test script in this thread does invoke getopts with parameters that are the same in all invocations, and without modifying OPTIND. I don't see anything else in the normative sections

Re: dash: read does not ignore trailing spaces

2015-12-04 Thread Harald van Dijk
On 03/12/2015 23:26, Harald van Dijk wrote: On 03/12/2015 22:17, Stephane Chazelas wrote: 2015-12-03 22:02:14 +0100, Harald van Dijk: [] $ for shell in bash mksh posh zsh; do printf %s: "$shell"; $shell -c 'IFS=,; echo a, | { read v; echo "<$v>"; }'; done

Re: dash: read does not ignore trailing spaces

2015-12-03 Thread Harald van Dijk
On 03/12/2015 22:17, Stephane Chazelas wrote: 2015-12-03 22:02:14 +0100, Harald van Dijk: [] $ for shell in bash mksh posh zsh; do printf %s: "$shell"; $shell -c 'IFS=,; echo a, | { read v; echo "<$v>"; }'; done bash: mksh: posh:<a,> zsh:<

Re: dash: read does not ignore trailing spaces

2015-12-03 Thread Harald van Dijk
r of arguments instead of fixing it up in readcmd_handle_line(). It returns in your test, and <a,> in mine. Feedback welcome. Cheers, Harald van Dijk This error is reproducible with dash 0.5.7 and with the current master git master branch, commit 2e5842258bd5b252ffdaa630db09c9a19a9717ca. [1]

Re: [PATCH v3] builtin: Fix handling of trailing IFS white spaces

2016-06-12 Thread Harald van Dijk
one " ", meaning "x \n" gets printed instead. I think this is intended to get fixed up in the if (ifsspc) block, but that block doesn't get executed when there are no more characters after the spaces. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line &qu

Re: [PATCH v3] builtin: Fix handling of trailing IFS white spaces

2016-06-12 Thread Harald van Dijk
On 12/06/16 13:06, Herbert Xu wrote: On Sun, Jun 12, 2016 at 12:35:15PM +0200, Harald van Dijk wrote: The results are a lot better now, but I did spot a problem: src/dash -c 'X="x "; echo $X' This is supposed to print "x\n", but the IFS breakup of $X generates two

Re: [PATCH v4] builtin: Fix handling of trailing IFS white spaces

2016-06-19 Thread Harald van Dijk
. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [BUG] Illegal function names are accepted after being used as aliases

2016-02-23 Thread Harald van Dijk
ently no requirement to support --, but that requirement is likely to come in the future. local is currently non-standard and it's hard to guess whether it will require support for -- if standardised. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash&

Re: echo do not print NUL byte

2016-01-20 Thread Harald van Dijk
0.5.8. That should mean it will be in the next release of dash, but you can check out the sources from Git if you want to see it now. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majo

Re: man fixes

2016-05-18 Thread Harald van Dijk
Hi, On 18/05/16 11:07, Svyatoslav Mishyn wrote: just found some typos and mdocml warnings. iff is not a typo. It's a fairly commonly used short form for "if and only if". Perhaps that could be made clearer somehow. Cheers, Harald van Dijk -- To unsubscribe from this list: sen

Re: [bug-diffutils] bug#24116: [platform-testers] new snapshot available: diffutils-3.3.50-0353

2016-08-05 Thread Harald van Dijk
On 05/08/2016 18:21, Eric Blake wrote: On 08/05/2016 07:13 AM, Harald van Dijk wrote: Unfortunately, POSIX currently requires the export command to not have any magic quoting, and any POSIX-conforming shell will make a="b c=d" export a=$a set a to b, and c to d. Not so

Re: why does dash save, dup, and restore redirected descriptor in the parent, rather than redirect in the child?

2017-01-31 Thread Harald van Dijk
not be the case, think of built-in commands such as echo for instance. Code is needed to handle that case, and I suspect that if that code is there anyway, then having one place to set up redirections is simpler than having two places to set up redirections. Cheers, Harald van Dijk -- To unsubscribe

Re: Parameter expansion, patterns and fnmatch

2016-09-03 Thread Harald van Dijk
On 02/09/16 16:51, Herbert Xu wrote: On Fri, Sep 02, 2016 at 09:49:53AM -0500, Eric Blake wrote: On 09/02/2016 09:29 AM, Herbert Xu wrote: On Fri, Sep 02, 2016 at 09:25:15AM -0500, Eric Blake wrote: This also affects case [a in [?) echo ok ;; *) echo bad ;; esac which should print ok.

Re: Parameter expansion, patterns and fnmatch

2016-09-03 Thread Harald van Dijk
On 03/09/16 15:58, Herbert Xu wrote: On Sat, Sep 03, 2016 at 03:19:57PM +0200, Harald van Dijk wrote: But yeah, sure, if the bug has been there for over 10 years, and I'm unable to find older versions of dash to check, I would have guessed that dash indeed has never worked this way. OK

Re: dash 0.5.9: break and continue bug

2016-08-23 Thread Harald van Dijk
On 23/08/16 22:23, Zdenek Kaspar wrote: Hi, I've noticed 0.5.9 does ignore break and continue statements, here is simple reproducer: one() { echo " one" break } two() { echo " two" } for i in 1 2 do echo "loop $i:" one two done dash-0.5.9: $ dash dash-break-test

Re: dash tested against ash testsuite: 17 failures

2016-10-10 Thread Harald van Dijk
a reduction in code size. There are probably a few corner cases I'm not handling correctly in this patch, though. Feedback welcome. Cheers, Harald van Dijk --- a/src/parser.c +++ b/src/parser.c @@ -106,6 +106,7 @@ STATIC void parseheredoc(void); STATIC int peektoken(void); STATIC int read

Re: ':' noop results in ':: not found'

2016-10-28 Thread Harald van Dijk
LC_ALL to override it.) dash won't be the only program that has problems with this; if you're building software yourself, setting LANG should generally be okay, but LC_ALL is best avoided. Unless you're doing it specifically to find and report/fix bugs, anyway. Cheers, Harald van Dijk Regards, T

Re: dash tested against ash testsuite: 17 failures

2016-10-10 Thread Harald van Dijk
to single-quoted strings. In double-quoted strings, backslash-newline should be removed just as when unquoted. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo info at ht

Re: dash tested against ash testsuite: 17 failures

2016-10-12 Thread Harald van Dijk
On 10/10/16 22:20, Harald van Dijk wrote: On 08/10/16 21:42, Martijn Dekker wrote: Op 01-10-16 om 19:17 schreef Denys Vlasenko: ash-vars/var_unbackslash.tests ITYM ash-vars/var_unbackslash1.tests echo Forty two:$\ (\ (\ 42\ )\ ) dash says: Syntax error: Missing

Re: echo "\\1"?

2017-07-27 Thread Harald van Dijk
two cases, can only mean behaving differently from bash in at least one of them. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [BUG] Here-document redirection with vi/emacs on

2017-06-29 Thread Harald van Dijk
/popstackmark() pair to solve the problem, as attached. Cheers, Harald van Dijk --- a/src/input.c +++ b/src/input.c @@ -147,8 +147,12 @@ retry: static const char *rl_cp; static int el_len; - if (rl_cp == NULL) + if (rl_cp == NULL) { + struct stackmark smark; + pushstackmark(, stackblo

Re: [PATCH] [BUILTIN] describe_command: fix incorrect path

2017-05-26 Thread Harald van Dijk
-by: Harald van Dijk <har...@gigawatt.nl> --- src/exec.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/src/exec.c b/src/exec.c index ec0eadd..1350da3 100644 --- a/src/exec.c +++ b/src/exec.c @@ -743,8 +743,6 @@ describe_command(out, command, path, verbose)

Re: [BUG] Redirection bug in subshell's EXIT trap

2017-10-19 Thread Harald van Dijk
". I suspect reset() needs to be split into two separate functions, but it may be a bit tricky to determine exactly what is supposed to go where. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-14 Thread Harald van Dijk
t fail: Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <> This is what I expect, and also what bash, ksh and posh do. With your patch: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' Cheers, Harald van Dijk -- To unsubscribe from this list: send the line &q

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-14 Thread Harald van Dijk
On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ [...] Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <> This is what I expect, and also what bash, ksh

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-14 Thread Harald van Dijk
On 2/14/18 10:44 PM, Harald van Dijk wrote: On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ [...] Currently: $ dash -c 'foo=a; echo "<${foo#[a\]]}>"' <> Thi

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-21 Thread Harald van Dijk
On 2/21/18 2:50 PM, Denys Vlasenko wrote: I propose replacing this: Agreed, that looks better. Thanks! Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo inf

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-18 Thread Harald van Dijk
On 2/14/18 11:50 PM, Harald van Dijk wrote: On 2/14/18 10:44 PM, Harald van Dijk wrote: On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ [...] Currently: $ dash -c 'foo

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-24 Thread Harald van Dijk
On 2/24/18 5:52 PM, Herbert Xu wrote: On Sat, Feb 24, 2018 at 10:47:07AM +0100, Harald van Dijk wrote: It seems like the new control character doesn't get escaped in one place where it should be, and gets lost because of that: Unpatched: $ dash -c 'x=`printf \\\211X`; echo $x | cat -v' M-^IX

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-24 Thread Harald van Dijk
On 2/24/18 1:33 AM, Herbert Xu wrote: On Wed, Feb 21, 2018 at 11:47:58PM +0100, Harald van Dijk wrote: On 2/21/18 2:50 PM, Denys Vlasenko wrote: I propose replacing this: Agreed, that looks better. Thanks! Good work guys. However, could you check to see if this patch works too? It passes

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-19 Thread Harald van Dijk
On 2/18/18 11:50 PM, Harald van Dijk wrote: On 2/14/18 11:50 PM, Harald van Dijk wrote: On 2/14/18 10:44 PM, Harald van Dijk wrote: On 2/14/18 9:03 PM, Harald van Dijk wrote: On 13/02/2018 14:53, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-25 Thread Harald van Dijk
*}"' \a My patch and Herbert's preserve dash's current behaviour, your patch makes it print a. None of that is correct, the result should be the same as bash. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-26 Thread Harald van Dijk
On 26/02/2018 08:03, Harald van Dijk wrote: On 2/13/18 2:53 PM, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ There's another case where this goes wrong, that isn't fixed by your, my, or Herbert's patches: $ dash -c 'a=\\a; echo "

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-26 Thread Harald van Dijk
On 26/02/2018 09:40, Harald van Dijk wrote: On 26/02/2018 08:03, Harald van Dijk wrote: On 2/13/18 2:53 PM, Denys Vlasenko wrote: $ >'\' $ >'\' $ dash -c 'echo "\*"' \ \ There's another case where this goes wrong, that isn't fixed by your, my, or H

Re: dash tested against ash testsuite: 17 failures

2018-03-07 Thread Harald van Dijk
On 3/7/18 7:51 AM, Herbert Xu wrote: On Wed, Mar 07, 2018 at 07:49:16AM +0100, Harald van Dijk wrote: This was wrong in the original patch, but I'm not seeing it in the updated patch that you replied to. When parsing a heredoc where part of delimiter is quoted, syntax==SQSYNTAX. Since

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-07 Thread Harald van Dijk
On 3/7/18 5:29 PM, Herbert Xu wrote: On Sun, Mar 04, 2018 at 10:29:25PM +0100, Harald van Dijk wrote: Another pre-existing dash parsing bug that I encountered now is $(( ( $(( 1 )) ) )). This should expand to 1, but gives a hard error in dash, again due to the non-recursive nature of dash's

Re: dash tested against ash testsuite: 17 failures

2018-03-06 Thread Harald van Dijk
On 3/6/18 9:45 AM, Herbert Xu wrote: On Wed, Oct 12, 2016 at 07:24:26PM +0200, Harald van Dijk wrote: I would have expected another exception to be in alias expansions that end in a backslash. Shells are not entirely in agreement there, but most appear to treat this the regular way, meaning

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-07 Thread Harald van Dijk
On 3/7/18 8:04 PM, Harald van Dijk wrote: On 3/7/18 5:29 PM, Herbert Xu wrote: On Sun, Mar 04, 2018 at 10:29:25PM +0100, Harald van Dijk wrote: Another pre-existing dash parsing bug that I encountered now is $(( ( $(( 1 )) ) )). This should expand to 1, but gives a hard error in dash, again

Re: dash tested against ash testsuite: 17 failures

2018-03-07 Thread Harald van Dijk
On 3/8/18 7:30 AM, Herbert Xu wrote: Could you please resend these patches as two separate emails please? Patchwork cannot handle two patches in one email: https://patchwork.kernel.org/patch/10264661/ Ah, didn't realise that. I'll keep that in mind for future mails. Actually, I'll

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-08 Thread Harald van Dijk
On 08/03/2018 08:55, Herbert Xu wrote: On Thu, Mar 08, 2018 at 01:40:32AM +0100, Harald van Dijk wrote: If the syntax stack is to be stored on the actual stack, then real recursion could be used instead, as attached. I tried to avoid recursion for the cases that unpatched dash already handled

Re: [PATCH] parser: Fix single-quoted patterns in here-documents

2018-03-09 Thread Harald van Dijk
On 3/9/18 4:07 PM, Herbert Xu wrote: On Thu, Mar 08, 2018 at 07:35:53PM +0100, Harald van Dijk wrote: Related: x=*; cat < I don't think this is related to our patches at all. Not related to our patches, but related to the original bug. It's another instance where quoted * is wron

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-02-28 Thread Harald van Dijk
is to be taken as a $'...' string. ksh (which does support $'...' strings too) prints the literal text $'\x41', and so does bash if invoked as sh. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Expansion-lookalikes in heredoc delimiters

2018-03-15 Thread Harald van Dijk
On 15/03/2018 15:52, Herbert Xu wrote: On Thu, Mar 15, 2018 at 12:41:10PM +0100, Harald van Dijk wrote: It is if you want to do it the way POSIX specifies. You're adding a special exception in the parser. I don't see how this approach can be extended to handle the other examples in my mail

Re: Expansion-lookalikes in heredoc delimiters

2018-03-15 Thread Harald van Dijk
On 15/03/2018 18:11, Herbert Xu wrote: On Thu, Mar 15, 2018 at 05:29:27PM +0100, Harald van Dijk wrote: That's because POSIX specifies that after ${, everything up to the matching }, not including nested strings, expansions, etc., is part of the word. No exception is made when it spans

Expansion-lookalikes in heredoc delimiters

2018-03-08 Thread Harald van Dijk
r, and I think it's good to have a record not only of what worked, what has been made to work, and what got broken, but also of what's never going to be work. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to ma

Re: dash bug: double-quoted "\" breaks glob protection for next char

2018-03-09 Thread Harald van Dijk
On 3/8/18 1:40 AM, Harald van Dijk wrote: If the syntax stack is to be stored on the actual stack, then real recursion could be used instead, as attached. Even though it won't be accepted in dash, I continued with this approach for my own use. I've now got it to about 1800 bytes smaller

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
it the way POSIX specifies, that it's more useful that way. On Tue, Mar 27, 2018 at 01:07:21PM +0200, Harald van Dijk wrote: Can you show me any shell other than bash that lets this optimisation affect the results? The fact is that the other shells are not doing what they do because they're

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 11:44, Herbert Xu wrote: On Tue, Mar 27, 2018 at 11:16:29AM +0200, Harald van Dijk wrote: If you say that quote removal takes place on the original token (meaning before parameter expansion), and if parameter expansion takes place before pathname expansion, then there's nothing

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 08:18, Herbert Xu wrote: On Tue, Mar 27, 2018 at 08:38:01PM +0200, Harald van Dijk wrote: My inclination is to just drop the /d\ev issue and use the bash behaviour until such a time that bash changes or POSIX changes. What would need to change in POSIX to convince you

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 09:31, Herbert Xu wrote: On Wed, Mar 28, 2018 at 08:52:57AM +0200, Harald van Dijk wrote: I seriously cannot understand the logic of pushing a break from traditional ash behaviour and from all shells apart from bash, giving POSIX as a reason for doing it, and then giving

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 08:23, Herbert Xu wrote: On Wed, Mar 28, 2018 at 12:19:17AM +0200, Harald van Dijk wrote: This introduces a buffer overread. When expmeta() sees a backslash, it assumes it can just skip the next character, assuming the next character is not a forward slash. By treating expanded

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
character is not a forward slash. By treating expanded backslashes as unquoted, it becomes possible for the next character to be the terminating '\0'. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kerne

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 11:52, Herbert Xu wrote: On Wed, Mar 28, 2018 at 08:44:28AM +0200, Harald van Dijk wrote: Test case: $v='*\' set -- $v I don't see how this would cause an overrun, can you please explain it for me? Line numbers are from 0.5.9.1. When expanded backslashes are no longer

Re: [PATCH v2] expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 12:37, Herbert Xu wrote: On Wed, Mar 28, 2018 at 12:03:24PM +0200, Harald van Dijk wrote: When expanded backslashes are no longer treated as quoted, this would call expmeta() with the pattern *\, that is with a single unquoted trailing backslash, so: [...] Thanks

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 17:14, Herbert Xu wrote: On Tue, Mar 27, 2018 at 02:57:12PM +0200, Harald van Dijk wrote: That's a good point and wouldn't have too much of an impact on performance of existing scripts. BTW, that means both expandmeta()'s metachars variable, and expmeta()'s if (metaflag == 0

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 18:04, Herbert Xu wrote: On Tue, Mar 27, 2018 at 05:54:53PM +0200, Harald van Dijk wrote: I was thinking about not making backslashes set metaflag in expmeta(): when the pathname component doesn't include *, ?, or [, but does include backslashes, then the if (metaflag == 0) block

Re: [PATCH v2] expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 13:00, Herbert Xu wrote: On Wed, Mar 28, 2018 at 12:53:31PM +0200, Harald van Dijk wrote: [un-snip] If a pattern ends with an unescaped , it is unspecified whether the pattern does not match anything or the pattern is treated as invalid. I don't think this allows dash's

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 19:02, Herbert Xu wrote: On Tue, Mar 27, 2018 at 06:48:45PM +0200, Harald van Dijk wrote: Backslashes coming from parameters, sure, but backslashes introduced by preglob(), I'm not so sure. Right. I guess we could change it so that CTLESC is preserved to distinguish between

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
On 27/03/2018 20:24, Herbert Xu wrote: On Tue, Mar 27, 2018 at 08:01:10PM +0200, Harald van Dijk wrote: Right. I guess we could change it so that CTLESC is preserved to distinguish between the backslashes from parameter expansion vs. backslashes from open input. Thinking about it some more

Re: expand: Fix buffer overflow in expandmeta

2018-03-25 Thread Harald van Dijk
On 3/26/18 4:54 AM, Herbert Xu wrote: On Mon, Mar 26, 2018 at 03:03:38AM +0200, Harald van Dijk wrote: This was already the case before your patch, but on systems that flat out reject paths longer than PATH_MAX (that includes GNU/Linux), it seems weird that expansion can produce paths which

Re: Backslashes in unquoted parameter expansions

2018-03-26 Thread Harald van Dijk
On 26/03/2018 11:34, Martijn Dekker wrote: Op 25-03-18 om 22:56 schreef Harald van Dijk:   case /dev in $pat) echo why ;; esac Now, bash and dash say that the pattern does match -- they take the backslash as unquoted, allowing it to escape the v. Most other shells (bosh, ksh93, mksh, pdksh

Re: expand: Fix buffer overflow in expandmeta

2018-03-26 Thread Harald van Dijk
that also needs to check enddir. Indeed, thanks. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 3/29/18 6:42 AM, Herbert Xu wrote: On Wed, Mar 28, 2018 at 03:06:32PM +0200, Harald van Dijk wrote: Since bash itself is inconsistent, and POSIX unclear, What exactly is unclear about the sentence of POSIX that I quoted? Is there a legitimate interpretation of "It is unspecified wh

Re: RFC/Feature request: UTF-8 support

2018-03-24 Thread Harald van Dijk
On 3/22/18 8:12 PM, Harald van Dijk wrote: I have a local patch set of stuff that I haven't considered in an appropriate state to send to the list. One of the features I've got working is multibyte locale support. It doesn't assume UTF-8 (only assumes stateless encodings) and is tolerant

Re: expand: Fix glibc glob(3) support

2018-03-25 Thread Harald van Dijk
GLOB_MAGCHAR. I thought that was an optimisation to avoid unnecessary file system access, to ensure that simple commands such as "echo ok" don't bother looking for files named "echo" and "ok" in the current directory. It indeed doesn't work, but I took that to be a

Re: expand: Fix buffer overflow in expandmeta

2018-03-25 Thread Harald van Dijk
t; error. Should the buffer perhaps simply be limited to PATH_MAX, taking care to just give up if something longer is found? That could avoid the need to handle reallocations and result in somewhat smaller and simpler code. The downside would be that if other systems do support paths longer than PA

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-28 Thread Harald van Dijk
On 28/03/2018 10:16, Herbert Xu wrote: On Wed, Mar 28, 2018 at 09:38:04AM +0200, Harald van Dijk wrote: Also, it's just as logical to restore the case pattern matching to its traditional behaviour to align it with pathname expansion. No I think that makes no sense and clearly contradicts

Re: expand: Do not quote backslashes in unquoted parameter expansion

2018-03-27 Thread Harald van Dijk
lained away as undefined behaviour. This is what allows extensions to glob syntax, if those extensions use shell special characters. p="*(ab)"; case abab in $p) echo match ;; esac This prints "match" in ksh93. Cheers, Harald van Dijk -- To unsubscribe from this list:

Re: RFC/Feature request: UTF-8 support

2018-03-22 Thread Harald van Dijk
, in glob() sorting, in trimming (#, ##, % and %%), in field splitting, and in "$*" joining. I'll see if I can polish it a bit and make it available when I've got some more free time again, but I'm not sure how soon that will be. Cheers, Harald van Dijk -- To unsubscribe from this list: sen

Re: [PATCH] don't record empty IFS scan regions

2018-03-22 Thread Harald van Dijk
regions and something else that triggers some bug, and perhaps that combination can no longer occur with your patch. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More majordomo inf

Re: RFC/Feature request: UTF-8 support

2018-03-22 Thread Harald van Dijk
On 22/03/2018 22:01, Martijn Dekker wrote: Op 22-03-18 om 20:12 schreef Harald van Dijk: Isn't all of busybox, including ash, GPL? Wouldn't that mean that if any busybox code is imported into dash, that from then on dash can only be distributed under GPL? I thought dash was already

Re: expand: Fix ghost fields with unquoted $@/$*

2018-03-23 Thread Harald van Dijk
ld already expand to A1 B. Your patch extends that to the non-quoted case. I do not know if you want to keep this extension in. Cheers, Harald van Dijk -- To unsubscribe from this list: send the line "unsubscribe dash" in the body of a message to majord...@vger.kernel.org More maj

Re: expand: Fix ghost fields with unquoted $@/$*

2018-03-23 Thread Harald van Dijk
On 23/03/2018 10:10, Herbert Xu wrote: On Fri, Mar 23, 2018 at 09:27:37AM +0100, Harald van Dijk wrote: Also the above. But it breaks a traditional ash extension: unset IFS set -- A1 B2 C3 echo ${@%2 C3} This used to print A1 B, but prints A1 B2 C3 with your patch. echo ${@%2

  1   2   >