Re: RFE: brace expansion sequences should do zero padding
Hi again!
I saw my first post made it to the list eventually as well. Sorry for
the duplicate. I hadn't realized that the newsgroup was moderated.
I'm a bit dishearted at the lack of response. On IRC many people pointed
out that usually this kind of issue can be solved by passing a sequence
to printf. Now I've come up with a real life example that isn't easy to
hack together using printf:
wget -x http://some.really.long/url/prefix/{,{000..123}.{html,jpg}}
There have been some concerns about changing the behaviour and thus
breaking existing scripts. The preferred solution in this case would be
to use ... instead of .. if one wanted to activate this feature.
Personally I believe that zero padded sequences in an existing
application that cares for the exact string and not only the numeric
value are so unlikely that adding another piece of syntax is not worth
the trouble, but I'd like your opinion on this.
Another thing worth mentioning is negative numbers. My padding pads all
numbers to a common width, not a common number of digits. This is what
printf does, and it's a wee little bit easier to implement. However it
could be changed to common number of digits as well. On IRC I got the
idea that {-07..003} should do common width, whereas {-007..003} should
do common number of digits. This, however, would add a lot of code. I
think negative numbers are so rare that they are not worth the effort.
Do you agree?
I hope to generate some feedback here. If you think this useful, tell me
about it, and I'll try a bit harder to get this into the offical
sources. If I get no single answer this time as well, I'll probably post
the patch somewhere online, patch my own version of bash, and that's it.
Greetings,
Martin von Gagern
signature.asc
Description: OpenPGP digital signature
extglobs in case constructs
Configuration Information [Automatically generated, do not change]:
Machine: i686
OS: linux-gnu
Compiler: i686-pc-linux-gnu-gcc
Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu'
-DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash'
-DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib
-DDEFAULT_PATH_VALUE='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
-DSTANDARD_UTILS_PATH='/bin:/usr/bin:/sbin:/usr/sbin'
-DSYS_BASHRC='/etc/bash/bashrc'
-DSYS_BASH_LOGOUT='/etc/bash/bash_logout' -DNON_INTERACTIVE_LOGIN_SHELLS
-DSSH_SOURCE_BASHRC -march=prescott -O2 -ggdb
uname output: Linux server 2.6.30-gentoo-r5 #1 SMP PREEMPT Tue Aug 18
11:26:16 CEST 2009 i686 Intel(R) Pentium(R) 4 CPU 3.00GHz GenuineIntel
GNU/Linux
Machine Type: i686-pc-linux-gnu
Bash Version: 4.0
Patch Level: 33
Release Status: release
Description:
I've found out that extglobs behave differently in different
constructs. To avoid syntax errors, the always have to be
enabled at parse time.
For comparison constructs like ``[[ $v == a@(a|b|c)c ]]'' this
is enough. For case constructs like ``case $v in a@(a|b|c)c)''
the extglob shopt has to be set at runtime as well. This is kind
of inconsistent.
I've read several messages by Chet on the bug-bash mailing list,
all claiming that the extglob shopt should change the parser
only. In this case one would expect that setting it at the time
a function gets defined should be enough, and that it shouldn't
be necessary when the function gets executed. Case constructs
inside a function render this assumption invalid.
Repeat-By:
-- Sample script ---
#!/bin/bash
shopt -s extglob
echo -n definition:
shopt extglob
f() {
v=abc
shopt $1 extglob
echo -n runtime:
shopt extglob
echo -n comparison uses extglob
if [[ $v == a@(a|b|c)c ]]; then
echo yes
else
echo no
fi
echo -n case uses extglob
case $v in
a@(a|b|c)c)
echo yes
;;
*)
echo no
;;
esac
}
f -s
f -u
-- Resulting output
definition: extglob on
runtime: extglob on
comparison uses extglob yes
case uses extglob yes
runtime: extglob off
comparison uses extglob yes
case uses extglob no
Re: extglobs in case constructs
Greg Wooledge wrote: You're using extglobs inside a function, and extglob was enabled at the time the function was delcared. The parser parsed them at that time, decided that they are extended globs, and therefore that's what they are. Then why don't they work as extended globs if the extglob shopt is set to off at runtime, from within the function, after it has been parsed? Most people have the opposite problem I know, I understand the reason for this, and I even approve of it, because after all, the use of extended globs is something the person defining the function should worry about, not the person using the function. Sadly, currently both have to worry. signature.asc Description: OpenPGP digital signature
Re: extglobs in case constructs
Martin von Gagern wrote:
I've found out that extglobs behave differently in different
constructs. To avoid syntax errors, the always have to be
enabled at parse time.
For comparison constructs like ``[[ $v == a@(a|b|c)c ]]'' this
is enough. For case constructs like ``case $v in a@(a|b|c)c)''
the extglob shopt has to be set at runtime as well. This is kind
of inconsistent.
OK, I guess this is due to the following difference:
execute_cmd.c:3225-3231 in execute_cond_node:
int oe;
oe = extended_glob;
extended_glob = 1;
result = binary_test (cond-op-word, arg1, arg2,
TEST_PATMATCH|TEST_ARITHEXP)
? EXECUTION_SUCCESS
: EXECUTION_FAILURE;
extended_glob = oe;
execute_cmd.c:2931 in execute_case_command:
match = strmatch (pattern, word, FNMATCH_EXTFLAG|FNMATCH_IGNCASE) !=
FNM_NOMATCH;
What's interesting is what you don't see here: the case command doesn't
temporarily set extended_glob to 1 as the cond node does. I guess this
is where the different behaviour at runtime comes from.
Is this difference intentionally, or is this a bug? If it is a bug, the
attached patch should fix it, although it might introduce other issues.
At least for my example script, the patch does its job.
I'm a bit surprised that execute_cond_node seems to unconditionally
enable the extended glob mode. I don't see any check whether the thing
has been parsed as an extended glob in the first place.
Is this because things that look like extended globs will always be
invalid syntax if extglob is off? So that if anything got parsed
successfully and still looks like an extended glob, the code can be sure
it was parsed as an extended glob? In that case, is there any point in
not always enabling extended globs, except if you are testing the
portability of your script? If so, I'll second the request to always
enable them, but I have some doubts that's the case.
As an alternative to my patch, one could maybe move all this temporary
extglob stuff to the function definitions. I.e. a function defined with
extglob on will also have extglob on when executed, and vice versa. This
might break some compatibility, though, especially for subshells. And
simply saving and restoring functins by listing their definition would
become a trouble as well. So it's just a thought so far.
Greetings,
Martin
Allow use of extended globs in case constructs at runtime,
even if the extglob shopt is not set.
2009-10-01 Martin von Gagern
References:
http://thread.gmane.org/gmane.comp.shells.bash.bugs/13518
Index: bash-4.0/execute_cmd.c
===
--- bash-4.0.orig/execute_cmd.c
+++ bash-4.0/execute_cmd.c
@@ -2915,6 +2915,7 @@ execute_case_command (case_command)
QUIT;
for (list = clauses-patterns; list; list = list-next)
{
+ int oe;
es = expand_word_leave_quoted (list-word, 0);
if (es es-word es-word-word *(es-word-word))
@@ -2928,8 +2929,11 @@ execute_case_command (case_command)
/* Since the pattern does not undergo quote removal (as per
Posix.2, section 3.9.4.3), the strmatch () call must be able
to recognize backslashes as escape characters. */
+ oe = extended_glob;
+ extended_glob = 1;
match = strmatch (pattern, word, FNMATCH_EXTFLAG|FNMATCH_IGNCASE) !=
FNM_NOMATCH;
free (pattern);
+ extended_glob = oe;
dispose_words (es);
Re: extglobs in case constructs
Chet Ramey wrote: In general, if you enable extglob before a function is parsed and leave it on when the function is executed, you should be ok. I'm thinking about this in terms of bash completion functions. Many of them use extended globs, but they should not force a specific permanent extglob setting on the user. And cleanly restoring the setting can a bit tricky. I believe I have a solution for this, though: https://alioth.debian.org/tracker/index.php?func=detailaid=311628group_id=100114atid=413095 Still, that solution brings a bit of overhead, and if that could have been avoided, I'd been even happier. The reason that [[ conditional commands don't require extglob to be enabled while they're executed (and case commands do) lies in how the commands are defined to behave. case commands are defined by Posix: the pattern is always a Posix shell pattern, and any extension to that must be enabled manually. The [[ command, which bash picked up from ksh, is defined so that the rhs of `==' is an extended pattern (ksh always uses extended patterns, without requiring a special option). For compatibility, bash forces the rhs of the [[ command's == and != operators to be matched as an extended pattern when executed. That's a good explanation, and a valid reason. If you say it's going to stay that way, I'll accept that and stop suggesting alternatives. Still, if there were a way to have the case command remember whether it was defined with or without extglob, then that state could be recreated when the command is executed. You'd still have to enable a manual switch if you wanted to diverge from the Posix standard, only now it would only have to be when you define the function. The benefit would be that the semantics of the function would stay unaffected from later modifications of this shopt. As an alternative, would you consider things like local shopts? I'm thinking about a flag like -l to shopt which would set the option for the duration of the current function, but neither export it to functions called therein nor leave it in place when the function returns. I believe this would be quite a major task, but might be worthwhile for functions that need to fine-tune these options but don't want to affect their surroundings. If you are interested, I could try writing a patch. In fact, to be perfectly compatible, bash should temporarily enable extglob when parsing the rhs of `==' and `!=' inside a conditional command. I will look at doing that for the next version. I don't know if I speak for a majority of users, but in my daily use I prefer bash to be usable intuitively, and consider compatibility to other shells (which I rarely if ever use) second to that. Things change if I want to write portable scripts, but in this case, I'd take plain sh as the common denominator, and would also invoke bash with that name. So yes, compatibility with other shells is nice, and always having extended globs as the rhs of an == would increase the ease of use. Different behaviour for different constructs on the other hand is quite a bit counterintuitive, and for me personally compatibility with other shells isn't important enough to warrant this quirk. Greetings, Martin signature.asc Description: OpenPGP digital signature
Mailcheck during completion
Hi! I often get a bash message about new mail during command line completion. Steps to reproduce: Type ls ~/.bas and press tab. Expected result: Command line completed to ls ~/.bash or whatever is right. Actual result: If new mails have arrived, the mail check is performed when pressing tab, rendering the whole command line like this: $ ls ~/.basYou have mail in /var/mail/user h This is a real problem for both readability and subsequent editing of the command line. I would therefore like to see bash modified in such a way that the mail check is never executed in response to a completion. Instead, the only places where such a check should be performed are in my opinion the initial prompt after startup and any prompt after a command was actually executed. Just pressing enter with an empty command line should also do the check, I think. Greetings, Martin von Gagern signature.asc Description: OpenPGP digital signature
Re: Mailcheck during completion
Hi Chet, thanks for the swift reply! On 11.08.2011 15:54, Chet Ramey wrote: I suspect that you have a completion defined for `ls' and it's running a command or process substitution that's causing the mail check. Can you run `set -x', then attempt the completion again and post the results? Yes, there is a completion function for ls coming from the _longopt function from the bash-completion project at debian: http://anonscm.debian.org/gitweb/?p=bash-completion/bash-completion.git;a=blob;f=bash_completion;h=66019379fe1e9ef9b1010bff53a0dc1baf4e73d9;hb=7c81ef895455d0f7543c65789ff62808e7465578#l1540 With -x enabled, I see several subprocess lines, i.e. lines indented by two + instead of one. Some triggered by eval, others by $(...). I wonder why this breaks things. After all, those subshells should never print a prompt, should never even look for interactive input. But I take it you've got your reasons to suspect them. Can this be fixed? Greetings, Martin signature.asc Description: OpenPGP digital signature
Re: Mailcheck during completion
On 12.08.2011 01:05, Chet Ramey wrote: It's eval. Try the attached patch. Works like a charm. Thanks a lot! Martin signature.asc Description: OpenPGP digital signature
Re: bash tab variable expansion question?
On -10.01.-28163 20:59, Chet Ramey wrote: Bash-4.2 tries to leave what the user typed alone, but that leads to an inherently ambiguous situation: when do you quote the `$' in a filename (or, in this case, a directory name)? It could be a shell variable, and it could be a character in the filename. I'm not sure I'd EVER want completion to escape a $. If I want a variable, I'll type $, either by itself or inside double quotation marks. If I want a $ included in a file name, then I'll type \$ or enclose it in single quotation marks. The above I'd do even for partial file names. So if bash sees an unquoted $, then I'd call it a variable, and no ambiguity involved. That is the problem, in a nutshell. I posted a partial patch at the end of March that applied a heuristic to avoid quoting variable expansions in most cases, but there was no way to get the bash-4.1 behavior back. Can you explain these heuristics? I am soliciting feedback on this iteration. Please try the new option (`direxpand') and let me know if it's missing anything. The option works, as far as I can see. ls $HOM completes to ls $HOME instead of ls $HOME/, though. The heuristics apparently don't work well with programmatic completion. With _longopt http://tinyurl.com/3va3hyz from the debian bashcomp project set up for ls, I get \$HOME/.bash as the completion for $HOME/.bash. And I get no completion at all for ls $HOM, probably because there is no dir of that name. Can we do the following for programmatic completion? 1. If the insertion point is at a variable reference, and the referenced variable does not exist, then complete variable name without delegating to programmatic completion 2. Otherwise, store the original String for each word 3. Perform variable expansion for each word 4. Perform completion on the expanded words 5. In the completed words, search for the expanded but uncomleted strings from 3. and replace them with the unexpanded forms from 2. I'm not sure what behaviour I'd expect if the variable contains several words. Don't care too much for that case, so I'll be happy with any solution. Greetings, Martin von Gagern signature.asc Description: OpenPGP digital signature
bash 5.2 bad substitution for ${x:+${x}[}
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2
uname output: Linux … 5.18.16-…-amd64 #1 SMP PREEMPT_DYNAMIC Debian
5.18.16-… (2022-…) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu
Bash Version: 5.2
Patch Level: !PATCHLEVEL!
Release Status: release
Description:
Some substitutions that used to work pre 5.2 are failing starting in
5.2. I did a git bisect on that, it's the initial 5.2 commit which
broke this,
https://git.savannah.gnu.org/cgit/bash.git/commit/?h=bash-5.2=74091dd4e8086db518b30df7f222691524469998.
According to my reproducing example below, this appears to be due to a
combination of conditional substitution, a variable enclosed in curly
braces and an unbalanced opening square bracket that I think should
only be considered as text without any semantic relevance in this
position.
Repeat-By:
bash -c 'x=""; echo "a${x:+${x}[}b"'
Work-around:
Omit the inner curly braces, i.e. use "a${x:+$x[}b".
