Re: array subscripts act differently for integers(ie. let)
also this variant does the same:
$ (('ar[$idbad2]+=11'))
Because (( and let are essentially equivalent.
I think the more important question isn't why does (( behave this way?,
but rather should (( behave this way?.
It's probably not reasonable to expect the author to know and take into
account that (( arr[$key] )) treats key's data as bash code. Really, the
behaviour of ((' arr[$key] ')) makes more sense as a default behaviour.
The former is broken (as the expectation that $key expands data is not met).
printf and the $ modifier
man 3 printf describes the following:
o An optional field, consisting of a decimal digit string followed
by a $, specifying the next argument to access. If this field is not
provided,
the argument following the last argument accessed will be used.
Arguments are numbered starting at 1. If unaccessed arguments in the format
string are interspersed with ones that are accessed the results
will be indeterminate.
This is a useful feature, allowing you to decouple the order of your
arguments with from the format. This is useful for localization and a few
other cases. In this example, I'd like to use it to simplify a regex
replace statement:
# s/(.*)foo(.*)/$2bar$1/
[[ 123foo567 =~ (.*)foo(.*) ]] printf '%2$sbar%1$s'
${BASH_REMATCH[@]:1}
Is there a particular reason why bash's built-in printf does not support
this format modifier? Does bash re-implement printf or does it use the
Standard C Library's printf? (If the former; why?)
--
*Maarten Billemont* (lhunath)
me: http://www.lhunath.com – business: http://www.lyndir.com –
http://masterpasswordapp.com
Inconsistent behaviour of +=() and existing array keys
So, referring to man bash, this is the description of +=: When += is applied to an array variable using compound assignment (see Arrays below), the variable's value is not unset (as it is when using =), and new values are appended to the array beginning at one greater than the array's maximum index (for indexed arrays) or added as additional key-value pairs in an associative array. This description does not help us understand what should happen when this syntax explicitly mentions keys and more specifically, ones that already exist within the array. Intuition would likely have us believe that since +=() mutates the array by adding the elements to it, consistency would imply that when you specify elements by key, the array would either gain an element for that key or change the existing element at that key. So if we were to run this code: declare -a indexed_array=( [0]=a ) declare -A associative_array=( [x]=a ) indexed_array+=( [0]=b [1]=c ) associative_array+=( [x]=b [y]=c ) declare -p indexed_array associative_array Here's what I would expect to see: declare -a indexed_array='([0]=b [1]=c)' declare -A associative_array='([x]=a [y]=c )' Instead, we get this: declare -a indexed_array='([0]=ab [1]=c)' declare -A associative_array='([x]=ab [y]=c )' So the current behaviour appears to change the meaning of +=() as either mutating the array or mutating the array elements, depending on whether the specified key already exists within the array. I believe this behaviour to not only be counter-intuitive but also dangerous: the same syntax now has two distinct meanings depending solely on what data already exists within the array. Even more interestingly, it is apparently also legal to use the += operator within the element assignment syntax: indexed_array+=( [0]+=b [1]+=c ) This appears to behave exactly as a normal = would there. What I would expect is for += inside +=() to behave as = does now, and = to behave as it does outside of +=(), which is to set the value, not append. Ergo: declare -a indexed_array=( [0]=a ) indexed_array=( [0]=b ) #= [0]=b -- because we unset the array and set the element so those given. indexed_array+=( [0]=b ) #= [0]=b -- because we mutate the array by setting the element to that given. indexed_array=( [0]+=b ) #= [0]=b -- because we unset the array and set the element by appending the given to nothing. indexed_array+=( [0]+=b ) #= [0]=bb -- because we mutate the array by appending the given to what is already there. -- *Maarten Billemont* (lhunath) me: http://www.lhunath.com – business: http://www.lyndir.com – http://masterpasswordapp.com
Re: Arithmetic + array allows for code injection
On Jun 2, 2014, at 9:34 AM, Greg Wooledge wool...@eeg.ccf.org wrote:
On Mon, Jun 02, 2014 at 03:08:17PM +0200, Andreas Schwab wrote:
Greg Wooledge wool...@eeg.ccf.org writes:
imadev:~$ : $((a[$x]))
bash: Mon Jun 2 08:06:39 EDT 2014: syntax error in expression (error token
is Jun 2 08:06:39 EDT 2014)
There's the code-injection problem that started the thread.
Here the index is '$(date)'.
*Note (bash) Arithmetic Expansion:: ... All tokens in the expression
undergo parameter and variable expansion, command substitution, and
quote removal. The result is treated as the arithmetic expression to be
evaluated.
Ah. And this is copied almost verbatim from POSIX:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04
$((expression))
The expression shall be treated as if it were in double-quotes, except that a
double-quote inside the expression is not treated specially. The shell shall
expand all tokens in the expression for parameter expansion, command
substitution, and quote removal.
So the reason it acts this way is because POSIX said so. Now it starts
to make sense!
(One could argue that POSIX's wording doesn't require the command
substitution be done in a second pass AFTER the parameter expansion.
But apparently it has been interpreted this way.)
As such, the bug is that the expansions triggered by $(( )) are IMPLICITLY
re-evaluated by [ ].
To summarize,
index='$( echo 2 foo )' # Literal shell code should never be evaluated unless
an 'eval' is involved.
echo ${array[ $index ]} # [] expands $index, results in a literal that [] does
not re-evaluate.
echo $(( $index )) # (( )) expands $index, results in a literal that (( )) does
not re-evaluate.
echo $(( array[ $index ] )) # (( )) expands $index, results in a literal that
[] DOES re-evaluate.
Whether or not the documentation justifies the above behaviour, I think we can
agree on the fact that no user will expect or even desire the behaviour of the
latter, never mind that it is dangerous. As such, it certainly is a bug and
should be corrected.
I think it's fair to trust, as an author, that shell code will only be
re-evaluated when there's an explicit eval, source, or similar statement
declaring such behaviour.
As a side note, expanding literal words as variables such as happens in
${array[ index ]} or the above code with index=foo is probably acceptable so
long as the value that results from this expansion is treated purely as data
(though bash does allow recursion on the expansion of literal variable names
here). The point being that in my opinion, bash should as a matter of
principle prohibit concealed code execution wherever possible; failing that it
will be nearly impossible to write safe shell code - or at least trust that
your shell code is safe.
On Jun 2, 2014, at 10:28 AM, Andreas Schwab sch...@suse.de wrote:
If you want to write robust scripts, don't use shell.
Andreas.
What does this comment justify? Other than stopping all maintenance on bash
today.
Re: Bug when trapping exit from subshell
the exit handler is called, you just muted it. remove the redirections,
and you'll also notice you need a space after your !
On 16 May 2014 12:41, Mark Ferrell ma...@homeonderanged.org wrote:
The following script properly reports the exit code to the calling
environment, , but the exit handler is not called if a function
triggers the exit vs an external command.
Script executed via:
bash script false;echo $?
bash script /bin/false;echo $?
Expected the function 'exit_handler()' to be called in both cases.
When using bash, exit_handler() was only called in the '/bin/false' case.
Note: this behaviour is only seen in bash (tested on 4.1.2 on CentOS
6.5 and 4.2.25 on Ubuntu 12.04). The following shells where also
tested and all behaved as expected (i.e. the exit handler was called):
ksh, dash, BusyBox's ash, and zsh.
Script:
set -e
exit_handler() { echo exit code: $?; }
false() { !:; }
main()
{(
trap exit_handler 0
$@ /dev/null 21
)}
main $@
echo after main
Re: Severe Bash Bug with Arrays
This mail has gone slightly off-topic. Hit mark as read if you don't care for
that sort of thing.
On 27 Apr 2012, at 21:02, Linda Walsh wrote:
Greg Wooledge wrote:
NEVER use eval plus a brace expansion to generate a list. That's just
freakin' evil. And unsafe:
But I _like_ evil sometimes! :-
I don't care if you like evil in your sexual play or in your vendettas. When
you write code, evil is nothing worth liking. In programming, evil is defined
as something that is stupid and not thought through. Doing evil has harmful
side effects. It's not cool to be evil, it's short-sighted.
But reality is things like that save me time in typing in stuff or writing
maint scripts on my machine -- they aren't meant for handling user input.
They take hand editing to gen the paths I want and don't take user input,
so it's a specialized usage in that case
It's fine to do short-sighted hacks on the prompt, where you know your input
well and will notice breakage so that you can recover.
It's not fine to do said things in scripts that run on unpredictable input and
where you don't always notice breakage until it's caused irreparable harm. I
don't really care what lame justifications you come up with to try and justify
broken system code to yourself, just don't try to convince us with them.
Maarten Billemont wrote:
On 26 Apr 2012, at 01:18, Linda Walsh wrote:
Ishtar: echo ${b[*]}
Please note that expanding array elements using [*] is usually NOT what
anyone wants.
It was exactly what I wanted for my example to work. Please don't suggest
non-working solutions to the example code I provided. ${b[@]} Will NOT
work as a replacement for [*]. It isn't not BETTER, it is DIFFERENT.
That's why there are both.
I didn't say never to use ${b[*]}. I said it's usually not what anyone
wants. It's mostly only useful for merging arrays into a single string such as
for user display of said array. I explained that. You are correct, of course,
there are two syntaxes, they both do something different (when quoted), and
that's fine. They each have their use case.
What I *did* say, is never to use ${b[*]}. And it also so happens that ${b[@]}
is not different from ${b[*]}. And I stick with that point. Never use
unquoted expansion of arrays, not with [*], not with [@].
your array like above, you then follow up by re-splitting the whole thing
using standard wordsplitting, not to mention pathname expanding the
resulting words.
I followed up? I don't understand this.
When you leave ${b[*]} unquoted, you expand each element of the array, then ask
bash to split the whole thing, and then pathname expand the result. Not only
did you just throw away any distinction between the different array elements,
you also made an entirely new distinction (whitespace), and then you're asking
bash to search the filesystems for any pathnames that match each of the words.
Seriously, nobody, ever, wants, this. If they did, they'd have used a string
instead. And even then I question the sanity behind it. Word-splitting is
useful in a primitive POSIX shell that has no concept like arrays etc. to
compensate. In bash, it is the major cause of bugs, and best avoided.
I would assume they'd have the
intelligence to know when to use * or @ and I wouldn't have to talk down to
them
and cover basics.
I can't help notice the slight irony in my having to explain to you the
necessity of quotes with array expansion.
You should always recommend the ${b[@]} variant.
I shouldn't ALWAYS do anything. It's dangerous.
You can do so for your examples... It won't work in the next
two any more than the last one of the previous...
What?
Anyway, think about this: An array holds elements in a way that each is
separated from the other. To put these elements in an array is to want each
element to be distinct from the others. Therefore, the default way of
expanding these elements should preserve that distinction.
Only in rare cases is the actual INTENT of the author to MERGE these elements
into a single unit, a single string, thus throwing away the distinction between
the elements. And in THOSE cases and those cases ALONE should the author use
${array[*]}. So yes, always recommend ${b[@]}, unless you know for a fact
that the author is looking to merge the elements.
sort an arg list:
args=-dpi -xinerama -clipboard -unixkill -nowinkill -ac +bs -fp
-multiwindow
readarray -t dflts ( a=( $args ); IFS=$'\n'; echo ${a[*]#?}|sort -k1.2 )
printf defaults=(%s) ${dflts[*]}
defaults=(-ac +bs -clipboard -dpi -fp -multiwindow -nowinkill -unixkill
-xinerama)
... What are you trying to prove? That with a very specific sent of input
values you can do some ugly and hacky mangling involving arrays and
wordsplitting to sort a word-list? Why are you even USING arrays here at all?
There's no point. Just to replace spaces with newlines?
Also, you're doing a lot
Re: Severe Bash Bug with Arrays
On 26 Apr 2012, at 01:18, Linda Walsh wrote:
Ishtar: echo ${b[*]}
Please note that expanding array elements using [*] is usually NOT what anyone
wants. Be careful about recommending it to anyone.
${b[*]} # This throws away any usefulness of the array by merging all the
array elements into one single string (argument), concatenating the elements
using the first character of IFS.
${b[*]} # Never do this. On top of throwing away the usefulness of your array
like above, you then follow up by re-splitting the whole thing using standard
wordsplitting, not to mention pathname expanding the resulting words.
${b[@]} # Expands each array element as a separate argument, protecting it
from wordsplitting and pathname expansion mutilation.
${b[@]} # Identical to and just as broken as ${b[*]}. Never do this.
You should always recommend the ${b[@]} variant. ${b[*]} is rarely useful
in the event that your intent is to merge the array into a single string, eg.
for displaying elements to a user:
dirs=(*/)
(IFS=,; echo The dirs are: ${dirs[*]})
But anyway, that's an aside from this topic, which has very little to do with
this.
smime.p7s
Description: S/MIME cryptographic signature
Re: Is it possible or RFE to expand ranges of *arrays*
On 26 Apr 2012, at 06:30, John Kearney wrote:
Am 26.04.2012 06:26, schrieb Linda Walsh:
I know I can get
a=abcdef echo ${a[2:4]} = cde
how do I do:
typeset -a a=(apple berry cherry date); then get:
echo ${a[1:2]} = berry cherry ( non-grouped args)
I tried to do it in a function and hurt myself.
echo ${a[@]:1:2}
I see little reason to ask bash to wordsplit the elements after expanding them.
You ought to quote that expansion.
smime.p7s
Description: S/MIME cryptographic signature
Re: how are aliases exported?
On 15 Apr 2012, at 22:52, Linda Walsh wrote: But I want the syntax include file.shh to just 'work', i.e. first time, it would call my include file, which defines the function... I'm sorry, tell me again why you don't just write source file.sh instead of having to invent your own include file.sh? If you really want to create custom syntax, write and enable your own built-in. As a side note, you should probably have a slash in there somewhere unless you intend for bash to search PATH for the file. smime.p7s Description: S/MIME cryptographic signature
Re: how are aliases exported?
On 16 Apr 2012, at 09:54, Maarten Billemont wrote: On 15 Apr 2012, at 22:52, Linda Walsh wrote: But I want the syntax include file.shh to just 'work', i.e. first time, it would call my include file, which defines the function... I'm sorry, tell me again why you don't just write source file.sh instead of having to invent your own include file.sh? I just realized, you appear to be looking for a way to define a source path to search for the file. PATH=/foo:/bar:/etc source file or PATH=$SOURCEPATH source file I know it's not as clean as include file, but it's certainly more clear as to what it does. And clarity is more important, IMO. Also, it conveniently also doesn't require insane hacks. smime.p7s Description: S/MIME cryptographic signature
Re: how are aliases exported?
On 16 Apr 2012, at 11:43, Linda Walsh wrote: But it won't work for files below the top level of the library directory. if I have include system/errno.shh, Source is crippled to not look in PATH. When there's a slash in the file, it stops searching PATH. So the reason you're not doing source /lib/system/errno.sh is because you want it to search multiple possible lib dirs for a system subdir? smime.p7s Description: S/MIME cryptographic signature
Re: inconsistency with readonly and scope
On 15 Apr 2012, at 03:21, Chet Ramey wrote: If I declare a variable readonly at global scope, I wouldn't expect it to be overridden by some local namespace pollution. I think that's fine; in the local context, your variable has a different meaning; it's another variable with the same name. Outside of the local scope, the original readonly variable remains unmodified.
Re: semicolon at beginning of line
On 10 Apr 2012, at 06:03, Elliott Forney wrote:
Here is another example that appears to defy my expectations. In this
case, the semicolon is allowed:
sine:~$ hello='echo hello'
sine:~$ world='echo world'
sine:~$ ${hello};${world}
hello
world
sine:~$ unset hello
sine:~$ ${hello};${world}
world
sine:~$ unset world
sine:~$ ${hello};${world}
Those are not empty statements. It's important to understand the difference
between:
$var
and:
eval $var
People should stop trying to execute code by parameter expansion, and
specifically stop thinking that parameter-expanded words are evaluated as bash
code.
Thanks!
Elliott Forney
On Mon, Apr 9, 2012 at 10:02 PM, Elliott Forney
elliott.for...@gmail.com wrote:
Sure, a comment can be used to place a line in your history but that
doesn't really address the examples I had. Just seems to me like a
lone semicolon could be treated as a newline/noop. I can't seem to
think of anything that this would break but, of course, that doesn't
mean it wouldn't. The end of a case in a switch statement is
certainly an interesting one, hadn't thought of that, but it should be
possible to handle that by checking for ;; as a token before ;.
OK, so you're saying, let's change bash so that an empty statement becomes a
noop statement. Except for when that empty statement is preceded by a
semicolon and happens to have no whitespace, because then it could be a case
delimiter.
Frankly, what are you hoping to gain from this? This will just introduce new
rules with new exceptions and inconsistencies. If it were possible to do a
blanket rule: empty statements before a semicolon are noops, I might be OK with
it, but if it requires adding additional addendums to the rule, oh wait,
except for this and that case, my vote is out.
I might mention that ksh, zsh and tcsh all allow lines to begin with a
semicolon. zsh even allows ; ; ; echo hello world ; ; ; although
ksh only allows a single ; at the beginning of a line.
On Sun, Apr 8, 2012 at 6:48 PM, Joseph Fredette jfred...@gmail.com wrote:
Could also use a #, no?
On Sun, Apr 8, 2012 at 8:46 PM, Steven W. Orr ste...@syslang.net wrote:
On 4/7/2012 4:00 PM, Elliott Forney wrote:
I wish bash would happily execute lines that begin with a semicolon,
i.e., treat it as a no-op followed by a command. The following
examples come to mind:
$ infloop echo hello
[2] 11361
hello
$ infloop; echo hello
bash: syntax error near unexpected token `;'
$ echo hello; echo world
hello
world
$ echo hello;; echo world
bash: syntax error near unexpected token `;;'
$ ; echo hello world
bash: syntax error near unexpected token `;'
Any thoughts?
Thanks,
Elliott Forney
Just use a colon.
: echo Hello world.
I use it all the time to 'park' a command in my history. Then when I'm
ready, I just back up to it and remove the colon.
--
Time flies like the wind. Fruit flies like a banana. Stranger things have
.0.
happened but none stranger than this. Does your driver's license say Organ
..0
Donor?Black holes are where God divided by zero. Listen to me! We are all-
000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net
smime.p7s
Description: S/MIME cryptographic signature
Re: status on $[arith] for eval arith vsl $((arith))??
On 09 Apr 2012, at 11:46, Linda Walsh wrote: Like it costs how much? I would **bet** that it cost more code to support (()) than to support [] as arith ops .. so if you want my opinion, lets toss (())... (like that'll happen)... Just thought I'd add that I personally prefer $((...)) mostly because of consistency with other syntax. $ is always an indicator of expansion, and since we have (( ... )) to perform an arithmetic evaluation (on its own as a statement or in a for etc.), I'm very happy that putting a $ in front of that is all it takes to expand the result of it. Just like putting a dollar in front of a subshell construct expands the output of it and putting a dollar in front of a parameter name expands the contents of it. If we were to move to $[ ... ], then to keep that consistency, we'd have to change (( ... )) to [ ... ], and it so happens that this command is already taken for a different type of test entirely. (Don't give me the spiel about how [...] is already arithmetic evaluation inside array indices, that's a different syntax entirely, and perfectly fine - unless you'd prefer to turn it into arr[ (( ... )) ], which is rather overkill.) smime.p7s Description: S/MIME cryptographic signature
Re: Exit status of if statement?
On 10 Apr 2012, at 08:26, Barry Margolin wrote: if [ $status = 0 ] As an aside, I don't think it's a good idea to recommend comparing numbers using string comparison operators. if (( status == 0 )) But if you wanted to treat them as strings for the sake of code simplicity, just go the case route: case $status in 0) ... ;; 1) ... ;; *) ... ;; esac smime.p7s Description: S/MIME cryptographic signature
Re: status on $[arith] for eval arith vsl $((arith))??
On 08 Apr 2012, at 21:30, Chet Ramey wrote: On 4/8/12 3:02 PM, Maarten Billemont wrote: Any particular reason for not removing old undocumented functionality, or is that mostly the nature of this beast - dragging along and maintaining ancient code for the sake of compatibility?= Because, as Linda discovered, there is still working code out there using it. Maybe we'll get to a point where it's all gone, but we're not there yet. IMO, the working code out there that relies on $[...] either runs on older versions of bash, or if the sysadmin decided to upgrade bash, he can assume the responsibility to fix the code. I suppose a deprecation route would make such a scenario least painful. Though I don't think bash has ever taken this approach yet (removal of features), it will be unexpected and people will likely complain. In the end one must decide between a lean code base and keeping the 1% happy to upgrade. smime.p7s Description: S/MIME cryptographic signature
Re: status on $[arith] for eval arith vsl $((arith))??
On 08 Apr 2012, at 01:47, Chet Ramey wrote: On 4/7/12 4:45 PM, Linda Walsh wrote: In modifying some released code on my distro,I ran into the extensive use of $[arith] as a means for returning arithmetic evaluations of the expression. I vaguely remember something like that from years ago, but never see any reference to it -- yet it works, and old code seems to rely on it -- and $[(1+2)/3] looks cleaner than $(((1+2)/3)). So what's up with that? It dates from Posix circa 1990 (1003.2d9, of which I've lost my paper copy). I implemented it after the Berkeley guys, mostly Marc Teitelbaum, put it into Posix. It ended up getting dropped in favor of the ksh $((...)) expansion, at which point everyone deprecated the old $[...]. I removed it from the manual sometime later, but it still works as it always has. Chet Any particular reason for not removing old undocumented functionality, or is that mostly the nature of this beast - dragging along and maintaining ancient code for the sake of compatibility? smime.p7s Description: S/MIME cryptographic signature
Re: Inconsistent quote and escape handling in substitution part of parameter expansions.
On 28 Feb 2012, at 18:52, John Kearney wrote:
On 02/28/2012 06:43 PM, Dan Douglas wrote:
On Tuesday, February 28, 2012 06:38:22 PM John Kearney wrote:
On 02/28/2012 06:31 PM, Dan Douglas wrote:
On Tuesday, February 28, 2012 05:53:32 PM Roman Rakus wrote:
On 02/28/2012 05:49 PM, Greg Wooledge wrote:
On Tue, Feb 28, 2012 at 05:36:47PM +0100, Roman Rakus
wrote:
And that means, there isn't way to substitute something
to ' (single quote) when you want to not perform word
splitting. I would consider it as a bug.
imadev:~$ q=\' imadev:~$ input=foosomethingbar imadev:~$
echo ${input//something/$q} foo'bar
I meant without temporary variable.
RR
ormaaj@ormaajbox ~ $ ( x=abc; echo ${x/b/$'\''} ) a'c
( x=abc; echo ${x/b/$'\''} ) -bash: bad substitution: no
closing `}' in ${x/b/'}
you forgot the double quotes ;)
I really did spend like an hour or 2 one day trying to figure it
out and gave up.
Hm good catch. Thought there might be a new quoting context over
there.
I think we can all agree its inconsistent, just not so sure we care??
i.e. we know workarounds that aren't so bad variables etc.
So essentially, we all agree it's buggy, but since we can work around the buggy
behavior and since POSIX doesn't *require* it not be buggy, let's leave it be
buggy?
How about making it behave properly instead? And with properly, I mostly mean,
make the quotes and backslashes behave as they do now inside, for example, a
process substitution instead of a parameter expansion.
smime.p7s
Description: S/MIME cryptographic signature
Re: Inconsistent quote and escape handling in substitution part of parameter expansions.
On 28 Feb 2012, at 23:23, Chet Ramey wrote:
On 2/28/12 5:18 PM, John Kearney wrote:
On 02/28/2012 11:07 PM, Chet Ramey wrote:
On 2/28/12 4:28 PM, John Kearney wrote:
On 02/28/2012 10:05 PM, Chet Ramey wrote:
On 2/28/12 12:26 PM, John Kearney wrote:
But that isn't how it behaves. ${test//str/}
because str is replaced with '' as such it is treating
the double quotes as string literals.
however at the same time these literal double quotes
escape/quote a single quote between them. As such they are
treated both as literals and as quotes as such
inconsistently.
I don't have a lot of time today, but I'm going to try and
answer bits and pieces of this discussion.
Yes, bash opens a new `quoting context' (for lack of a better
term) inside ${}. Posix used to require it, though after
lively discussion it turned into well, we said that but it's
clearly not what we meant.
There are a couple of places in the currently-published version
of the standard, minus any corregendia, that specify this. The
description of ${parameter} reads, in part,
The matching closing brace shall be determined by counting
brace levels, skipping over enclosed quoted strings, and
command substitutions.
The section on double quotes reads, in part:
Within the string of characters from an enclosed ${ to the
matching '}', an even number of unescaped double-quotes or
single-quotes, if any, shall occur.
Chet
yhea but I think the point is that the current behavior is
useless. there is no case where I want a to be printed and
start a double quoted string? and thats the current behavior.
Not so important how you treat it just need to pick 1. then you
can at least work with it. Now you have to use a temp variable.
as a side note ksh93 is pretty good, intuitive ksh93 -c
'test=teststrtest ; echo ${test//str/dd dd}' testdd ddtest
ksh93 -c '( test=teststrtest ; echo ${test//str/dd '\''dd} )'
testdd 'ddtest
The real question is whether or not you do quote removal on the
stuff inside the braces when they're enclosed in double quotes.
Double quotes usually inhibit quote removal.
The Posix solution to this is to require quote removal if a
quote character (backslash, single quote, double quote) is used to
escape or quote another character. Somewhere I have the reference
to the Austin group discussion on this.
1${A:-B}2
Logically for consistancy having double quotes at position 1 and 2
should have no effect on how you treat string B.
Maybe, but that's not how things work in practice. Should the following
expansions output the same thing? What should they output?
bar=abc
echo ${foo:-'$bar'}
echo ${foo:-'$bar'}
Chet
Personally, I'd prefer it to behave in the same way bash deals with this sort
of thing in other contexts.
Eg.
echo $(echo '$bar')
echo $(echo '$bar')
Both obviously result in $bar getting output.
For the sake of consistency with the rest of bash, I would expect the same from
your example.
smime.p7s
Description: S/MIME cryptographic signature
Re: Another mapfile question.
On 05 Aug 2011, at 16:32, Steven W. Orr wrote: On 8/5/2011 9:08 AM, Maarten Billemont wrote: IFS= aa=() while read line do aa+=($line) done fn You should really put that IFS= on your 'read', so as not to break the default wordsplitting for the rest of your script: For purposes of giving concise examples on this email list, I wrote it as above. In practice, something like the above construct that modifies IFS would be responsible for restoring IFS. I'm very much against changing the standard operation of the shell in a broader scope than where you need its effect. old_IFS=$IFS IFS=$'\n' while read line do stuff done (fn) IFS=$old_IFS The point here is to change IFS for 'read'. Unfortunately, you're also changing it for all of 'stuff'. Your scope is too broad, and as a result, stuff's behavior is non-standard. Making IFS local-scoped is really the same thing, still too broad. You wouldn't notice until you did something with IFS in 'stuff', and actually did notice the bug in that. It's dangerous. Do this: while IFS= read line do stuff done (fn) It's much cleaner and it's more correct. Same goes for set -f hackery. I'm always against it, because there's no way to cleanly scope it, it changes the way the shell works substantially, and there's usually constructs that express the logic more correctly. smime.p7s Description: S/MIME cryptographic signature
Re: Another mapfile question.
IFS= aa=() while read line do aa+=($line) done fn You should really put that IFS= on your 'read', so as not to break the default wordsplitting for the rest of your script: while IFS= read -r line vs. IFS=$'\n' aa=($( fn)) Don't leave expansions unquoted! you're still permitting pathname expansion here. IFS=$'\n' read -rd '' -a aa (fn) vs. mapfile -t aa fn When compared to the above read, this will also read empty lines. The read operation will merge multiple newlines into a single delimiter (as it does for all types of whitespace in IFS). The while read works fine but is verbose. The mapfile is a bit more concise. The read -a is fine and concise so long as you can live with the lack of empty lines. I doubt mapfile is much use to you that while read doesn't already give you. smime.p7s Description: S/MIME cryptographic signature
Process substitution file consumed?
In the following snippet, I expect the two ls' to give the same output.
However, it appears the process substitution's temporary file sometimes
vanishes for the second. Can someone explain why and whether this is the
desired behavior or not?
A Linux (2.6.27.6) VM in a QEMU says this:
# moo() { ls -l $1; ls -l $1; }; moo (true)
l-wx-- 1 root root 64 Aug 2 14:00 /dev/fd/63 - pipe:[245]
ls: cannot access /dev/fd/63: No such file or directory
A Linux (2.6.32-5-amd64) machine says this:
# moo() { ls -l $1; ls -l $1; }; moo (true)
l-wx-- 1 devel devel 1 Aug 2 16:03 /dev/fd/63 - pipe:[527968]
l-wx-- 1 devel devel 1 Aug 2 16:03 /dev/fd/63 - pipe:[527968]
My Mac says this:
# moo() { ls -l $1; ls -l $1; }; moo (true)
prw-rw 0 lhunath staff 0 2 Aug 15:55 /dev/fd/63|
prw-rw 0 lhunath staff 0 2 Aug 15:55 /dev/fd/63|
And the latter two break again when I toss a (true) inbetween the two ls':
# moo() { ls -al $1; (true); ls -al $1; }; moo (true)
l-wx-- 1 devel devel 1 Aug 2 16:06 /dev/fd/63 - pipe:[528022]
ls: cannot access /dev/fd/63: No such file or directory
# moo() { ls -al $1; (true); ls -al $1; }; moo (true)
prw-rw 0 lhunath staff 0 2 Aug 16:07 /dev/fd/63|
ls: /dev/fd/63: Bad file descriptor
Surely neither case where the FD can no longer be accessed or its file has
disappeared can be correct behavior?
smime.p7s
Description: S/MIME cryptographic signature
Re: Negative indexes in ${arr[@]:off:length}
On 30/06/11 14:15, Greg Wooledge wrote:
On Wed, Jun 29, 2011 at 06:41:25PM +0200, Maarten Billemont wrote:
For those to whom a cursor between elements seems confusing: think
of your prompt's cursor and imagine an element is a single character.
No wonder this makes no sense to me. Have you actually looked at the
cursor in a terminal? I use rxvt, but xterm is the same: the cursor is
a black[1] rectangle that *overlies* one of the characters, not a skinny
little wedge/line that sits *between* characters.
The only time I've ever seen a cursor that is *between* characters is
in GUI stuff, like a textarea in Firefox. But for people like me,
that's the oddball case, not the norm. I certainly wouldn't expect a
programming language to work like this.
[1] Or white, if you are one of those people who use a dark background.
Or possibly amber or green, if you're *really* old school.
My cursor is also a block, but perhaps you use vim. Write abcabc in a
vim document, then select the second a. You move your cursor so that
the block is on the a or the line is in front of it. You go into visual
mode and go forward by one. ${arr[@]:0:1}.
Now instead of going forward, go backward one (${arr[@]:0:-1}) and
you'll have selected the c.
It really doesn't matter whether your cursor is a block or a line. To
select elements from a set, you need a starting point and a direction,
and IMO, pretending that ${arr[@]:0:1} should be the same thing as
${arr[@]:0:-1} is by far the more confusing way to think about arrays.
Re: Negative indexes in ${arr[@]:off:length}
On 27 Jun 2011, at 16:25, Chet Ramey wrote:
I don't even understand what the second one is supposed to mean at
all -- the :1: means to start with b and the -2 means to go back 2
elements...? How do you derive a c from any possible interpretation
of this?
I assume that he wants to be able to treat an indexed array as a circular
buffer without having to do any of the work in a script.
Exactly, let's draw the array in the example:
arr=(a b c)
values: [ a | b | c ]
indexes: 0 1 2 3
If you expand ${arr[@]:1:2}, you get the following:
values: [ a | b | c ]
indexes: 0 1 2 3
expand: [ 1 2 ] = b c
^ start from 1
^ length 2
I propose we let a negative length iterate backward, so with ${arr[@]:1:-2},
you get the following:
values: [ a | b | c ]
indexes: 0 1 2 3
expand:1 ] [ 2 = a c
^ start from 1
^ length 2
Since ${arr[-1]} already does exactly this, I figure it would merely add to the
syntax' consistency.
Re: Negative indexes in ${arr[@]:off:length}
Since ${arr[-1]} already does exactly this, I figure it would merely add to
the syntax' consistency.
Consider the fact that ${arr[x]} is in fact the equivalent of ${arr[@]:x:1}.
Re: Negative indexes in ${arr[@]:off:length}
On 29 Jun 2011, at 14:05, Mart Frauenlob wrote: On 29.06.2011 13:42, Maarten Billemont wrote: On 27 Jun 2011, at 16:25, Chet Ramey wrote: [...] Exactly, let's draw the array in the example: arr=(a b c) values: [ a | b | c ] indexes: 0 1 2 3 [...] 4 members? 3 members. The index cursor precedes its member, as is commonly done when depicting arrays. Apologies for mentioning the index 3 and somehow confusing you. Note that I prefixed the line with the word indexes, not values. I think we all know value a has the index 0 and value c has the index 2.
Re: Negative indexes in ${arr[@]:off:length}
On 29 Jun 2011, at 15:01, Greg Wooledge wrote:
On Wed, Jun 29, 2011 at 01:42:06PM +0200, Maarten Billemont wrote:
If you expand ${arr[@]:1:2}, you get the following:
values: [ a | b | c ]
indexes: 0 1 2 3
expand: [ 1 2 ] = b c
^ start from 1
^ length 2
I propose we let a negative length iterate backward, so with ${arr[@]:1:-2},
you get the following:
values: [ a | b | c ]
indexes: 0 1 2 3
expand:1 ] [ 2 = a c
^ start from 1
^ length 2
Firstly, I see my indenting of the ^ [label] got whacked off target. Let me
try to correct that, if my MUA allows:
${arr[@]:1:2}
values: [ a | b | c ]
indexes: 0 1 2 3
expand: [ 1 2 ] = b c
^ start from 1
^ length 2
${arr[@]:1:-2}
values: [ a | b | c ]
indexes: 0 1 2 3
expand:1 ] [ 2 = a c
^ start from 1
^ length 2
But if you're starting with element 1, why wouldn't it be b a?
This is where preceding the element with the index comes in handy: We start
with the index 1, which lays between element 0 and element 1. We go back 1,
that gives us element 0 and we're now between element 0 and the last element of
the array (assuming a circular view on the array). Go back one more and you
now also get the last element in the array, and the cursor ends up between the
last and the before last element.
For those to whom a cursor between elements seems confusing: think of your
prompt's cursor and imagine an element is a single character. Type this:
abcabc
Put your cursor before the second a, imagine you're now at index 0:
abc|abc
Index 0 refers to element a, because when you select 1 character
(${arr[@]:0:1}), that'll be a:
abc[a]bc.
Select 2 elements (${arr[@]:0:2}):
abc[ab]c.
Now try going backwards with your cursor to select 2 elements, instead of
forwards (${arr[@]:0:-2}):
a[bc]abc
My example was ${arr[@]:1:-2}, so that would look like this:
ab[ca]bc
Re: Feature Request - Allow mapfile to handle NUL-delimited data
On 13 May 2011, at 20:07, DJ Mills wrote: A -0 option, or even an option to use a different field separator, would be an excellent feature for mapfile. Currently, of course, you can use: while IFS= read -rd '' file; do array+=($file); done mapfile is considerably faster, however, and I think this would be quite handy. seconded. I'm not sure why it only works for lines. Sure, on the shell data is often line-based, but more often than that, line-based parsing is bad and should be done with a safe delimiter rather than \n instead.
Re: BASH_SUBSHELL documentation misleading
On 23 Mar 2011, at 21:15, Sam Liddicott wrote: On 23/03/11 18:52, Chris F.A. Johnson wrote: On Wed, 23 Mar 2011, Sam Liddicott wrote: Configuration Information [Automatically generated, do not change]: Machine: i686 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i686' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='i686-pc-linux-gnu' -DCONF_VENDOR='pc' -DLOCALEDIR='/usr/share/locale' -DP$ uname output: Linux sojo 2.6.35-28-generic-pae #49-Ubuntu SMP Tue Mar 1 14:58:06 UTC 2011 i686 GNU/Linux Machine Type: i686-pc-linux-gnu Bash Version: 4.1 Patch Level: 5 Release Status: release Description: man page says: BASH_SUBSHELL Incremented by one each time a subshell or subshell environment is spawned. The initial value is 0. This suggests that: echo $BASH_SUBSHELL ; ( echo ) ; echo $BASH_SUBSHELL would not give the same answer for BASH_SUBSHELL No, it suggests that: echo $BASH_SUBSHELL ; ( echo $BASH_SUBSHELL ) would not give the same answer for BASH_SUBSHELL It should suggest that, because that is how it actually works. But the man page is misleading and does not actually suggest this. In your example, the second echo $BASH_SUBSHELL is at the same depth as the first. Yes. But a new subshell environment has been spawned. Each time that happens BASH_SUBSHELL should increase. Of course I know how it does work, but the man page isn't clear. It doesn't say that the increase is only visible within the subshell and therefore it is a measurement of subshell depth. Sam I must concur. In the bug example at the second expansion of BASH_SUBSHELL occurs after a subshell or subshell environment is spawned. The documentation does not specify that BASH_SUBSHELL is actually unmodified in the running shell and merely copied to the subshell and incremented there. What's more, the fact that this is not mentioned, grammatically suggests that BASH_SUBSHELL is modified, which is not actually the case.
Re: BASH_SUBSHELL documentation misleading
On 23 Mar 2011, at 21:28, Chet Ramey wrote: OK. What wording would you like to see? I don't mind the wording he proposed: On 23 Mar 2011, at 17:12, Sam Liddicott wrote: maybe it should say BASH_SUBSHELL Incremented by one in each nested subshell or subshell evironment. It is always 0 when $BASH_PID=$$
Re: variable name and its' value are the same characters causes recursion error
On 10 Mar 2011, at 15:23, Chet Ramey wrote: On 3/10/11 8:14 AM, Greg Wooledge wrote: On Thu, Mar 10, 2011 at 10:18:26AM +0800, Clark J. Wang wrote: Actually I don't like the recursion here. Does POSIX require that? For example: unset a; declare a=a; [[ a -lt 3 ]]; echo $? bash: [[: a: expression recursion level exceeded (error token is a) 1 POSIX doesn't even have a [[ command. This is all bash. Not really. There is substantial agreement among shells that implement arithmetic expansion. bash, ksh93, zsh, mksh (and other pdksh derivatives that implement `[[') all behave the same way. For the most part, it's the same way with `['; zsh is a notable exception there. Chet Personally, I would much rather see (( a )) fail if a doesn't contain a number rather than go search for a parameter named by its contents. If the parameter a contains a word that's not a number, I can't imagine any case where this would be an expected and wanted scenario, rather than a bug. If it were expected, the author would've used the indirection operator directly. Which leaves us with a bug, one that's often terribly hard to detect and diagnose. That's ignoring the fact that you're leaving the door wide open for user input to go and load any parameter it chooses in its stead. Really, whenever this happens, it's either breaking things in completely unexpected and often invisible ways or it's somebody exploiting your code to do something it wasn't supposed to or reveal something it doesn't want to show. Why would we want this feature?
Re: Why escape char `:' with `\' when auto completing filenames?
On 17 Feb 2011, at 23:02, Chet Ramey wrote:
Clark J. Wang dearv...@gmail.com writes:
I think char `:' is not special in bash.
$ printf %q\n $COMP_WORDBREAKS
$' \t\n\'=;|(:'
I don't think that explain the issue.
/* characters that need to be quoted when appearing in filenames. */
rl_filename_quote_characters = \t\n\\\'@=;|()#$`?*[!:{~;
/*}*/
So that's problem. I don't think @=: need to be treated specially. Any
reason?
They are used as word break characters during completion.
That's the way it's implemented but that does not mean that's reasonable.
For pete's sake. If you don't think they should be word break characters,
modify the value of COMP_WORDBREAKS. For the record, @ causes a word break
so you can complete hostnames more easily, = breaks so you can complete
filenames on the rhs of assignment statements, and : breaks so you can
complete filenames in words that look like $PATH.
Are we really spending this much time on a cosmetic issue?
Why are we escaping all word break characters? rm file:name and rm file\:name
are effectively identical, I'm not sure I see the need for escaping it.
Re: Can someone explain this?
On 12 Feb 2011, at 09:28, Andreas Schwab wrote: Bob Proulx b...@proulx.com writes: Dennis Williamson wrote: Yes, do your quoting like this: ssh localhost 'bash -c cd /tmp; pwd' I am a big fan of piping the script to the remote shell. $ echo cd /tmp pwd | ssh example.com bash /tmp Even better: $ ssh example.com bash \EOF cd /tmp pwd EOF That avoids having to watch out for ' vs quoting. Andreas. The trouble with using stdin is that it becomes much harder to pass user data. If it's simple strings, one might be tempted to expand them instead: ssh example.com bash EOF cd $remoteDir pwd EOF But that would be a really bad idea, since you're injecting data into bash code (if you don't see it yet, imagine a user forces remoteDir to contain mypath; rm -rf ~; : ). The only sane way I can think of to solve this problem in a generically applicable fashion, is to use a construct such as: ssh example.com bash $(printf 'cd %q; pwd' $remoteDir)
Re: leading 0 inconsistence?
On 12 Feb 2011, at 11:57, Ralf Goertz wrote: Hi, I am wondering what the reasoning might be for this seeming inconsistence. i=08 if [ $i -lt 9 ] ; then echo ok; fi ok if [ $((i)) -lt 9 ] ; then echo ok; fi bash: 08: value too great for base (error token is 08) Why is 08 not tried to be interpreted as octal when *numerically* compared using test? I stand by my general recommendation of always using (( when performing arithmetic tests, [[ when testing strings and files, and never [, unless your shebang is not actually bash. It will save you from a great many pitfalls and inconsistencies.
Re: Why sh does not return a false value?
On 05 Feb 2011, at 13:47, ali hagigat wrote: if (sh -c exit 34) then echo p;fi p The following condition should be false, because our exit value is non-zero. but 'if' considers the condition as true and executes 'echo' command. Why? You are giving -c the argument 'exit' and setting 34 as the zero'th argument to the script. It's vital to understand what word splitting is. Quotes are used to keep words together so they are passed as a single argument. -c takes only one single argument, so to pass the command exit 34 to -c, you quote it.
Re: How can i specify the scripting language used for parsing a script?
On 05 Feb 2011, at 17:09, Andreas Schwab wrote: Maarten Billemont lhun...@gmail.com writes: The comment is called a hashbang or shebang. It tells the kernel which program to start. Your script is passed over stdin to the interpreter. No, it isn't, it's passed as the argument. Andreas. Of course, otherwise ./myscript file wouldn't work.
Re: printf %q and $'...'
You can output $'\x00' just fine: $ printf '\0' Note that -d $'\x00' is the same thing as -d '', for the reason I mentioned earlier. The argument to the -d option that read takes is a C-string. Understand the difference between *strings* and *streams*. A stream (standard output of printf '\0' or find . -print0) can contain any byte. A C-string can not contain NUL bytes. If you want strings with NUL bytes, you need Pascal-strings. You can *not*, however, output a NUL byte by using $'\x00' as an argument. Because arguments can't contain NUL bytes (they are C-strings). So outputting NUL a byte with this will *fail*: $ echo $'\x00' This will *also* fail: $ printf $'\x00' The first example I gave doesn't fail because the argument is not a NUL byte (empty), it is a backslash followed by a zero. printf sees this argument and understands you want it to expand that into a NUL byte, then emits it on its output STREAM. On 25 Nov 2009, at 14:35, Antonio Macchi wrote: it sounds strange, beacuse $ find . -print0 | while read -d $'\x00'; do touch $REPLY; done works fine. but if I try to output $'\x00', I can't.
Re: printf %q and $'...'
On 25 Nov 2009, at 14:58, Antonio Macchi wrote: my goal is very very stupid, like this $ printf %q $(/bin/ls) to get a ascii form of a binary file (something like uuencode) but, as you can see, it does not work only for two binary chars 0x00, and 0x0a That doesn't sound like a goal; more like an approach to achieve some other goal. 1. Remember to quote your expansions: $(..) 2. You can't put binaries in arguments for the very reason you just encountered: Arguments cannot hold NUL bytes. If you state the true goal (why are you trying to convert binary files into some sort of undefined form?) perhaps a more appropriate suggestion can be made. printf doesn't read from a stream, so you can't use it to %q-encode your binary data directly. Though %q-encoding binary data sounds like an odd requirement. You did mention uuencode so I assume you considered that already as well as base64 encoding and found it insufficient. Which leaves me wondering what you could possibly be trying to do.
Re: printf %q and $'...'
As for NUL out outputting anything in your result, the cause is C-strings.
Arguments are C-strings and those are delimited by NUL bytes. Therefore, the
NUL byte that you're putting in it is actually marking the end of the string.
So the argument ends BEFORE your NUL byte. So it's empty.
As or \x0a, that's a newline. And command substitution trims trailing
newlines. So a string [newline] gets trimmed to .
On 25 Nov 2009, at 08:19, Antonio Macchi wrote:
Hi, I'm using older bash 3.2.39, so please forgiveme if in your newer bash
this issue does not arise.
0x00 and 0x0a has no output in printf %q
$ for i in {0..9} a; do printf %q\n `echo -en \\x0$i\`; done
''
$'\001'
$'\002'
$'\003'
$'\004'
$'\005'
$'\006'
$'\a'
$'\b'
$'\t'
''
$'\x00' outputs nothing
hd (echo $'\x00')
0a|.|
0001
$'\x01' outputs twice.
$ hd (echo -n $'\x01')
01 01 |..|
0002
thanks for your great job
have a nice day
(God Save Linux!)
bye
