Re: Bash-5.1-beta available
On 10/8/20 12:37 PM, Andreas Schwab wrote: > On Okt 08 2020, Chet Ramey wrote: > >> There's no good way to tell whether or not a terminal supports bracketed >> paste, but I suppose this is as good a start as any. Remember that gdb >> can always turn it off while running the test suite. > > It needs to be controllable from the outside. It is controllable from the outside. > the testsuite framework sets INPUTRC=/dev/null on purpose (to avoid any > disturbance). This is not a good general solution. What gdb is doing is relying on the default readline settings to be appropriate, if not for every situation, at least for running its tests. There's no way it can guarantee that such an unrepresentative environment will handle the defaults, as we've seen here. It would be easy enough to use the standard INPUTRC=file mechanism to control (and disable) what it needs. Now, in the end, it's not going to matter here. Turning off bracketed paste when the terminal name is "dumb" will probably work for gdb's purposes. The thing is, it's probably going to bite someone else down the road. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
On Okt 08 2020, Chet Ramey wrote: > There's no good way to tell whether or not a terminal supports bracketed > paste, but I suppose this is as good a start as any. Remember that gdb > can always turn it off while running the test suite. It needs to be controllable from the outside. gdb itself doesn't provide any command line interface to the readline configuration, and the testsuite framework sets INPUTRC=/dev/null on purpose (to avoid any disturbance). Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: Bash-5.1-beta available
On 10/7/20 12:42 PM, Andreas Schwab wrote: > On Sep 10 2020, Chet Ramey wrote: > >> h. Bracketed paste mode is enabled by default (for now). > > Shouldn't that be disabled on a dumb terminal? This wrecks havoc on the > gdb testsuite. It sets TERM=dumb to tell readline not to do any fancy > output, but this no longer works. There's no good way to tell whether or not a terminal supports bracketed paste, but I suppose this is as good a start as any. Remember that gdb can always turn it off while running the test suite. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
On Okt 07 2020, Chet Ramey wrote: > On 10/7/20 12:42 PM, Andreas Schwab wrote: >> On Sep 10 2020, Chet Ramey wrote: >> >>> h. Bracketed paste mode is enabled by default (for now). >> >> Shouldn't that be disabled on a dumb terminal? This wrecks havoc on the >> gdb testsuite. It sets TERM=dumb to tell readline not to do any fancy >> output, but this no longer works. > > The gdb test suite tests pasting? It uses expect. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: Bash-5.1-beta available
On 10/7/20 12:42 PM, Andreas Schwab wrote:
> On Sep 10 2020, Chet Ramey wrote:
>
>> h. Bracketed paste mode is enabled by default (for now).
>
> Shouldn't that be disabled on a dumb terminal? This wrecks havoc on the
> gdb testsuite. It sets TERM=dumb to tell readline not to do any fancy
> output, but this no longer works.
Does gdb have access to set readline variables from its test suite? (Other
than specifying an INPUTRC file, of course.)
If so, it can run
rl_variable_bind ("enable-bracketed-paste", "off");
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
On 10/7/20 12:42 PM, Andreas Schwab wrote: > On Sep 10 2020, Chet Ramey wrote: > >> h. Bracketed paste mode is enabled by default (for now). > > Shouldn't that be disabled on a dumb terminal? This wrecks havoc on the > gdb testsuite. It sets TERM=dumb to tell readline not to do any fancy > output, but this no longer works. The gdb test suite tests pasting? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
On Sep 10 2020, Chet Ramey wrote: > h. Bracketed paste mode is enabled by default (for now). Shouldn't that be disabled on a dumb terminal? This wrecks havoc on the gdb testsuite. It sets TERM=dumb to tell readline not to do any fancy output, but this no longer works. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: Bash-5.1-beta available
Hi,
On Mon, Sep 21, 2020 at 09:13:13PM -0400, Dale R. Worley wrote:
> Andreas Schwab writes:
> I assume that if you really want the old effect, you can still do
>
> exec {dup}<&1
You mean:
exec {dup}<&0
>
> ... <( ... <$dup ) ...
and:
... <( ... <&$dup ) ...
>
> exec {dup}<&-
>
> Dale
Sample:
This worked under previous bash versions (I use 5.0.3(1)-release):
testForkInput () {
local line
while read line ;do
echo "$line"
done < <(
sed 's/^/> /'
)
}
But with 5.1.0(1)-beta, I have to replace this with:
testForkInput () {
local dup line
exec {dup}<&0
while read line ;do
echo "$line"
done < <(
sed 's/^/> /' <&$dup
)
exec {dup}>&-
}
testForkInput < <(seq 1 5)
> 1
> 2
> 3
> 4
> 5
On Tue, Sep 22, 2020 at 08:57:57AM +0200, Andreas Schwab wrote:
> That point is that it silently breaks existing scripts.
I agree.
--
Félix Hauri -- http://www.f-hauri.ch
Re: Bash-5.1-beta available
Andreas Schwab writes: > On Sep 21 2020, Dale R. Worley wrote: > >> Andreas Schwab writes: >>> On Sep 10 2020, Chet Ramey wrote: >>> yy. Process substitution processes now get their input from /dev/null, since they are asynchronous, not interactive, and not jobs. >>> >>> That breaks scripts that want to filter stdin with a process >>> substitution, eg: >>> >>> while read ...; do ...; done < <(filter) >> >> I'm assuming you mean "<( ... )" process substitutions, > > Which is exactly what I wrote. By "you" in the above, I meant the author of item "yy" above. Sorry about the inexactness. Dale
Re: Bash-5.1-beta available
On Sep 21 2020, Dale R. Worley wrote: > Andreas Schwab writes: >> On Sep 10 2020, Chet Ramey wrote: >> >>> yy. Process substitution processes now get their input from /dev/null, since >>> they are asynchronous, not interactive, and not jobs. >> >> That breaks scripts that want to filter stdin with a process >> substitution, eg: >> >> while read ...; do ...; done < <(filter) > > I'm assuming you mean "<( ... )" process substitutions, Which is exactly what I wrote. > I assume that if you really want the old effect, you can still do That point is that it silently breaks existing scripts. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: Bash-5.1-beta available
Andreas Schwab writes:
> On Sep 10 2020, Chet Ramey wrote:
>
>> yy. Process substitution processes now get their input from /dev/null, since
>> they are asynchronous, not interactive, and not jobs.
>
> That breaks scripts that want to filter stdin with a process
> substitution, eg:
>
> while read ...; do ...; done < <(filter)
I'm assuming you mean "<( ... )" process substitutions, as ">( ... )"
process substitutions are defined to get their stdin from the pipe.
The interesting thing to me is that this change irritates me in its
irregularity. But I do use process substitutions occasionally, and I
went through my script that use "<( ... )". There aren't many, and none
of the contained processes read stdin. So the change would have no
practical consequences for me.
I assume that if you really want the old effect, you can still do
exec {dup}<&1
... <( ... <$dup ) ...
exec {dup}<&-
Dale
Re: Bash-5.1-beta available
On 9/17/20 5:13 PM, Robert Elz wrote: > | I don't have list-specific email configs. > > Are there any lists for which you want to direct replies to yourself > rather than the list? That doesn't have much to do with my email configs, which are not specific to mailing lists, since those constitute a small portion of my email. And aside from me, do you encounter almost anyone > whose MUA actually implements Reply-To properly, and replies only to you? I don't pay a lot of attention to it. > | Because that's where the incentives are. Nobody cares if you implement > | "what is right" if you fail a standards conformance test. > > I guess that depends upon your objectives. I don't care in the > slightest about conformance tests, or their results - which is why > I won't implement "cd -L" and why NetBSD refuses to supply exec'able > forms of "cd" "umask" ... Sure, but others do. You're fortunate not to have to. > | Then the standard needs to be clarified, doesn't it? > > Yes, we were kind of at that point earlier ... and we know the result > can only be "unspecified" which isn't really very helpful. You never know. Geoff might just say "well, it's obviously this." > It would be nicer if before that happens we could just agree on what > is the better result, and do that, and try to get mksh and ksh93 to > do the same. Then perhaps the standard would not need to say "unspecified". We already have two camps, and netbsd/freebsd/historical sh are the ones that print \"A\". Bash (current beta version), yash, dash, ksh93, and mksh print "A". Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
Date:Thu, 17 Sep 2020 14:40:04 -0400
From:Chet Ramey
Message-ID:
| That's why I find yash so interesting to test against. It's written by
| someone with almost no contact with the standards community, yet attempts
| to implement the letter of POSIX.
Yes. I've had a few discussions with Yuki about various issues,
at least one of which led to a posix bug report.
| I don't have list-specific email configs.
Are there any lists for which you want to direct replies to yourself
rather than the list? And aside from me, do you encounter almost anyone
whose MUA actually implements Reply-To properly, and replies only to you?
(Except when manually overridden, which I usually remember to do.)
| Because that's where the incentives are. Nobody cares if you implement
| "what is right" if you fail a standards conformance test.
I guess that depends upon your objectives. I don't care in the
slightest about conformance tests, or their results - which is why
I won't implement "cd -L" and why NetBSD refuses to supply exec'able
forms of "cd" "umask" ...
When there are two options, neither of which looks better than the other,
and the standard has picked one, I'll do the same. When one is clearly
a better result (which involves both what happens and why, and also how
whatever it is is actually being used in the wild) then I'll do that one,
regardless of what the standard says should happen.
| The shell implementation still has to do something, even if the user is
| supposed to ensure it,
Of course.
| and that something really should try to reflect what
| the user intends (determining that is always a tricky business).
When possible, yes. But in something like "${foo:-abc"def}" it is
kind of hard to guess any intent. An error would make sense, but so
would expanding to abc"def (if foo is unset or null) just as if that
string had been single quoted, if that was possible there - just for the
purpose of the " of course, any $ expansion in it would still be processed.
| Then the standard needs to be clarified, doesn't it?
Yes, we were kind of at that point earlier ... and we know the result
can only be "unspecified" which isn't really very helpful.
It would be nicer if before that happens we could just agree on what
is the better result, and do that, and try to get mksh and ksh93 to
do the same. Then perhaps the standard would not need to say "unspecified".
kre
Re: Bash-5.1-beta available
On 9/16/20 6:45 PM, Robert Elz wrote:
> Date:Wed, 16 Sep 2020 11:35:41 -0400
> From:Chet Ramey
> Message-ID: <210592e5-f42c-32ee-7c85-9418d3e29...@case.edu>
>
> | That's what gives the impression that the standards committees are a
> | private club.
>
> This one isn't confined to the standards industry, it exists everywhere.
> At a particular time, in a particular community, there tends to be a
> common understanding of all kinds of things. Different time, different
> community, and that understanding just isn't there any more.
That's why I find yash so interesting to test against. It's written by
someone with almost no contact with the standards community, yet attempts
to implement the letter of POSIX.
> | I read it all. The hills are littered with bodies.
>
> Would be nice if you could delete the Reply-To your messages all
> carry, since it is (every time I have seen anyway) the same as the
> From it is useless if intended as a From replacement, and I kind of
> doubt that you always want to request that no-one reply to the list.
I don't have list-specific email configs.
>
> | The problem is the standard has changed over the years, and now we all
> | have compatibility issues dealing with past attempts to implement what
> | ended up being a moving target.
>
> Yes, that is a problem. Partly caused by trying to implement the
> standard, instead of implementing what is right, and then making sure
> the standard says what is implemented.
Because that's where the incentives are. Nobody cares if you implement
"what is right" if you fail a standards conformance test.
> | Of course it can. It can check and throw an error if desired.
>
> That isn't ensuring that there are an even number of quotes, it is
> objecting when there aren't. The two are related, but are not the
> same thing.
The shell implementation still has to do something, even if the user is
supposed to ensure it, and that something really should try to reflect what
the user intends (determining that is always a tricky business).
> But to go back to the original issue, in a here-doc, the " is always just
> a character, it is never special, and hence a \" combination in a here doc
> (except in those cases where a new quoting context is established - which
> it isn't in a ${foo+word} expansion - never has been) should always
> generate \"
Then the standard needs to be clarified, doesn't it? Current experience
shows the wording to be ambiguous, since the world is divided into two
camps here.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
Date:Wed, 16 Sep 2020 11:35:41 -0400
From:Chet Ramey
Message-ID: <210592e5-f42c-32ee-7c85-9418d3e29...@case.edu>
| That's what gives the impression that the standards committees are a
| private club.
This one isn't confined to the standards industry, it exists everywhere.
At a particular time, in a particular community, there tends to be a
common understanding of all kinds of things. Different time, different
community, and that understanding just isn't there any more.
| Well, that and the "it was a bad idea in 1988, so it's
| a good idea now!"
That one is a different problem.
| I read it all. The hills are littered with bodies.
Would be nice if you could delete the Reply-To your messages all
carry, since it is (every time I have seen anyway) the same as the
>From it is useless if intended as a From replacement, and I kind of
doubt that you always want to request that no-one reply to the list.
| The problem is the standard has changed over the years, and now we all
| have compatibility issues dealing with past attempts to implement what
| ended up being a moving target.
Yes, that is a problem. Partly caused by trying to implement the
standard, instead of implementing what is right, and then making sure
the standard says what is implemented.
| That's the issue. Even if it's an application requirement, the shell
| implementation still has to do some enforcement,
No, it doesn't. When applications violate the requirements upon them
what the implementation does is always unspecified. Detecting and
reporting the error is nice, when it can be done (particularly when it
needs to be detected, and there's nothing else better to do) but other
things are possible, including dreaming up some new meaning for it.
| Of course it can. It can check and throw an error if desired.
That isn't ensuring that there are an even number of quotes, it is
objecting when there aren't. The two are related, but are not the
same thing.
| > Actually I disagree (and not because you used backticks as opening quotes
| > there).
|
| It's a typographical convention of many years.
Yes, I assumed, that was irrelevant.
| It might be unspecified in the current draft of the upcoming version of the
| standard, or at some time in the future, but even the currently published
| version of the standard talks about embedded quoted strings without
| qualification.
Yes, I agree, but it has to be an embedded quoted string, not just a
string that happens to contain quote characters.
Eg:
foo'bar'bletch
contains an embedded quoted string.
"foo'bar'bletch"
does not (it is a quoted string, it does not contain an embedded one).
The single quotes are simply characters.
| The current standard says
|
| "an even number of unescaped double-quotes or single-quotes, if any, shall
| occur. A preceding character shall be used to escape a literal
| '{' or '}'"
|
| so it's there.
OK, though I'd say half there, that text is much improved in the forthcoming
version.
| Given this script:
|
| foo=bar
| echo "${foo+"x}z"}"
|
| every modern shell I tested produces
|
| x}z
|
| where the historical Bourne shell would give `xz}'.
All except bosh in the tests I ran (bosh seems to go into a cpu loop).
I'm just not sure this is correct according to the current standard.
Which is probably why it doesn't really matter, as this is clearly going
to be unspecified in the future.
But to go back to the original issue, in a here-doc, the " is always just
a character, it is never special, and hence a \" combination in a here doc
(except in those cases where a new quoting context is established - which
it isn't in a ${foo+word} expansion - never has been) should always
generate \"
kre
Re: Bash-5.1-beta available
On 9/15/20 2:09 PM, Robert Elz wrote:
> Date:Tue, 15 Sep 2020 11:47:58 -0400
> From:Chet Ramey
> Message-ID:
>
>
> | This wouldn't be the first time the standard relied on readers
> | drawing an inference that's not explicit.
>
> No, it isn't. It is really hard to avoid that however, as quite often
> when the text is written, everyone simply assumes that the meaning is
> obvious.
That's what gives the impression that the standards committees are a
private club. Well, that and the "it was a bad idea in 1988, so it's
a good idea now!"
> Same with the Reply-To discussion happening on the austin list
> (where it doesn't really belong, and wouldn't have happened if they
> hadn't butchered the list) where the ancient text in RFC822 has been
> read to justify behaviour that makes no sense, and was never intended.
> (And I really hope that you personally are paying attention!)
I read it all. The hills are littered with bodies.
>
> | when no-one is relying
> | > upon stuff like
> | > "${var:-"unquoted-text"}"
> | > any more.
> |
> | I don't think you can rely on this right now.
>
> No you can't. The standard says (or is going to say) it is unspecified,
> and some implementations have started doing bizarre things. It certainly
> never documented the weird 7th edn sh (and everything else in the early days)
> behaviour.
The problem is the standard has changed over the years, and now we all
have compatibility issues dealing with past attempts to implement what
ended up being a moving target.
>
> My point was that even though this usage has become obsolete, we
> can't fix the quoting anyway, because of "${var:-text that is quoted}"
>
> | The last time we talked about the "even number" sentence, we all just
> | hand-waved it away by characterizing it as an "application requirement,"
>
> Yes, but there's nothing else it can be, even with:
>
> | even though there's no mention of "the application shall ensure" there.
>
> as if there's a requirement, then either the implementation or the
> application (or both) needs to be bound by it.
That's the issue. Even if it's an application requirement, the shell
implementation still has to do some enforcement, and that means checking
that embedded quotes are properly paired.
Here the implementation
> can't ensure there are an even number of quotes, it isn't writing the
> text, so this has to be directed at the application.
Of course it can. It can check and throw an error if desired.
>
> The implementation can error out if it wants if it detects the error
> (I don't think anyone does) but the error is that of the application.
> It must be.
But that doesn't matter. The burden is still on the implementation to check
it and react accordingly. Previous versions of the standard required it.
> | The xyz is semi-quoted, because no matter what else the inner quotes
might
> | do, they are supposed to prevent recognizing the closing brace. If it
were
> | not `xyz' but `x}z', the inner quotes are supposed to `protect' the brace
> | there.
>
> Actually I disagree (and not because you used backticks as opening quotes
> there).
It's a typographical convention of many years.
>
> There's no issue in something like
>
> ${var-"}"}
>
> in that one there is an enclosed quoted string (whether double or
> single quoted - backslash quoted works too). but in
>
> "${var-"}"}"
>
> there is no enclosed quoted string, the (lexically) second " char either
> closes the quoted string opened by the first, or has unspecified behaviour,
> but it isn't creating a new quoted string.
It might be unspecified in the current draft of the upcoming version of the
standard, or at some time in the future, but even the currently published
version of the standard talks about embedded quoted strings without
qualification. It's unclear whether or not the standard really means the
quotes within the braces constitute a separate quoted string, but there is
broad consensus that the inner pair of double quotes prevent the brace from
ending the parameter expansion.
> (I think it glosses over the \
> escaping rules in "" strings though, the '}' needs to be added to the list
> of chars that \ escapes, and I don't think that is ever explicitly done, it
> is simply implied that it ought to work that way).
The current standard says
"an even number of unescaped double-quotes or single-quotes, if any, shall
occur. A preceding character shall be used to escape a literal
'{' or '}'"
so it's there.
> | The current version of the standard makes that clear, shells have
> | implemented it that way,
>
> Some shells. Based upon a mis-reading I believe. It isn't the
> way you're reading it.
Given this script:
foo=bar
echo "${foo+"x}z"}"
every modern shell I tested produces
x}z
where the historical Bourne shell would give `xz}'.
> There are scripts that rely on all kinds of bizarre things, sometimes
> we simply have to break
Re: Bash-5.1-beta available
Date:Tue, 15 Sep 2020 11:47:58 -0400
From:Chet Ramey
Message-ID:
| This wouldn't be the first time the standard relied on readers
| drawing an inference that's not explicit.
No, it isn't. It is really hard to avoid that however, as quite often
when the text is written, everyone simply assumes that the meaning is
obvious. Same with the Reply-To discussion happening on the austin list
(where it doesn't really belong, and wouldn't have happened if they
hadn't butchered the list) where the ancient text in RFC822 has been
read to justify behaviour that makes no sense, and was never intended.
(And I really hope that you personally are paying attention!)
| when no-one is relying
| > upon stuff like
| > "${var:-"unquoted-text"}"
| > any more.
|
| I don't think you can rely on this right now.
No you can't. The standard says (or is going to say) it is unspecified,
and some implementations have started doing bizarre things. It certainly
never documented the weird 7th edn sh (and everything else in the early days)
behaviour.
My point was that even though this usage has become obsolete, we
can't fix the quoting anyway, because of "${var:-text that is quoted}"
| The last time we talked about the "even number" sentence, we all just
| hand-waved it away by characterizing it as an "application requirement,"
Yes, but there's nothing else it can be, even with:
| even though there's no mention of "the application shall ensure" there.
as if there's a requirement, then either the implementation or the
application (or both) needs to be bound by it. Here the implementation
can't ensure there are an even number of quotes, it isn't writing the
text, so this has to be directed at the application.
The implementation can error out if it wants if it detects the error
(I don't think anyone does) but the error is that of the application.
It must be.
| It's all just a mess,
It is that, and really all prompted by pdp-11 address space limitations!
| and the standard is avoiding saying anything definitive.
It has little choice, it isn't a legislature. But at least when things
are made unspecified it gives us implementors (or maintainers more in my
case) an opportunity to agree what interpretation makes most sense, and
all agree to implement that. When we can actually make that happen we
might eventually end up in a situation where a later standard rev might
be able to specify things again.
| The xyz is semi-quoted, because no matter what else the inner quotes might
| do, they are supposed to prevent recognizing the closing brace. If it were
| not `xyz' but `x}z', the inner quotes are supposed to `protect' the brace
| there.
Actually I disagree (and not because you used backticks as opening quotes
there).
There's no issue in something like
${var-"}"}
in that one there is an enclosed quoted string (whether double or
single quoted - backslash quoted works too). but in
"${var-"}"}"
there is no enclosed quoted string, the (lexically) second " char either
closes the quoted string opened by the first, or has unspecified behaviour,
but it isn't creating a new quoted string. Nor would it be using '', which
aren't special at all inside a "" string. The only safe way to quote that
(intended to be literal) '}' is using \ quoting, as that does work in a
double quoted string, and somewhere the standard, or some interpretation,
does say that, I remember reading it. (I think it glosses over the \
escaping rules in "" strings though, the '}' needs to be added to the list
of chars that \ escapes, and I don't think that is ever explicitly done, it
is simply implied that it ought to work that way).
Aside: for readers following along, who can't really believe some of this,
note that the original Bourne shell would happily allow
"${var-*"*}
where (when var is unset) the result is a list of all files starting with
a (literal) '*'. Not something one can rely upon any more (fortunately).
That is, despite this being useful functionally, the closest we can
currently do is
${var-\**}
(using any form of quoting you like for the first '*'). But this suffers
from leaving the expansion of var (when var is set) unquoted, whereas
the old form did not. There is currently no safe simple way to have a quoted
var expansion with an unquoted alternative to use if the var is unset.
| The current version of the standard makes that clear, shells have
| implemented it that way,
Some shells. Based upon a mis-reading I believe. It isn't the
way you're reading it.
| and I'm sure there are scripts somewhere that rely on it.
There are scripts that rely on all kinds of bizarre things, sometimes
we simply have to break them. Otherwise we can't even fix bugs, because
someone might be relying upon the buggy behaviour.
kre
Re: Bash-5.1-beta available
On 9/14/20 2:56 PM, Robert Elz wrote:
> | "However, the double-quote character ( '"' ) [edited, since the HTML on
> the
> | web site is malformed]
>
> So it is, what a mess, the pdf formatted version is fine, so that is just
> a conversion error. Wonder if it is fixed yet? If not you should file
> an Online Pubs bug about it. I know some similar ones were fixed, but
> I don't see any immediate mention of that one (ref bugs 955 & 1060 for
> similar issues).
>
> | shall not be treated specially within a here-
> | document, except when the double-quote appears within "$()", "``", or
> | "${}"."
>
> The first $() is clear, that starts a new quoting context, just thinking
> about the 2nd almost makes my head explode, I have no idea what should be
> done with that one, or how it can be justified, the ${} case (since it is
> in a here doc, is already quoted, and hence the only defined quotes are
> after # ## % or %% operators - so that's what it must be referring to).
Plausible, perhaps likely. But it's not clear. This wouldn't be the first
time the standard relied on readers drawing an inference that's not
explicit.
> For a long time I was hoping that by making what happens with quotes inside
> a var expansion (the substring forms excepted) unspecified, and hence not
> something that applications can rely upon, would mean that eventually (perhaps
> way into the future) shells would be able to do things properly, rather than
> all follow the original shell because scripts break otherwise(the reason
> apparently that Korn switched ksh back from sane) when no-one is relying
> upon stuff like
> "${var:-"unquoted-text"}"
> any more.
I don't think you can rely on this right now. The only thing the currently-
published version of the standard dares to say about that is "an even
number of unescaped double quotes shall occur" and the "matching closing
brace shall be determined by ... skipping over enclosed quoted strings ...".
The last time we talked about the "even number" sentence, we all just
hand-waved it away by characterizing it as an "application requirement,"
even though there's no mention of "the application shall ensure" there.
The upcoming issue8 makes it explicitly unspecified, but the damage is
already done.
The situation with the "matching closing brace" text isn't any better.
The word "enclosed" is gone in the current draft, but "quoted string"
is still there, and we still have shells implementing the current
standard's language.
It's all just a mess, and the standard is avoiding saying anything definitive.
> I'm rambling Never mind.
:-)
> So that just leaves
> ${foo+"xyz"}
>
> Interpreting those double quote chars as quoting the xyz in a here doc,
> when, whatever they do, they definitely don't quote the xyz elswhere if
> (to emulate a here doc) this appears in a double quoted string
>
> "${foo+"xyz"}"
>
> whatever those "inner" quotes do (original Bourne shell had ${foo+ quoted,
> and the final }, and xyz unquoted), these days almost anything is possible,
> except for making xyz quoted (it would be without the inner double quotes).
The xyz is semi-quoted, because no matter what else the inner quotes might
do, they are supposed to prevent recognizing the closing brace. If it were
not `xyz' but `x}z', the inner quotes are supposed to `protect' the brace
there. The current version of the standard makes that clear, shells have
implemented it that way, and I'm sure there are scripts somewhere that rely
on it.
> Anyway, making the quotes when they appear in a here doc become quoting
> chars, when they don't when not in a here doc, when in a here doc double
> quote chars normally aren't quoting chars, would be simply bizarre.
>
> If POSIX needs even more attention to this, then I guess that's what we
> need to do (but we know that as soon as we do that, the result will be
> that the meaning will become unspecified...)
There's no meaning now! At least if it's explicitly unspecified we can at
some point -- maybe -- specify something. And who knows? The discussion
may prove illuminating.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
On 9/13/20 4:51 PM, Robert Elz wrote:
> | The specific construct is
> |
> | P=A
> | cat < | ${P+\"$P\"}
> | EOF
>
> That should output \"A\"
OK, let's discuss it.
>
> | In this case, the usual proscription on double quotes in here-documents
> | does not apply, since the double quote appears within ${}.
>
> Huh? Where does that come from, at best a " inside a quoted ${} is
> unspecified. But in a here doc, " is simply not a quoting char at all.
"However, the double-quote character ( '"' ) [edited, since the HTML on the
web site is malformed] shall not be treated specially within a here-
document, except when the double-quote appears within "$()", "``", or
"${}"."
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04
That implies that the double quote *is* treated specially when it appears
within a ${} expansion inside the body of a here-document, so the backslash
quotes it. The question is how special [insert eye-roll emoji], and what
rules you follow. It seems like shells follow the double-quoting rules if
they follow any.
(There's also whether or not the double quotes in the above text are
grammatical or syntactically significant.)
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
On 9/13/20 2:45 PM, Andreas Schwab wrote: > You have a regression here though: > > cat < \" > EOF Thanks, you're right. I'll take a look. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
Date:Sun, 13 Sep 2020 14:29:15 -0400 From:Chet Ramey Message-ID: | The specific construct is | | P=A | cat <
Re: Bash-5.1-beta available
You have a regression here though: cat <
Re: Bash-5.1-beta available
On 9/13/20 8:21 AM, Andreas Schwab wrote: > On Sep 10 2020, Chet Ramey wrote: > >> qqq. Fixed a bug that could cause backslashes quoting double quotes in here >> document bodies to not be removed when expanding the body. > > Are you sure about this? My reading of POSIX says that a backslash > before a double quote should not be removed, as double quotes are not > special in here docs. The specific construct is P=A cat
Re: Bash-5.1-beta available
Date:Sun, 13 Sep 2020 14:21:23 +0200 From:Andreas Schwab Message-ID: <87imchg94s@igel.home> | My reading of POSIX says that a backslash | before a double quote should not be removed, as double quotes are not | special in here docs. That would be a correct reading, double (and for that matter, single) qoutes in here doc contents are no different than any other char (like 'x' or space). That is, unless they are within some other construct that does have meaning, and where quoting is defined to work, like a $( ) command substitution, or the pattern part of one of the substring variable expansions. kre
Re: Bash-5.1-beta available
On Sep 10 2020, Chet Ramey wrote: > qqq. Fixed a bug that could cause backslashes quoting double quotes in here > document bodies to not be removed when expanding the body. Are you sure about this? My reading of POSIX says that a backslash before a double quote should not be removed, as double quotes are not special in here docs. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: Bash-5.1-beta available
On Sep 11 2020, Jesse Hathaway wrote: > On Fri, Sep 11, 2020 at 5:40 AM Andreas Schwab wrote: >> The reason for using a process substitution is so that the loop can set >> shell variables. > > Would using lastpipe be an option instead? No, the scripts are out there and will break. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
Re: Bash-5.1-beta available
On 9/11/20 6:39 AM, Andreas Schwab wrote: > On Sep 10 2020, Chet Ramey wrote: > >> yy. Process substitution processes now get their input from /dev/null, since >> they are asynchronous, not interactive, and not jobs. > > That breaks scripts that want to filter stdin with a process > substitution, eg: > > while read ...; do ...; done < <(filter) > > The reason for using a process substitution is so that the loop can set > shell variables. The problem is that asynchronous processes should not be able to change the terminal or terminal settings, and you don't want the process substitution and parent process to be trying to read from the terminal at the same time. Job control and process groups usually take care of that, but process substitution is not a job control process and runs in the same process group (and terminal process group) as the parent. Look at something like https://lists.gnu.org/archive/html/bug-bash/2019-09/msg00061.html Maybe it would be better to do this only if the shell is interactive and the standard input is a terminal. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Bash-5.1-beta available
11 Eylül 2020 Cuma tarihinde Andreas Schwab yazdı:
> On Sep 10 2020, Chet Ramey wrote:
>
> > yy. Process substitution processes now get their input from /dev/null,
> since
> > they are asynchronous, not interactive, and not jobs.
>
> That breaks scripts that want to filter stdin with a process
> substitution, eg:
>
> while read ...; do ...; done < <(filter)
>
> The reason for using a process substitution is so that the loop can set
> shell variables.
The loop can still set shell variables. Move the program that generates the
input for `filter' into the process substitution or if the script's stdin
is to be filtered just enable lastpipe.
set -o lastpipe
filter | while read ...; do ...; done
Allowing substituted processes to read from the shell's stdin doesn't make
any sense to me. The shell doesn't wait for those processes to complete
before reading the next command, so you can easily have two processes
racing to read from the same file handle if you allow substituted processes
to read from the shell's stdin. The command below for example, fills file1
and file2 with fragments of seq's output on bash 5.0.11.
seq 100 | { : <(cat >file1); cat >file2; }
>
> Andreas.
>
> --
> Andreas Schwab, sch...@linux-m68k.org
> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
> "And now for something completely different."
>
>
--
Oğuz
Re: Bash-5.1-beta available
On Fri, Sep 11, 2020 at 5:40 AM Andreas Schwab wrote: > The reason for using a process substitution is so that the loop can set > shell variables. Would using lastpipe be an option instead? lastpipe If set, and job control is not active, the shell runs the last command of a pipeline not executed in the back‐ ground in the current shell environment.
Re: Bash-5.1-beta available
On Sep 10 2020, Chet Ramey wrote: > yy. Process substitution processes now get their input from /dev/null, since > they are asynchronous, not interactive, and not jobs. That breaks scripts that want to filter stdin with a process substitution, eg: while read ...; do ...; done < <(filter) The reason for using a process substitution is so that the loop can set shell variables. Andreas. -- Andreas Schwab, sch...@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different."
